cu_abstract.tex

This paper focuses on  \emph{coordinate update methods}, which are useful for solving problems involving large or high-dimensional datasets. They decompose a problem into simple subproblems, where each  updates one, or a small block of, variables while fixing others. These methods can deal with linear and nonlinear mappings,  smooth and nonsmooth functions, as well as convex and nonconvex problems. In addition, they are easy to parallelize.  %Coordinate update  sits at a high level of abstraction and includes many special cases such as the Jacobi, Gauss-Seidel, alternated projection, as well as {coordinate descent} methods. They  have found greatly many applications throughout computational sciences.

The great performance of coordinate update methods depends on solving simple subproblems. To derive simple subproblems for several new classes of applications, this paper systematically studies  \emph{coordinate friendly} operators that perform low-cost coordinate updates. 

%In this paper, we abstract many problems to the fixed-point problem $x=\mathcal{T} x$ and study the favorable structures in the operator $\mathcal{T}$ such that the coordinate update $x_i^{k+1} = (\mathcal{T} x^k)_i$ can be performed at a relatively low cost. The selection of updating index can be sequential, parallel, or async-parallel. 

Based on the discovered coordinate friendly operators, as well as operator splitting techniques, we obtain new coordinate update algorithms for a variety of problems in machine learning, image processing, as well as sub-areas of optimization. Several problems are treated with coordinate update for the first time in history. The obtained algorithms are scalable to  large instances through parallel and even asynchronous computing. We present numerical examples to illustrate how effective these algorithms are.