Coordinate descent algorithms

By Stephen J. Wright. 2015

1,641 citations recorded by the source corpus.

Coordinate descent algorithms solve optimization problems by successively performing approximate minimization along coordinate directions or coordinate hyperplanes. They have been used in applications for many years, and their popularity continues to grow because of their usefulness in data analysis, machine learning, and other areas of current interest. This paper describes the fundamentals of the coordinate descent approach, together with variants and extensions and their convergence properties, mostly with reference to convex objectives. We pay particular attention to a certain problem structure that arises frequently in machine learning applications, showing that efficient implementations of accelerated coordinate descent algorithms are possible for problems of this type. We also present some parallel variants and discuss their convergence properties under several models of parallel execution.

Subfields: Artificial Intelligence, Computational Mechanics, Optimization & Training, Algorithms & Complexity

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

  1. Peter Richtárik
  2. Martin Jaggi
  3. Nicolas Loizou
  4. Jakub Konecný
  5. Sebastian U. Stich
  6. Michael I. Jordan
  7. Percy Liang
  8. Mohit Bansal

In Computational Mechanics

  1. Peter Richtárik
  2. Nicolas Loizou
  3. Jakub Konecný
  4. Martin Takác
  5. Martin Jaggi
  6. Sebastian U. Stich
  7. Michael I. Jordan
  8. Francis R. Bach

In Optimization & Training

  1. Peter Richtárik
  2. Jakub Konecný
  3. Nicolas Loizou
  4. Martin Jaggi
  5. Sebastian U. Stich
  6. Martin Takác
  7. Simon Lacoste-Julien
  8. Francis R. Bach

In Algorithms & Complexity

  1. Nicolas Loizou
  2. Aurélien Bellet
  3. Martin Jaggi
  4. Sebastian U. Stich
  5. Peter Richtárik
  6. Marc Tommasi
  7. Rachid Guerraoui
  8. Stephen P. Boyd

Citations in the corpus