Group sparse optimization by alternating direction method

By Wei Deng, Wotao Yin, Yin Zhang. Optics & Photonics - Optical Engineering + Applications, 2013

309 citations recorded by the source corpus.

This paper proposes efficient algorithms for group sparse optimization with mixed l2,1-regularization, which arises from the reconstruction of group sparse signals in compressive sensing, and the group Lasso problem in statistics and machine learning. It is known that encoding the group information in addition to sparsity can often lead to better signal recovery/feature selection. The l2,1-regularization promotes group sparsity, but the resulting problem, due to the mixed-norm structure and possible grouping irregularity, is considered more difficult to solve than the conventional l1-regularized problem. Our approach is based on a variable splitting strategy and the classic alternating direction method (ADM). Two algorithms are presented, one derived from the primal and the other from the dual of the l2,1-regularized problem. The convergence of the proposed algorithms is guaranteed by the existing ADM theory. General group configurations such as overlapping groups and incomplete covers can be easily handled by our approach. Computational results show that on random problems the proposed ADM algorithms exhibit good efficiency, and strong stability and robustness.

Subfields: Computational Mechanics, Signal Processing

View Paper: https://www.semanticscholar.org/paper/e09926adb42ae2046b24f6d6b3a2374930390c66

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computational Mechanics

In Signal Processing

Citations in the corpus

Less Is More: Towards Compact CNNs (2016, methodology): “According to [32], backward step in forward backward splitting method at k -th iteration now becomes: where τ is the learning rate and ˆ w k ∗ lj is the optimized neuron from the forward step in forward backward splitting.”
Locating and quantifying gas emission sources using remotely obtained concentration data (2012, background): “ariants have gained a lot of attention recently to solve other problems, especially in the elds of image processing, machine learning and compressed sensing (Yin et al. (2008); Yang and Zhang (2009); Deng and Zhang (2011)) where the objective function can be written as the sum of convex subproblems,”
On the Global and Linear Convergence of the Generalized Alternating Direction Method of Multipliers (2016, background): “We refer to [2,4,8,12,20,22,27,31,34,36] for a few examples of applications.”
Pruning and Quantization for Deep Neural Network Acceleration: A Survey (2021, background): “Sparse constraints [259] applied tensor low rank constraints [147] and group sparsity [54] achieving a 70% reduction of neurons with a 0.”