Algorithms for Fitting the Constrained Lasso - PubMed (original) (raw)

Algorithms for Fitting the Constrained Lasso

Brian R Gaines et al. J Comput Graph Stat. 2018.

Abstract

We compare alternative computing strategies for solving the constrained lasso problem. As its name suggests, the constrained lasso extends the widely-used lasso to handle linear constraints, which allow the user to incorporate prior information into the model. In addition to quadratic programming, we employ the alternating direction method of multipliers (ADMM) and also derive an efficient solution path algorithm. Through both simulations and benchmark data examples, we compare the different algorithms and provide practical recommendations in terms of efficiency and accuracy for various sizes of data. We also show that, for an arbitrary penalty matrix, the generalized lasso can be transformed to a constrained lasso, while the converse is not true. Thus, our methods can also be used for estimating a generalized lasso, which has wide-ranging applications. Code for implementing the algorithms is freely available in both the Matlab toolbox SparseReg and the Julia package ConstrainedLasso. Supplementary materials for this article are available online.

Keywords: Alternating direction method of multipliers; Convex optimization; Generalized lasso; Linear constraints; Penalized regression; Regularization path.

PubMed Disclaimer

Figures

Figure 1:

Figure 1:

Isotonic regression fit shows a monotone trend in temperature abnormalities. The constrained lasso solution at ρ = 0 is identical to isotonic regression.

Figure 2:

Figure 2:

Different simulation settings show consistently comparable or superior performance of the path algorithm whereas performances of ADMM and QP vary depending on the size and complexity of the problem. The runtimes for the solution path algorithm are averaged across the number of kinks in the path to make the runtimes more comparable to the other algorithms estimated at one value of the tuning parameter, ρ = _ρ_scale · _ρ_max.

Figure 3:

Figure 3:

Generalized lasso and constrained lasso produce identical sparse fused lasso estimates on the brain tumor data

Figure 4:

Figure 4:

Path algorithm yields the constrained lasso solution path for the Ames housing data with 48 factors. The dashed line marks the model with the lowest BIC (left panel) as well as the identity line (right panel). Here the response is the log-transformed sale price that is standardized to have mean 0 and standard deviation 1.

Similar articles

Cited by

References

    1. Altenbuchinger M, Rehberg T, Zacharias H, Stämmler F, Dettmer K, Weber D, Hiergeist A, Gessner A, Holler E, Oefner PJ, et al. (2017), “Reference point insensitive molecular data analysis,” Bioinformatics, 33, 219–226. - PubMed
    1. Arnold TB and Tibshirani RJ (2014), genlasso: Path Algorithm for Generalized Lasso Problems, R package version 1.3.
    1. Arnold TB (2016), “Efficient Implementations of the Generalized Lasso Dual Path Algorithm,” Journal of Computational and Graphical Statistics, 25, 1–27.
    1. Boyd S, Parikh N, Chu E, Peleato B, and Eckstein J (2011), “Distributed Optimization and Statistical Learning via the Alternating Direction Method of Multipliers,” Foundations and Trends in Machine Learning, 3, 1–122.
    1. Bredel M, Bredel C, Juric D, Harsh GR, Vogel H, Recht LD, and Sikic BI (2005), “High-Resolution Genome-Wide Mapping of Genetic Alterations in Human Glial Brain Tumors,” Cancer Research, 65, 4088–4096. - PubMed

LinkOut - more resources