Algorithms for Fitting the Constrained Lasso - PubMed (original) (raw)
Algorithms for Fitting the Constrained Lasso
Brian R Gaines et al. J Comput Graph Stat. 2018.
Abstract
We compare alternative computing strategies for solving the constrained lasso problem. As its name suggests, the constrained lasso extends the widely-used lasso to handle linear constraints, which allow the user to incorporate prior information into the model. In addition to quadratic programming, we employ the alternating direction method of multipliers (ADMM) and also derive an efficient solution path algorithm. Through both simulations and benchmark data examples, we compare the different algorithms and provide practical recommendations in terms of efficiency and accuracy for various sizes of data. We also show that, for an arbitrary penalty matrix, the generalized lasso can be transformed to a constrained lasso, while the converse is not true. Thus, our methods can also be used for estimating a generalized lasso, which has wide-ranging applications. Code for implementing the algorithms is freely available in both the Matlab toolbox SparseReg and the Julia package ConstrainedLasso. Supplementary materials for this article are available online.
Keywords: Alternating direction method of multipliers; Convex optimization; Generalized lasso; Linear constraints; Penalized regression; Regularization path.
Figures
Figure 1:
Isotonic regression fit shows a monotone trend in temperature abnormalities. The constrained lasso solution at ρ = 0 is identical to isotonic regression.
Figure 2:
Different simulation settings show consistently comparable or superior performance of the path algorithm whereas performances of ADMM and QP vary depending on the size and complexity of the problem. The runtimes for the solution path algorithm are averaged across the number of kinks in the path to make the runtimes more comparable to the other algorithms estimated at one value of the tuning parameter, ρ = _ρ_scale · _ρ_max.
Figure 3:
Generalized lasso and constrained lasso produce identical sparse fused lasso estimates on the brain tumor data
Figure 4:
Path algorithm yields the constrained lasso solution path for the Ames housing data with 48 factors. The dashed line marks the model with the lowest BIC (left panel) as well as the identity line (right panel). Here the response is the log-transformed sale price that is standardized to have mean 0 and standard deviation 1.
Similar articles
- A Path Algorithm for Constrained Estimation.
Zhou H, Lange K. Zhou H, et al. J Comput Graph Stat. 2013;22(2):261-283. doi: 10.1080/10618600.2012.681248. J Comput Graph Stat. 2013. PMID: 24039382 Free PMC article. - A New Algorithm and Theory for Penalized Regression-based Clustering.
Wu C, Kwon S, Shen X, Pan W. Wu C, et al. J Mach Learn Res. 2016;17:188. J Mach Learn Res. 2016. PMID: 31662706 Free PMC article. - [Generalized interaction LASSO based on alternating direction method of multipliers for liver disease classification].
Li J, Tong X, Wang J. Li J, et al. Sheng Wu Yi Xue Gong Cheng Xue Za Zhi. 2017 Jun 1;34(3):350-356. doi: 10.7507/1001-5515.201508026. Sheng Wu Yi Xue Gong Cheng Xue Za Zhi. 2017. PMID: 29745499 Free PMC article. Chinese. - Efficient methods for estimating constrained parameters with applications to lasso logistic regression.
Tian GL, Tang ML, Fang HB, Tan M. Tian GL, et al. Comput Stat Data Anal. 2008 Mar 15;52(7):3528-3542. doi: 10.1016/j.csda.2007.11.007. Comput Stat Data Anal. 2008. PMID: 18443660 Free PMC article. - Penalized and constrained LAD estimation in fixed and high dimension.
Wu X, Liang R, Yang H. Wu X, et al. Stat Pap (Berl). 2022;63(1):53-95. doi: 10.1007/s00362-021-01229-0. Epub 2021 Mar 31. Stat Pap (Berl). 2022. PMID: 33814727 Free PMC article.
Cited by
- A PET/CT radiomics model for predicting distant metastasis in early-stage non-small cell lung cancer patients treated with stereotactic body radiotherapy: a multicentric study.
Yu L, Zhang Z, Yi H, Wang J, Li J, Wang X, Bai H, Ge H, Zheng X, Ni J, Qi H, Guan Y, Xu W, Zhu Z, Xing L, Dekker A, Wee L, Traverso A, Ye Z, Yuan Z. Yu L, et al. Radiat Oncol. 2024 Jan 22;19(1):10. doi: 10.1186/s13014-024-02402-z. Radiat Oncol. 2024. PMID: 38254106 Free PMC article. - Trajectories of systolic blood pressure decline in kidney transplant donors prior to circulatory death and delayed graft function.
Lin Y, Teixeira-Pinto A, Craig JC, Opdam H, Chapman JC, Pleass H, Carter A, Rogers NM, Davies CE, McDonald S, Yang J, Lim WH, Wong G. Lin Y, et al. Clin Kidney J. 2023 Mar 15;16(7):1170-1179. doi: 10.1093/ckj/sfad047. eCollection 2023 Jul. Clin Kidney J. 2023. PMID: 37398694 Free PMC article. - On sparse ensemble methods: An application to short-term predictions of the evolution of COVID-19.
Benítez-Peña S, Carrizosa E, Guerrero V, Jiménez-Gamero MD, Martín-Barragán B, Molero-Río C, Ramírez-Cobo P, Romero Morales D, Sillero-Denamiel MR. Benítez-Peña S, et al. Eur J Oper Res. 2021 Dec 1;295(2):648-663. doi: 10.1016/j.ejor.2021.04.016. Epub 2021 Apr 18. Eur J Oper Res. 2021. PMID: 36569384 Free PMC article. - Sign-based Shrinkage Based on an Asymmetric LASSO Penalty.
Kawaguchi ES, Darst BF, Wang K, Conti DV. Kawaguchi ES, et al. J Data Sci. 2021;19(3):429-449. doi: 10.6339/21-JDS1015. Epub 2021 Jun 2. J Data Sci. 2021. PMID: 35222618 Free PMC article. - Interactions Between Donor Age and 12-Month Estimated Glomerular Filtration Rate on Allograft and Patient Outcomes After Kidney Transplantation.
Lim WH, Ooi E, Pilmore HL, Johnson DW, McDonald SP, Clayton P, Hawley C, Mulley WR, Francis R, Collins MG, Jaques B, Larkins NG, Davies CE, Wyburn K, Chadban SJ, Wong G. Lim WH, et al. Transpl Int. 2022 Feb 7;35:10199. doi: 10.3389/ti.2022.10199. eCollection 2022. Transpl Int. 2022. PMID: 35185379 Free PMC article.
References
- Altenbuchinger M, Rehberg T, Zacharias H, Stämmler F, Dettmer K, Weber D, Hiergeist A, Gessner A, Holler E, Oefner PJ, et al. (2017), “Reference point insensitive molecular data analysis,” Bioinformatics, 33, 219–226. - PubMed
- Arnold TB and Tibshirani RJ (2014), genlasso: Path Algorithm for Generalized Lasso Problems, R package version 1.3.
- Arnold TB (2016), “Efficient Implementations of the Generalized Lasso Dual Path Algorithm,” Journal of Computational and Graphical Statistics, 25, 1–27.
- Boyd S, Parikh N, Chu E, Peleato B, and Eckstein J (2011), “Distributed Optimization and Statistical Learning via the Alternating Direction Method of Multipliers,” Foundations and Trends in Machine Learning, 3, 1–122.
- Bredel M, Bredel C, Juric D, Harsh GR, Vogel H, Recht LD, and Sikic BI (2005), “High-Resolution Genome-Wide Mapping of Genetic Alterations in Human Glial Brain Tumors,” Cancer Research, 65, 4088–4096. - PubMed
LinkOut - more resources
Full Text Sources
Other Literature Sources
Research Materials