Emergence of Compositional Representations in Restricted Boltzmann Machines - PubMed (original) (raw)
Emergence of Compositional Representations in Restricted Boltzmann Machines
J Tubiana et al. Phys Rev Lett. 2017.
Abstract
Extracting automatically the complex set of features composing real high-dimensional data is crucial for achieving high performance in machine-learning tasks. Restricted Boltzmann machines (RBM) are empirically known to be efficient for this purpose, and to be able to generate distributed and graded representations of the data. We characterize the structural conditions (sparsity of the weights, low effective temperature, nonlinearities in the activation functions of hidden units, and adaptation of fields maintaining the activity in the visible layer) allowing RBM to operate in such a compositional phase. Evidence is provided by the replica analysis of an adequate statistical ensemble of random RBMs and by RBM trained on the handwritten digits data set MNIST.
Similar articles
- Learning Compositional Representations of Interacting Systems with Restricted Boltzmann Machines: Comparative Study of Lattice Proteins.
Tubiana J, Cocco S, Monasson R. Tubiana J, et al. Neural Comput. 2019 Aug;31(8):1671-1717. doi: 10.1162/neco_a_01210. Epub 2019 Jul 1. Neural Comput. 2019. PMID: 31260391 - Expected energy-based restricted Boltzmann machine for classification.
Elfwing S, Uchibe E, Doya K. Elfwing S, et al. Neural Netw. 2015 Apr;64:29-38. doi: 10.1016/j.neunet.2014.09.006. Epub 2014 Sep 28. Neural Netw. 2015. PMID: 25318375 - Where do features come from?
Hinton G. Hinton G. Cogn Sci. 2014 Aug;38(6):1078-101. doi: 10.1111/cogs.12049. Epub 2013 Jun 25. Cogn Sci. 2014. PMID: 23800216 - Thermodynamics of the Ising Model Encoded in Restricted Boltzmann Machines.
Gu J, Zhang K. Gu J, et al. Entropy (Basel). 2022 Nov 22;24(12):1701. doi: 10.3390/e24121701. Entropy (Basel). 2022. PMID: 36554106 Free PMC article. - Applications of Deep Learning in Biomedicine.
Mamoshina P, Vieira A, Putin E, Zhavoronkov A. Mamoshina P, et al. Mol Pharm. 2016 May 2;13(5):1445-54. doi: 10.1021/acs.molpharmaceut.5b00982. Epub 2016 Mar 29. Mol Pharm. 2016. PMID: 27007977 Review.
Cited by
- Probing T-cell response by sequence-based probabilistic modeling.
Bravi B, Balachandran VP, Greenbaum BD, Walczak AM, Mora T, Monasson R, Cocco S. Bravi B, et al. PLoS Comput Biol. 2021 Sep 2;17(9):e1009297. doi: 10.1371/journal.pcbi.1009297. eCollection 2021 Sep. PLoS Comput Biol. 2021. PMID: 34473697 Free PMC article. - RBM-MHC: A Semi-Supervised Machine-Learning Method for Sample-Specific Prediction of Antigen Presentation by HLA-I Alleles.
Bravi B, Tubiana J, Cocco S, Monasson R, Mora T, Walczak AM. Bravi B, et al. Cell Syst. 2021 Feb 17;12(2):195-202.e9. doi: 10.1016/j.cels.2020.11.005. Epub 2020 Dec 17. Cell Syst. 2021. PMID: 33338400 Free PMC article. - The recurrent temporal restricted Boltzmann machine captures neural assembly dynamics in whole-brain activity.
Quiroz Monnens S, Peters C, Hesselink LW, Smeets K, Englitz B. Quiroz Monnens S, et al. Elife. 2024 Nov 5;13:RP98489. doi: 10.7554/eLife.98489. Elife. 2024. PMID: 39499540 Free PMC article. - Generative and interpretable machine learning for aptamer design and analysis of in vitro sequence selection.
Di Gioacchino A, Procyk J, Molari M, Schreck JS, Zhou Y, Liu Y, Monasson R, Cocco S, Šulc P. Di Gioacchino A, et al. PLoS Comput Biol. 2022 Sep 29;18(9):e1010561. doi: 10.1371/journal.pcbi.1010561. eCollection 2022 Sep. PLoS Comput Biol. 2022. PMID: 36174101 Free PMC article. - Interpretable Machine Learning of Amino Acid Patterns in Proteins: A Statistical Ensemble Approach.
Braghetto A, Orlandini E, Baiesi M. Braghetto A, et al. J Chem Theory Comput. 2023 Sep 12;19(17):6011-6022. doi: 10.1021/acs.jctc.3c00383. Epub 2023 Aug 8. J Chem Theory Comput. 2023. PMID: 37552831 Free PMC article.
LinkOut - more resources
Full Text Sources
Other Literature Sources