Emergence of Compositional Representations in Restricted Boltzmann Machines - PubMed (original) (raw)
Emergence of Compositional Representations in Restricted Boltzmann Machines
J Tubiana et al. Phys Rev Lett. 2017.
Abstract
Extracting automatically the complex set of features composing real high-dimensional data is crucial for achieving high performance in machine-learning tasks. Restricted Boltzmann machines (RBM) are empirically known to be efficient for this purpose, and to be able to generate distributed and graded representations of the data. We characterize the structural conditions (sparsity of the weights, low effective temperature, nonlinearities in the activation functions of hidden units, and adaptation of fields maintaining the activity in the visible layer) allowing RBM to operate in such a compositional phase. Evidence is provided by the replica analysis of an adequate statistical ensemble of random RBMs and by RBM trained on the handwritten digits data set MNIST.
Similar articles
- Learning Compositional Representations of Interacting Systems with Restricted Boltzmann Machines: Comparative Study of Lattice Proteins.
Tubiana J, Cocco S, Monasson R. Tubiana J, et al. Neural Comput. 2019 Aug;31(8):1671-1717. doi: 10.1162/neco_a_01210. Epub 2019 Jul 1. Neural Comput. 2019. PMID: 31260391 - Expected energy-based restricted Boltzmann machine for classification.
Elfwing S, Uchibe E, Doya K. Elfwing S, et al. Neural Netw. 2015 Apr;64:29-38. doi: 10.1016/j.neunet.2014.09.006. Epub 2014 Sep 28. Neural Netw. 2015. PMID: 25318375 - Where do features come from?
Hinton G. Hinton G. Cogn Sci. 2014 Aug;38(6):1078-101. doi: 10.1111/cogs.12049. Epub 2013 Jun 25. Cogn Sci. 2014. PMID: 23800216 - Performance of a Computational Model of the Mammalian Olfactory System.
Benjaminsson S, Herman P, Lansner A. Benjaminsson S, et al. In: Persaud KC, Marco S, Gutiérrez-Gálvez A, editors. Neuromorphic Olfaction. Boca Raton (FL): CRC Press/Taylor & Francis; 2013. Chapter 6. In: Persaud KC, Marco S, Gutiérrez-Gálvez A, editors. Neuromorphic Olfaction. Boca Raton (FL): CRC Press/Taylor & Francis; 2013. Chapter 6. PMID: 26042330 Free Books & Documents. Review. - Applications of Deep Learning in Biomedicine.
Mamoshina P, Vieira A, Putin E, Zhavoronkov A. Mamoshina P, et al. Mol Pharm. 2016 May 2;13(5):1445-54. doi: 10.1021/acs.molpharmaceut.5b00982. Epub 2016 Mar 29. Mol Pharm. 2016. PMID: 27007977 Review.
Cited by
- The recurrent temporal restricted Boltzmann machine captures neural assembly dynamics in whole-brain activity.
Quiroz Monnens S, Peters C, Hesselink LW, Smeets K, Englitz B. Quiroz Monnens S, et al. Elife. 2024 Nov 5;13:RP98489. doi: 10.7554/eLife.98489. Elife. 2024. PMID: 39499540 Free PMC article. - Interpretable Machine Learning of Amino Acid Patterns in Proteins: A Statistical Ensemble Approach.
Braghetto A, Orlandini E, Baiesi M. Braghetto A, et al. J Chem Theory Comput. 2023 Sep 12;19(17):6011-6022. doi: 10.1021/acs.jctc.3c00383. Epub 2023 Aug 8. J Chem Theory Comput. 2023. PMID: 37552831 Free PMC article. - Funneling modulatory peptide design with generative models: Discovery and characterization of disruptors of calcineurin protein-protein interactions.
Tubiana J, Adriana-Lifshits L, Nissan M, Gabay M, Sher I, Sova M, Wolfson HJ, Gal M. Tubiana J, et al. PLoS Comput Biol. 2023 Feb 2;19(2):e1010874. doi: 10.1371/journal.pcbi.1010874. eCollection 2023 Feb. PLoS Comput Biol. 2023. PMID: 36730443 Free PMC article. - Generative and interpretable machine learning for aptamer design and analysis of in vitro sequence selection.
Di Gioacchino A, Procyk J, Molari M, Schreck JS, Zhou Y, Liu Y, Monasson R, Cocco S, Šulc P. Di Gioacchino A, et al. PLoS Comput Biol. 2022 Sep 29;18(9):e1010561. doi: 10.1371/journal.pcbi.1010561. eCollection 2022 Sep. PLoS Comput Biol. 2022. PMID: 36174101 Free PMC article. - Probing T-cell response by sequence-based probabilistic modeling.
Bravi B, Balachandran VP, Greenbaum BD, Walczak AM, Mora T, Monasson R, Cocco S. Bravi B, et al. PLoS Comput Biol. 2021 Sep 2;17(9):e1009297. doi: 10.1371/journal.pcbi.1009297. eCollection 2021 Sep. PLoS Comput Biol. 2021. PMID: 34473697 Free PMC article.
LinkOut - more resources
Full Text Sources
Other Literature Sources