Ben Smith | University of Illinois at Urbana-Champaign (original) (raw)
Papers by Ben Smith
Machine learning models are useful and attractive tools for the interactive computer musician, en... more Machine learning models are useful and attractive tools for the interactive computer musician, enabling a breadth of in- terfaces and instruments. With current consumer hardware it becomes possible to run advanced machine learning algo- rithms in demanding performance situations, yet expertise remains a prominent entry barrier for most would-be users. Currently available implementations predominantly employ supervised machine learning techniques, while the adaptive, self-organizing capabilities of unsupervised models are not generally available. We present a free, new toolbox of unsu- pervised machine learning algorithms implemented in Max 5 to support real-time interactive music and video, aimed at the non-expert computer artist.
Automated creativity, giving a machine the ability to origi- nate meaningful new concepts and ide... more Automated creativity, giving a machine the ability to origi- nate meaningful new concepts and ideas, is a significant challenge. Ma- chine learning models make advances in this direction but are typically limited to reproducing already known material. Self-motivated reinforce- ment learning models present new possibilities in computational creativ- ity, conceptually mimicking human learning to enable automated discov- ery of interesting or surprising patterns. This work describes a musical intrinsically motivated reinforcement learning model, built on adaptive resonance theory algorithms, towards the goal of producing humanly valuable creative music. The capabilities of the prototype system are ex- amined through a series of short, promising compositions, revealing an extreme sensitivity to feature selection and parameter settings, and the need for further development of hierarchical models.
Recent developments in machine listening present opportunities for innovative new paradigms for c... more Recent developments in machine listening present opportunities for innovative new paradigms for computer-human interaction. Voice recognition systems demonstrate a typical approach that conforms to event oriented control models. However, acoustic sound is continuous, and highly dimensional, presenting a rich medium for computer interaction. Unsupervised machine learning models present great potential for real-time machine listening and understanding of audio and sound data. We propose a method for harnessing unsupervised machine learning algorithms, Adaptive Resonance Theory specifically, in order to inform machine listening, build musical context information, and drive real-time interactive performance systems. We present the design and evaluation of this model leveraging the expertise of trained, improvising musicians.
Models of computational creativity promise to provide insight into the nature of human creative w... more Models of computational creativity promise to provide insight into the nature of human creative work and innovation. Intrinsically motivated, automated, creative agents present a potential avenue for the exploration of creativity in the arts, and in music in particular. A novel reinforcement learning agent designed to improvise music in a self-motivated fashion is described, formulated to prove the capabilities of an artificially creative musical system. The prototype employs unsupervised adaptive resonance theory algorithms to model theories of human perception, cognition, and creativity. While the generated results are constrained for initial evaluation further extensions suggest the potential to create meaningful, aesthetically valuable compositions.
The origins of computer music are closely tied to the development of the first high-performance c... more The origins of computer music are closely tied to the development of the first high-performance computers associated with major academic and research institutions. These institutions have continued to build extremely powerful computers, now containing thousands of CPUs with incredible processing power. Their precursors were typically designed to operate in non-real time, “batch” mode, and that tradition has remained a dominant paradigm for high performance computing. We describe experimental research in developing the interactive use of a modern high- performance machine, the Abe supercomputer at the National Center for Supercomputing Applications on the University of Illinois at Urbana-Champaign campus, for real-time musical and artistic purposes. We describe the requirements, development, problems, and observations from this project.
Supervised machine learning enables complex many-to-many mappings and control schemes needed in i... more Supervised machine learning enables complex many-to-many mappings and control schemes needed in interactive performance systems. One of the persistent problems in these applications is generating, identifying and choosing input output pairings for training. This poses problems of scope (limiting the realm of potential control inputs), effort (requiring significant pre-performance training time), and cog- nitive load (forcing the performer to learn and remember the control areas). We discuss the creation and implementation of an automatic “supervisor,” using unsupervised machine learning algorithms to train a supervised neural network on the fly. This hierarchical arrangement enables network creation and training in real time based on the musical or gestural control inputs employed in a performance, aiming at freeing the performer to operate in a creative, intuitive realm, making the machine control transparent and auto- matic. Three implementations of this self supervised model driven by iPod, iPad, and acoustic violin are described.
Urbana, Jan 1, 2009
Virtual worlds, 3D simulations of real or imagined worlds, are far richer and more dynamic than s... more Virtual worlds, 3D simulations of real or imagined worlds, are far richer and more dynamic than standard 2D computer applications.
Machine learning models are useful and attractive tools for the interactive computer musician, en... more Machine learning models are useful and attractive tools for the interactive computer musician, enabling a breadth of in- terfaces and instruments. With current consumer hardware it becomes possible to run advanced machine learning algo- rithms in demanding performance situations, yet expertise remains a prominent entry barrier for most would-be users. Currently available implementations predominantly employ supervised machine learning techniques, while the adaptive, self-organizing capabilities of unsupervised models are not generally available. We present a free, new toolbox of unsu- pervised machine learning algorithms implemented in Max 5 to support real-time interactive music and video, aimed at the non-expert computer artist.
Automated creativity, giving a machine the ability to origi- nate meaningful new concepts and ide... more Automated creativity, giving a machine the ability to origi- nate meaningful new concepts and ideas, is a significant challenge. Ma- chine learning models make advances in this direction but are typically limited to reproducing already known material. Self-motivated reinforce- ment learning models present new possibilities in computational creativ- ity, conceptually mimicking human learning to enable automated discov- ery of interesting or surprising patterns. This work describes a musical intrinsically motivated reinforcement learning model, built on adaptive resonance theory algorithms, towards the goal of producing humanly valuable creative music. The capabilities of the prototype system are ex- amined through a series of short, promising compositions, revealing an extreme sensitivity to feature selection and parameter settings, and the need for further development of hierarchical models.
Recent developments in machine listening present opportunities for innovative new paradigms for c... more Recent developments in machine listening present opportunities for innovative new paradigms for computer-human interaction. Voice recognition systems demonstrate a typical approach that conforms to event oriented control models. However, acoustic sound is continuous, and highly dimensional, presenting a rich medium for computer interaction. Unsupervised machine learning models present great potential for real-time machine listening and understanding of audio and sound data. We propose a method for harnessing unsupervised machine learning algorithms, Adaptive Resonance Theory specifically, in order to inform machine listening, build musical context information, and drive real-time interactive performance systems. We present the design and evaluation of this model leveraging the expertise of trained, improvising musicians.
Models of computational creativity promise to provide insight into the nature of human creative w... more Models of computational creativity promise to provide insight into the nature of human creative work and innovation. Intrinsically motivated, automated, creative agents present a potential avenue for the exploration of creativity in the arts, and in music in particular. A novel reinforcement learning agent designed to improvise music in a self-motivated fashion is described, formulated to prove the capabilities of an artificially creative musical system. The prototype employs unsupervised adaptive resonance theory algorithms to model theories of human perception, cognition, and creativity. While the generated results are constrained for initial evaluation further extensions suggest the potential to create meaningful, aesthetically valuable compositions.
The origins of computer music are closely tied to the development of the first high-performance c... more The origins of computer music are closely tied to the development of the first high-performance computers associated with major academic and research institutions. These institutions have continued to build extremely powerful computers, now containing thousands of CPUs with incredible processing power. Their precursors were typically designed to operate in non-real time, “batch” mode, and that tradition has remained a dominant paradigm for high performance computing. We describe experimental research in developing the interactive use of a modern high- performance machine, the Abe supercomputer at the National Center for Supercomputing Applications on the University of Illinois at Urbana-Champaign campus, for real-time musical and artistic purposes. We describe the requirements, development, problems, and observations from this project.
Supervised machine learning enables complex many-to-many mappings and control schemes needed in i... more Supervised machine learning enables complex many-to-many mappings and control schemes needed in interactive performance systems. One of the persistent problems in these applications is generating, identifying and choosing input output pairings for training. This poses problems of scope (limiting the realm of potential control inputs), effort (requiring significant pre-performance training time), and cog- nitive load (forcing the performer to learn and remember the control areas). We discuss the creation and implementation of an automatic “supervisor,” using unsupervised machine learning algorithms to train a supervised neural network on the fly. This hierarchical arrangement enables network creation and training in real time based on the musical or gestural control inputs employed in a performance, aiming at freeing the performer to operate in a creative, intuitive realm, making the machine control transparent and auto- matic. Three implementations of this self supervised model driven by iPod, iPad, and acoustic violin are described.
Urbana, Jan 1, 2009
Virtual worlds, 3D simulations of real or imagined worlds, are far richer and more dynamic than s... more Virtual worlds, 3D simulations of real or imagined worlds, are far richer and more dynamic than standard 2D computer applications.