Gil Weinberg | Georgia Institute of Technology (original) (raw)

Papers by Gil Weinberg

Abstract A digital musical instrument, the Musical Playpen, was developed in an effort to provide... more Abstract A digital musical instrument, the Musical Playpen, was developed in an effort to provide children with immersive musical experiences. The instrument senses children's motion inside a playpen filled with plastic balls and maps their gestures to musical output. This allows the players to use familiar and fun play activities while exploring the musical consequences of their actions.

Abstract This paper describes the implementation of a leader-follower model in a musical HRI base... more Abstract This paper describes the implementation of a leader-follower model in a musical HRI based on beat detection analysis and a novel turn taking scheme. The project enables Haile, a robotic percussionist, to fluidly interact with humans in the context of an improvisatory jam session. The long-term goal of this work is to facilitate dynamic interactions between humans and machines that will lead to novel and inspiring musical outcomes.

Abstract The iltur system features a novel method of interaction between expert and novice musici... more Abstract The iltur system features a novel method of interaction between expert and novice musicians through a set of musical controllers called Beatbugs. Beatbug players can record live musical input from MIDI and acoustic instruments and respond by transforming the recorded material in real-time, creating motif-and-variation call-and-response routines on the fly.

Abstract Informed by a proposed theoretical framework for the field of interconnected musical net... more Abstract Informed by a proposed theoretical framework for the field of interconnected musical networks (Weinberg 2005), I describe a set of local musical networks that utilise novel gestural controllers for interdependent collaborative performance. The paper begins by contextualising developments in the field of musical networks in correlation with development of technological innovations, leading to the utilisations of gestural controllers in local musical networks.

Abstract We discuss our ongoing research in sonification of neural activity as demonstrated in th... more Abstract We discuss our ongoing research in sonification of neural activity as demonstrated in the" Fish and Chips" and" BrainWaves" projects. We argue that sonification can serve as an effective technique for the representation of complex spatial information such as neural activity due to the auditory system's ability to perceive stimuli at a wide spatial cover and its inclination to perceive spatial patterns in sonic input.

The author discusses research in music cognition and education indicating that novices and untrai... more The author discusses research in music cognition and education indicating that novices and untrained students perceive and learn music in a fundamentally different manner than do expert musicians. Based on these studies, he suggests implementing high-level musical percepts and constructionist learning schemes in new expressive musical instruments that would provide thoughtful and joyful musical activities for novices and experts alike.

Design A stable groundwork of relatively independent modules, capable of continued additions and ... more Design A stable groundwork of relatively independent modules, capable of continued additions and evolution, was our primary design goal. To this end, our design focused on the development of a simple yet robust algorithm. It is simple in that the design does not rely on a complex network of rules and conditions; it is robust in that the music produced by the algorithm should be capable of effectively representing a diversity of musical gestures.

Abstract The paper presents the theoretical background and the design scheme for a perceptual and... more Abstract The paper presents the theoretical background and the design scheme for a perceptual and improvisational robotic marimba player that interacts with human musicians in a visual and acoustic manner. Informed by an evaluation of a previously developed robotic percussionist, we present the extension of our work to melodic and harmonic realms with the design of a robotic player that listens to, analyzes and improvises pitch-based musical materials.

The Robotic Musicianship project aims to facilitate meaningful musical interactions between human... more The Robotic Musicianship project aims to facilitate meaningful musical interactions between humans and machines, leading to novel musical experiences and outcomes. The project combines computational modelling of music perception, interaction, and improvisation, with the capacity to produce acoustic responses in physical and visual manners.

Abstract This paper presents an interactive and improvisational jam session, including human play... more Abstract This paper presents an interactive and improvisational jam session, including human players and two robotic musicians. The project was developed in an effort to create novel and inspiring music through human-robot collaboration. The jam session incorporates Shimon, a newly-developed socially-interactive robotic marimba player, and Haile, a perceptual robotic percussionist developed in previous work.

ABSTRACT Meaning crossword of sound, Crossole is a musical metainstrument where the music is visu... more ABSTRACT Meaning crossword of sound, Crossole is a musical metainstrument where the music is visualized as a set of virtual blocks that resemble a crossword puzzle. In Crossole, the chord progressions are visually presented as a set of virtual blocks. With the aid of the Kinect sensing technology, a performer controls music by manipulating the crossword blocks using hand movements.

The paper describes an interactive musical system that utilizes a genetic algorithm in an effort ... more The paper describes an interactive musical system that utilizes a genetic algorithm in an effort to create inspiring collaborations between human musicians and an improvisatory robotic xylophone player. The robot is designed to respond to human input in an acoustic and visual manner, evolving a human-generated phrase population based on a similarity driven fitness function in real time.

Abstract In this paper, we connect local properties in a mobile planar multi-robot team to the ta... more Abstract In this paper, we connect local properties in a mobile planar multi-robot team to the task of creating decentralized real time algorithmic music. Using a nonlinear formation control law inspired by the consensus equation, we map the local motion parameters of robots to Euclidean rhythms with the use of sequencers. The control parameters allow a human user to direct this decentralized musical process by guiding and interfering with the robots' motion, which subsequently affects their musical activity.

ABSTRACT We present the current state of development of an interactive robotic percussionist, tit... more ABSTRACT We present the current state of development of an interactive robotic percussionist, titled Haile, which is designed to demonstrate musicianship. We define robotic musicianship in this context as a combination of low-and high-level music perception skills, symbolic generation and transformation of musical material, and the capacity to produce varied acoustic responses.

Abstract We present Shimon, an interactive improvisational robotic marimba player, developed for ... more Abstract We present Shimon, an interactive improvisational robotic marimba player, developed for research in Robotic Musicianship. The robot listens to a human musician and continuously adapts its improvisation and choreography, while playing simultaneously with the human. We discuss the robot's mechanism and motion-control, which uses physics simulation and animation principles to achieve both expressivity and safety.

ABSTRACT We conducted a rhythmic pattern learning and synchronization experiment. During the expe... more ABSTRACT We conducted a rhythmic pattern learning and synchronization experiment. During the experiment, each of 20 experiment subjects was learning 7 patterns in different level of difficulty from a drummer robot. They played all the patterns twice in 2 different visual conditions: being able to see, and not being able to see the robot's movement. 10 of the subjects could see the robot the first time they played the 7 patterns, and they then played the patterns the second time without seeing the robot.

Abstract ZooZBeat is a gesture-based mobile music studio. It is designed to provide users with ex... more Abstract ZooZBeat is a gesture-based mobile music studio. It is designed to provide users with expressive and creative access to music making on the go. ZooZBeat users shake the phone or tap the screen to enter notes. The result is quantized, mapped onto a musical scale, and looped. Users can then use tilt and shake movements to manipulate and share their creation in a group.

ABSTRACT A mixed media tool was created that promotes ensemble virtuosity through tight coordinat... more ABSTRACT A mixed media tool was created that promotes ensemble virtuosity through tight coordination and interdepence in musical performance. Two different types of performers interact with a virtual space using Wii remote and tangible interfaces using the reacTIVision toolkit [11]. One group of performers uses a tangible tabletop interface to place and move sound objects in a virtual environment. The sound objects are represented by visual avatars and have audio samples associated with them.

This extended abstract describes Drawn Together, an interactive art installation in which a perso... more This extended abstract describes Drawn Together, an interactive art installation in which a person takes turns drawing with a computer. We describe the process of the interaction and the methods used to creatively sonify the process and the animations. There are three main states in the interactive process that are sonically represented using audio samples in a mix of background and foreground sounds. The lines drawn by the computer are sonified using a set of features describing length, rate of time drawn, location, and curviness.

(cont.) In order to addressee the latter challenge I have decided to employ the digital network--... more (cont.) In order to addressee the latter challenge I have decided to employ the digital network--a promising candidate for bringing a unique added value to the musical experience of collaborative group playing.

Abstract A digital musical instrument, the Musical Playpen, was developed in an effort to provide... more Abstract A digital musical instrument, the Musical Playpen, was developed in an effort to provide children with immersive musical experiences. The instrument senses children's motion inside a playpen filled with plastic balls and maps their gestures to musical output. This allows the players to use familiar and fun play activities while exploring the musical consequences of their actions.

Abstract This paper describes the implementation of a leader-follower model in a musical HRI base... more Abstract This paper describes the implementation of a leader-follower model in a musical HRI based on beat detection analysis and a novel turn taking scheme. The project enables Haile, a robotic percussionist, to fluidly interact with humans in the context of an improvisatory jam session. The long-term goal of this work is to facilitate dynamic interactions between humans and machines that will lead to novel and inspiring musical outcomes.

Abstract The iltur system features a novel method of interaction between expert and novice musici... more Abstract The iltur system features a novel method of interaction between expert and novice musicians through a set of musical controllers called Beatbugs. Beatbug players can record live musical input from MIDI and acoustic instruments and respond by transforming the recorded material in real-time, creating motif-and-variation call-and-response routines on the fly.

Abstract Informed by a proposed theoretical framework for the field of interconnected musical net... more Abstract Informed by a proposed theoretical framework for the field of interconnected musical networks (Weinberg 2005), I describe a set of local musical networks that utilise novel gestural controllers for interdependent collaborative performance. The paper begins by contextualising developments in the field of musical networks in correlation with development of technological innovations, leading to the utilisations of gestural controllers in local musical networks.

Abstract We discuss our ongoing research in sonification of neural activity as demonstrated in th... more Abstract We discuss our ongoing research in sonification of neural activity as demonstrated in the" Fish and Chips" and" BrainWaves" projects. We argue that sonification can serve as an effective technique for the representation of complex spatial information such as neural activity due to the auditory system's ability to perceive stimuli at a wide spatial cover and its inclination to perceive spatial patterns in sonic input.

The author discusses research in music cognition and education indicating that novices and untrai... more The author discusses research in music cognition and education indicating that novices and untrained students perceive and learn music in a fundamentally different manner than do expert musicians. Based on these studies, he suggests implementing high-level musical percepts and constructionist learning schemes in new expressive musical instruments that would provide thoughtful and joyful musical activities for novices and experts alike.

Design A stable groundwork of relatively independent modules, capable of continued additions and ... more Design A stable groundwork of relatively independent modules, capable of continued additions and evolution, was our primary design goal. To this end, our design focused on the development of a simple yet robust algorithm. It is simple in that the design does not rely on a complex network of rules and conditions; it is robust in that the music produced by the algorithm should be capable of effectively representing a diversity of musical gestures.

Abstract The paper presents the theoretical background and the design scheme for a perceptual and... more Abstract The paper presents the theoretical background and the design scheme for a perceptual and improvisational robotic marimba player that interacts with human musicians in a visual and acoustic manner. Informed by an evaluation of a previously developed robotic percussionist, we present the extension of our work to melodic and harmonic realms with the design of a robotic player that listens to, analyzes and improvises pitch-based musical materials.

The Robotic Musicianship project aims to facilitate meaningful musical interactions between human... more The Robotic Musicianship project aims to facilitate meaningful musical interactions between humans and machines, leading to novel musical experiences and outcomes. The project combines computational modelling of music perception, interaction, and improvisation, with the capacity to produce acoustic responses in physical and visual manners.

Abstract This paper presents an interactive and improvisational jam session, including human play... more Abstract This paper presents an interactive and improvisational jam session, including human players and two robotic musicians. The project was developed in an effort to create novel and inspiring music through human-robot collaboration. The jam session incorporates Shimon, a newly-developed socially-interactive robotic marimba player, and Haile, a perceptual robotic percussionist developed in previous work.

ABSTRACT Meaning crossword of sound, Crossole is a musical metainstrument where the music is visu... more ABSTRACT Meaning crossword of sound, Crossole is a musical metainstrument where the music is visualized as a set of virtual blocks that resemble a crossword puzzle. In Crossole, the chord progressions are visually presented as a set of virtual blocks. With the aid of the Kinect sensing technology, a performer controls music by manipulating the crossword blocks using hand movements.

The paper describes an interactive musical system that utilizes a genetic algorithm in an effort ... more The paper describes an interactive musical system that utilizes a genetic algorithm in an effort to create inspiring collaborations between human musicians and an improvisatory robotic xylophone player. The robot is designed to respond to human input in an acoustic and visual manner, evolving a human-generated phrase population based on a similarity driven fitness function in real time.

Abstract In this paper, we connect local properties in a mobile planar multi-robot team to the ta... more Abstract In this paper, we connect local properties in a mobile planar multi-robot team to the task of creating decentralized real time algorithmic music. Using a nonlinear formation control law inspired by the consensus equation, we map the local motion parameters of robots to Euclidean rhythms with the use of sequencers. The control parameters allow a human user to direct this decentralized musical process by guiding and interfering with the robots' motion, which subsequently affects their musical activity.

ABSTRACT We present the current state of development of an interactive robotic percussionist, tit... more ABSTRACT We present the current state of development of an interactive robotic percussionist, titled Haile, which is designed to demonstrate musicianship. We define robotic musicianship in this context as a combination of low-and high-level music perception skills, symbolic generation and transformation of musical material, and the capacity to produce varied acoustic responses.

Abstract We present Shimon, an interactive improvisational robotic marimba player, developed for ... more Abstract We present Shimon, an interactive improvisational robotic marimba player, developed for research in Robotic Musicianship. The robot listens to a human musician and continuously adapts its improvisation and choreography, while playing simultaneously with the human. We discuss the robot's mechanism and motion-control, which uses physics simulation and animation principles to achieve both expressivity and safety.

ABSTRACT We conducted a rhythmic pattern learning and synchronization experiment. During the expe... more ABSTRACT We conducted a rhythmic pattern learning and synchronization experiment. During the experiment, each of 20 experiment subjects was learning 7 patterns in different level of difficulty from a drummer robot. They played all the patterns twice in 2 different visual conditions: being able to see, and not being able to see the robot's movement. 10 of the subjects could see the robot the first time they played the 7 patterns, and they then played the patterns the second time without seeing the robot.

Abstract ZooZBeat is a gesture-based mobile music studio. It is designed to provide users with ex... more Abstract ZooZBeat is a gesture-based mobile music studio. It is designed to provide users with expressive and creative access to music making on the go. ZooZBeat users shake the phone or tap the screen to enter notes. The result is quantized, mapped onto a musical scale, and looped. Users can then use tilt and shake movements to manipulate and share their creation in a group.

ABSTRACT A mixed media tool was created that promotes ensemble virtuosity through tight coordinat... more ABSTRACT A mixed media tool was created that promotes ensemble virtuosity through tight coordination and interdepence in musical performance. Two different types of performers interact with a virtual space using Wii remote and tangible interfaces using the reacTIVision toolkit [11]. One group of performers uses a tangible tabletop interface to place and move sound objects in a virtual environment. The sound objects are represented by visual avatars and have audio samples associated with them.

This extended abstract describes Drawn Together, an interactive art installation in which a perso... more This extended abstract describes Drawn Together, an interactive art installation in which a person takes turns drawing with a computer. We describe the process of the interaction and the methods used to creatively sonify the process and the animations. There are three main states in the interactive process that are sonically represented using audio samples in a mix of background and foreground sounds. The lines drawn by the computer are sonified using a set of features describing length, rate of time drawn, location, and curviness.

(cont.) In order to addressee the latter challenge I have decided to employ the digital network--... more (cont.) In order to addressee the latter challenge I have decided to employ the digital network--a promising candidate for bringing a unique added value to the musical experience of collaborative group playing.