Alternative Controllers
Sonic Banana (2000)
The Sonic Banana consists of four bend sensors in a row, running the length of a rubber tube, with a pushbutton switch at the end. Software in Max converts this data into musical functions, turning the Sonic Banana into a versatile performance instrument.
Arpeggiator: The bend sensors are used to play an arpeggiation algorithm in Max. The sensors control several parameters of the algorithm, including chord selection, tempo, note duration, tempo and volume.
Harmonica:
Flexing the Banana simulates blowing a harmonica. Flexing in one
direction plays one chord, while reversing direction plays another.
The button is used to change chord pairs. The Max patch uses physics
formula to simulate air flow, creating a pumping effect.
Wireless MIDI Glove (1999)
The Wireless MIDI Glove sends hand gesture information to a computer via MIDI. It has bend sensors, which send bend data for each of four fingers, and force sensing resistors, mounted on the fingertips, which send pressure data for each of four fingertips. Thus, the performer has eight streams of continuous control with which to play. The control streams are sent into Max, where a Max patch is used to create a variety of musical functions.
Air Drums: Each fingertip triggers a sound effect from one of four sets of
sounds. When the performer bends a finger, a new sound from the
corresponding set is selected for triggering. The performer is thus
able to play "air drums" with the sound effects. The performance
patch is implemented in Max and uses my AiffPlayer object for sample
playback.
Conducting: The performer uses his fingertips to "conduct" a music sequence. The
rate of tapping controls the tempo as well as the volume of the
sequence. Also, tapping on different fingers causes the sequence to
play with a different timbre or instrument. The performance patch is
implemented in Max and uses my SeqPlayer object for conducting.
Electronic Baton for Aria (1996)
I created an electronic baton using an Ascension Bird tracker for the Aria installation. I wrote Max software to read data from the sensor and analyze for beat tempo, beat amplitude and horizontal position of the baton. I used my Max external object, SeqPlayer, for conducting a MIDI sequence in time to the beats.
PhotoTheremin (1996)
I designed and built the PhotoTheremin, a MIDI controller with eight photosensors. The photosensors sense hand distance by detecting how much light is blocked by the player's hands. I used the PhotoTheremin to control a virtual Zydeco band by mapping a user's hand activity to musical parameters such as loudness, tempo, liveliness, etc.
Radio Baton Software (1992-94)
Created by Max Matthews, the Radio Baton is an electronic conducting surface with two batons, each generating X-Y-Z position data. As assistant to Richard Boulanger, I created Max software which extended the capabilities of the Radio Baton as a performance instrument (for example, The Dark Wind system). I also performed with Dr. Boulanger on Radio Baton in Europe and the United States.
PowerGlove Software (1992-94)
The Mattel PowerGlove was originally a controller for the Nintendo Entertainment System. By using a GoldBrick interface, one can connect the PowerGlove to a Macintosh computer and read glove data in Max. As assistant to Richard Boulanger, I created Max software which turned the PowerGlove into a musical instrument, controlling algorithmic improvisation. I also performed with Dr. Boulanger on PowerGlove in Europe and the United States.
Interactive Performance Systems
Max/MSP Programming (1993-present)
I have been using Max since 1993 as an integral part of my work
and for interactive performance
systems I have created for others. I have created and released numerous
Max external objects (see cyclops+max.)
to extend Max's capabilities. I have also been using
MSP since its release in 1998.
Max is a visual programming environment for creating music and multimedia applications on Macintosh computers. MSP is a set of extensions to Max for doing real-time synthesis and signal processing on PowerPC Macs.
Max Sax (1992 - present)
I programmed this system in Max for my own works for solo saxophone and effects. The system analyzes sax input via a pitch-to-MIDI converter and generates MIDI to control a digital effects processor which is processing the sax.
Marx System (1998-99)
I programmed this system in Max for
Richard Teitelbaum's works for solo pianist and two
Yamaha
Disklaviers (MIDI player pianos). The performer plays one Disklavier,
which send MIDI input to Max. The input is processed to generate MIDI
output, which plays the other Disklavier. Four independent processing
modules, each with numerous transformative and generative algorithms,
are available for configuration by the composer. These include inversion,
transposition, delay, echo, sampling and ornamentation. Processing modules
can be crosslinked to create chains and feedback loops for complex effects.
The Dark Wind System (1994)
I programmed this system in Max for Richard
Boulanger's piece The Dark Wind for
Radio
Baton and bass clarinet. The patch receives input from the Radio Baton,
foot pedals and the bass clarinet (via pitch-to-MIDI). It allows the Radio
Baton performer to control effects processing of the bass clarinet; grab
live samples of the bass clarinet and control algorithmic playback of
the samples; trigger sounds from a sampler, CD player and sound files;
and generate and control MIDI algorithmic generators.
Networked Multimedia Environments
Botanica Virtual (1995-96)
Botanica
Virtual was part of the NYU
Center for Advanced Technology (CAT) Spirits
of the Bayou installation created for SIGGRAPH '96 in collaboration
with the Laboratorio
de Sistemas Integraveis, University of San Paolo. Botanica Virtual
is a VR world set in a bayou, inhabited by spirits from the Voodoo religion.
Users navigating through the world meet and interact with artificially-intelligent
characters, animated using NYU CAT's software.
For this installation, I created Max software to control the audio environment experienced by the user. Running on two Macintosh computers, the software communicates over a network with the SGI computer running Improv and controlling the visual environment.
The Max software consists of three major components: Sound Map, which tracks the user and controls environmental sound and sound effects based on the user's location in the world; Actor Speech, which plays character voice responses and controls lip synching; and Voice Recognition, which interfaces with Apple voice recognition software to recognize responses from the user and and communicate them to Improv.
Botanica Virtual is described in the paper Improv: Interactive Improvisational Animation and Music.
Aria (1995-96)
Aria,
also part of the SIGGRAPH '96 Spirits of the Bayou installation,
was created by the Laboratorio de Sistemas Integraveis, University of
San Paolo in collaboration with the NYU CAT. In the Aria installation,
a user conducts a performance by an artificially-intelligent opera singer,
animated using NYU CAT's Improv software. An electronic baton allows the
user to conduct the opera while controlling tempo, volume and timbre of
the opera singer and accompanying orchestra.
For Aria, I created Max software to interpret data from a magnetic sensor in a baton and generate conducting information; control MIDI score playback; control singing synthesis running in Csound on an SGI; and send performance information to the Improv opera singer. The Max software communicated with Improv and Csound over a network.
Aria is described in the paper Improv: Interactive Improvisational Animation and Music.
A Flock of Words (1995)
A
Flock of Words is a multi-media piece for chamber ensemble with interactive
computer music, video, animation, lighting effects and holograms. The
piece was composed by Robert Rowe and the visual environment and holography
was created by Doris
Vila. As the piece is played, the ensemble's performance is tracked
and analyzed by computer. Analysis data is used to generate synthesized
accompaniment; control playback of video clips and real-time animation;
and control lighting and holographic effects. For the piece, I created
Macintosh software to network with the analysis computer; control and
send video playback to three projectors; and generate and control animation
of words using Craig Reynolds' Boids
flocking algorithm. I also designed and built servo motor controllers
for use in holographic lighting effects.
A Flock of Words is described in the paper Two Highly Integrated Real-Time Music and Graphics Performance Systems.
Interface Design
VSS GUI Software (1999-2000)
I created GUI-building tools and control software in Java for use with
VSS,
a software sound synthesizer. The software enables the VSS user to create
a graphical user interface to control synthesis parameters.
(VSS is a software sound synthesizer created by the Virtual Environments Group/Audio Development Group at University of Illinois, Urbana-Champaign.)
Virtual Musicians
Interactive Virtual Musicians Project (1996-97)
In
collaboration with Robert Rowe, I helped create software for the Interactive
Virtual Musicians project at the NYU
Center for Advanced Technology (CAT). This software controlled an
animated musician, created using NYU CAT's Improv software. Gregor, a
virtual sax player, listens to a jazz piano accompaniment and improvises
on saxophone. Gregor attunes his musical performance and his body language
based on analyzed attributes of the piano performance (for example, a
"hotter" piano performance yields a livelier musical and visual performance
by Gregor).
Interactive Virtual Musicians is described in the paper Improv: Interactive Improvisational Animation and Music.
Sonic Banana (2000)
The Sonic Banana consists of four bend sensors in a row, running the length of a rubber tube, with a pushbutton switch at the end. Software in Max converts this data into musical functions, turning the Sonic Banana into a versatile performance instrument.
Arpeggiator: The bend sensors are used to play an arpeggiation algorithm in Max. The sensors control several parameters of the algorithm, including chord selection, tempo, note duration, tempo and volume.
Harmonica:
Flexing the Banana simulates blowing a harmonica. Flexing in one
direction plays one chord, while reversing direction plays another.
The button is used to change chord pairs. The Max patch uses physics
formula to simulate air flow, creating a pumping effect.
Wireless MIDI Glove (1999)
The Wireless MIDI Glove sends hand gesture information to a computer via MIDI. It has bend sensors, which send bend data for each of four fingers, and force sensing resistors, mounted on the fingertips, which send pressure data for each of four fingertips. Thus, the performer has eight streams of continuous control with which to play. The control streams are sent into Max, where a Max patch is used to create a variety of musical functions.
Air Drums: Each fingertip triggers a sound effect from one of four sets of
sounds. When the performer bends a finger, a new sound from the
corresponding set is selected for triggering. The performer is thus
able to play "air drums" with the sound effects. The performance
patch is implemented in Max and uses my AiffPlayer object for sample
playback.
Conducting: The performer uses his fingertips to "conduct" a music sequence. The
rate of tapping controls the tempo as well as the volume of the
sequence. Also, tapping on different fingers causes the sequence to
play with a different timbre or instrument. The performance patch is
implemented in Max and uses my SeqPlayer object for conducting.
Electronic Baton for Aria (1996)I created an electronic baton using an Ascension Bird tracker for the Aria installation. I wrote Max software to read data from the sensor and analyze for beat tempo, beat amplitude and horizontal position of the baton. I used my Max external object, SeqPlayer, for conducting a MIDI sequence in time to the beats.
PhotoTheremin (1996)
I designed and built the PhotoTheremin, a MIDI controller with eight photosensors. The photosensors sense hand distance by detecting how much light is blocked by the player's hands. I used the PhotoTheremin to control a virtual Zydeco band by mapping a user's hand activity to musical parameters such as loudness, tempo, liveliness, etc.
Radio Baton Software (1992-94)
Created by Max Matthews, the Radio Baton is an electronic conducting surface with two batons, each generating X-Y-Z position data. As assistant to Richard Boulanger, I created Max software which extended the capabilities of the Radio Baton as a performance instrument (for example, The Dark Wind system). I also performed with Dr. Boulanger on Radio Baton in Europe and the United States.
PowerGlove Software (1992-94)
The Mattel PowerGlove was originally a controller for the Nintendo Entertainment System. By using a GoldBrick interface, one can connect the PowerGlove to a Macintosh computer and read glove data in Max. As assistant to Richard Boulanger, I created Max software which turned the PowerGlove into a musical instrument, controlling algorithmic improvisation. I also performed with Dr. Boulanger on PowerGlove in Europe and the United States.
Interactive Performance Systems
Max/MSP Programming (1993-present)
I have been using Max since 1993 as an integral part of my work
and for interactive performance
systems I have created for others. I have created and released numerous
Max external objects (see cyclops+max.)
to extend Max's capabilities. I have also been using
MSP since its release in 1998.Max is a visual programming environment for creating music and multimedia applications on Macintosh computers. MSP is a set of extensions to Max for doing real-time synthesis and signal processing on PowerPC Macs.
Max Sax (1992 - present)
I programmed this system in Max for my own works for solo saxophone and effects. The system analyzes sax input via a pitch-to-MIDI converter and generates MIDI to control a digital effects processor which is processing the sax.
Marx System (1998-99)
I programmed this system in Max for
Richard Teitelbaum's works for solo pianist and two
Yamaha
Disklaviers (MIDI player pianos). The performer plays one Disklavier,
which send MIDI input to Max. The input is processed to generate MIDI
output, which plays the other Disklavier. Four independent processing
modules, each with numerous transformative and generative algorithms,
are available for configuration by the composer. These include inversion,
transposition, delay, echo, sampling and ornamentation. Processing modules
can be crosslinked to create chains and feedback loops for complex effects.The Dark Wind System (1994)
I programmed this system in Max for Richard
Boulanger's piece The Dark Wind for
Radio
Baton and bass clarinet. The patch receives input from the Radio Baton,
foot pedals and the bass clarinet (via pitch-to-MIDI). It allows the Radio
Baton performer to control effects processing of the bass clarinet; grab
live samples of the bass clarinet and control algorithmic playback of
the samples; trigger sounds from a sampler, CD player and sound files;
and generate and control MIDI algorithmic generators.Networked Multimedia Environments
Botanica Virtual (1995-96)
Botanica
Virtual was part of the NYU
Center for Advanced Technology (CAT) Spirits
of the Bayou installation created for SIGGRAPH '96 in collaboration
with the Laboratorio
de Sistemas Integraveis, University of San Paolo. Botanica Virtual
is a VR world set in a bayou, inhabited by spirits from the Voodoo religion.
Users navigating through the world meet and interact with artificially-intelligent
characters, animated using NYU CAT's software.For this installation, I created Max software to control the audio environment experienced by the user. Running on two Macintosh computers, the software communicates over a network with the SGI computer running Improv and controlling the visual environment.
The Max software consists of three major components: Sound Map, which tracks the user and controls environmental sound and sound effects based on the user's location in the world; Actor Speech, which plays character voice responses and controls lip synching; and Voice Recognition, which interfaces with Apple voice recognition software to recognize responses from the user and and communicate them to Improv.
Botanica Virtual is described in the paper Improv: Interactive Improvisational Animation and Music.
Aria (1995-96)
Aria,
also part of the SIGGRAPH '96 Spirits of the Bayou installation,
was created by the Laboratorio de Sistemas Integraveis, University of
San Paolo in collaboration with the NYU CAT. In the Aria installation,
a user conducts a performance by an artificially-intelligent opera singer,
animated using NYU CAT's Improv software. An electronic baton allows the
user to conduct the opera while controlling tempo, volume and timbre of
the opera singer and accompanying orchestra.For Aria, I created Max software to interpret data from a magnetic sensor in a baton and generate conducting information; control MIDI score playback; control singing synthesis running in Csound on an SGI; and send performance information to the Improv opera singer. The Max software communicated with Improv and Csound over a network.
Aria is described in the paper Improv: Interactive Improvisational Animation and Music.
A Flock of Words (1995)
A
Flock of Words is a multi-media piece for chamber ensemble with interactive
computer music, video, animation, lighting effects and holograms. The
piece was composed by Robert Rowe and the visual environment and holography
was created by Doris
Vila. As the piece is played, the ensemble's performance is tracked
and analyzed by computer. Analysis data is used to generate synthesized
accompaniment; control playback of video clips and real-time animation;
and control lighting and holographic effects. For the piece, I created
Macintosh software to network with the analysis computer; control and
send video playback to three projectors; and generate and control animation
of words using Craig Reynolds' Boids
flocking algorithm. I also designed and built servo motor controllers
for use in holographic lighting effects.A Flock of Words is described in the paper Two Highly Integrated Real-Time Music and Graphics Performance Systems.
Interface Design
VSS GUI Software (1999-2000)
I created GUI-building tools and control software in Java for use with
VSS,
a software sound synthesizer. The software enables the VSS user to create
a graphical user interface to control synthesis parameters.(VSS is a software sound synthesizer created by the Virtual Environments Group/Audio Development Group at University of Illinois, Urbana-Champaign.)
Virtual Musicians
Interactive Virtual Musicians Project (1996-97)
In
collaboration with Robert Rowe, I helped create software for the Interactive
Virtual Musicians project at the NYU
Center for Advanced Technology (CAT). This software controlled an
animated musician, created using NYU CAT's Improv software. Gregor, a
virtual sax player, listens to a jazz piano accompaniment and improvises
on saxophone. Gregor attunes his musical performance and his body language
based on analyzed attributes of the piano performance (for example, a
"hotter" piano performance yields a livelier musical and visual performance
by Gregor).
Interactive Virtual Musicians is described in the paper Improv: Interactive Improvisational Animation and Music.