VirtualBand is a music interaction system based on style modeling. The idea behind VirtualBand is to capture essential elements of the style of a musician – either solo or comping styles – and reuse these elements for musical improvisation.
VirtualBand proceeds in two steps: a recording and a playing phase. First, selected musicians are recorded using a specific procedure. Musicians are given playing instructions so that their recorded material covers a large range of musical situations, as well as transitions between them. These recording are then analysed using signal processing techniques (mostly borrowed from the domain of music information rsetrieval) to build a style database.
In the playing phase, the user (say, a sax player) first selects a song and a set of virtual musicians (for instance a virtual drummer, virtual guitarist and virtual bassist). When the user plays a music generator uses the style database (for each virtual musician) to produce music that fits with the song structure (harmony, metrics). Most importantly, VirtualBand listens to the soloist and analyzes a variety of factors such as volume, density of notes, pitch and reacts with logical and musical responses in real time. For instance, if the user plays louder, then the generator selects parts of the database that are themselves louder, or contain more notes, etc.
Reflexive VirtualBand
A development of the VirtualBand software is the Reflexive VirtualBand.
The idea behind Reflexive VirtualBand is to capture on the fly essential elements of the style of the musician playing with it and reuse these elements for real-time interaction. In this way the musician can build up a duo or even a trio, playing with virtual representations of her/himself and without having to previously “teach” their style to the system.
Check the videos below to see what VirtualBand can do with jazz and beyond.