Jonathan Bell’s piece Au Commencement was entirely composed (and delivered to the singers) with the help of the bach.roll space-time notation environment. The SmartVox project uses filmed sequences (mp4) of these new forms of audio-visual scores together with a local web server, so that each singer receives his part directly in the browser of his smartphone.

bach’s notational environment serves here both ends of the composer-performer dialogue:

1. For the composer, bach.roll is used as an experimental workspace. It turns traditional “pen and paper” notational sketches into an audio-visual animated notation. This offers a very different approach to time, and allows a great control over the musical material.

2. For the performers, this animated space-time notation profoundly simplifies temporal coordination between them, particularly in the case where music is more based on durations than rhythmic patterns. The performer can’t get lost, and no pages need to be turned. The audio-score (sent via an earpiece, individually to each player/singer) also frees the performer from the anxiety to have to realise something that would normally be too difficult to pitch. Therefore, such forms of augmented notation appear to be very practical for singers when they have long parts to learn, when they are asked to distance themselves from each other, or if the language requires singing in (microtonal) just intonation.

This kind of setup profoundly changes the usual relationship between composer, score and performer, and a lot remains to be experimented in this field.

1/ bach.roll and bach.score as workspaces for the composer, as symbolic DAWs

In these compositions, each note (or group of notes) was used here as a reservoir containing information about how to sculpt vocal lines, so as to combine them into a polyphony.

	1/ A sample of spoken text is stored inside the filename slot of each note (or group of notes) of this bach.roll.
	2/ cage.ezsampler, and psych~ then allow to retrieve all the information stored inside these notes, and “force-pitch” the corresponding sample of spoken text to the microtonal pitch displayed on the roll. (The Psych Acronym stands for : Pitch Synchronous Yin-based Choral Harmoniser. The psych~ module is still available in Max/MSP but Psychoirtrist~ is considered more completed. Psych~ and Psychoirtrist~ were implemented by Norbert Schnell at IRCAM.)
	3/ Different types of information about how this sample should be played are also stored as slot content (in this example: speed, glissando, and ‘respect’ or spectral envelope), and then retrieved with the help of the bach.keys object.

This simple setup allows an infinite variety of vocal polyphony mock-ups, with an almost-like-real result. This enables the composer to experiment for instance within the tiniest microtonal inflections, harmony portamento, vibrato, ornaments….

2/ Notation as a medium which conveys information to the performer

Technology is primarily used here as a means to “augment” traditional notation, with the help of audio-scores and screen-based animated notation (screen-scores). Bach was therefore used in my piece as a means to provide singers with audio-guides (through an earpiece) and screen-scores displayed on their smartphones.

The technical challenge here was to send, almost instantly, a large amount of data (a video of several Mo) to a large amount of participants (a choir), and all on different types of platforms (Androids, iOS, OSX, windows…). Evidently, the easiest way to do this today is to rely on web technologies, and this led to the development of the SmartVox web application (based on the COSIMA’s Soundworks framework, from ISMM research team at IRCAM). This application is based on a local server and allows to send individually to each performer’s devices (via the browser of their phone or tablet), videos and audio-guides, all composed in the Bach environment. The server acts here as a sort of conductor which sends the score to each participant, and synchronises all the scores together to a common clock.

In technological terms, two things can be observed here:
Data sending and communication all happen in the browser of the participant’s smartphones, in a web page (a single-page web application), so nothing needs to be installed on the phone, there is no app to download.
This setup forms a distributed system, where the processing is distributed across many machines (smartphones, tablets…), and interact with each other in order to achieve a common goal.
In addition to the notation generated in Bach, several difficulties needed to be tackled in terms of user interface, which led to different versions of the app: a flexible one for rehearsal and a more robust and automatised version for concert.
This web application does not rely on the internet when it is running in performance. Rather, it is based on a LAN (Local Area Network) build over WIFI.

So as to get an idea of the finality which lies behind all this set up, click here to watch a full performance of Au Commencement, the first piece realised with the SmartVox app, sung here by the Mangata ensemble.