Virtual Rehearsal - Versailles (2019)
This project was one of the first virtual rehearsal studies I participated in. Conducted within the EVAA project, it explored how room acoustics influence musical interpretation. The focus was on historically informed baroque performance, comparing musicians' reactions in historical and modern concert spaces, both real and virtual.
Concept
The study compared performances recorded in two real rooms and in their virtual counterparts. By analysing timing, articulation, vibrato, dynamics, and other musical features, we investigated whether musicians adapted their playing to the acoustic environment.
Two questions motivated the work. First, do musicians react similarly in a real room and its virtual reproduction? Second, how strongly does the acoustic character of a room influence interpretation? Together, these questions address the ecological validity of virtual rehearsal environments.
Musicians rehearsing in Versailles and its virtual reproduction
Selected Rooms
Two contrasting venues were selected. The Salon des Nobles de la Reine in Versailles represents the type of space for which much of the studied repertoire was originally written. The Cité de la Musique auditorium in Paris, on the other hand, provides a modern concert environment with very different acoustical characteristics.
Salon des Nobles de la Reine, Versailles
Cité de la Musique, Paris
Musicians rehearsing in The Cité de la Musique auditorium and its virtual reproduction
Auralisation
The virtual acoustics were reproduced using real-time convolution with ambisonic room impulse responses simulated in CATT-Acoustic/TUCT. The acoustic models were calibrated using measurements carried out in the real rooms.
The direct sound component was removed from the simulated responses, as it was naturally produced by the musicians themselves. The reverberant field was rendered over a 32-loudspeaker array surrounding the performers after decoding from ambisonics.
Instrument position and orientation were tracked throughout the performance. This information was used to adapt the auralisation in real time according to the directivity of each instrument. Under the hood, the system relied on a set of pre-simulated acoustic responses corresponding to different source radiation patterns. A custom plugin developed by our team combined these responses after convolution to recreate the directional behaviour of the instrument within the virtual room.
CATT acoustic lobe model and directivity recomposition workflow
Visual Environment
The virtual rehearsals took place in MocapVR, the laboratory's immersive performance space.
We deliberately avoided head-mounted displays so that musicians could see their instruments, read scores, and interact naturally with one another. At the same time, we believed that visual context would help create rehearsal conditions closer to those of the real venues.
To achieve this, we built a small CAVE-like projection system composed of four projection surfaces surrounding the performers. A single projector illuminated the entire structure, while a custom rendering pipeline handled image warping and perspective correction.
CAD of the screen rig
Screens mounted on the final rig
CAD of the projection setup
Using an OptiTrack system, the rendered viewpoint continuously adapted to the tracked musician position, creating the illusion of looking through a window into the virtual venue. The system was based on a modified version of the UniCAVE Unity framework combined with projection techniques originally explored during the BlenderVR project.
UniCAVE proxy projection scene (foreground) and rendered scene (background)
Final adaptive visual rendering in MocapVR
Results
I only participated in the preliminary prototyping and pilot studies. For details on the final experimental protocol and results, see: Eley, N., Lavandier, C., Psychoyou, T., & Katz, B. F. (2024). Listener perception of changes in historically informed performance of solo baroque music due to room acoustics. Acta Acustica, 8, 6. (DOI).
Side Project
As part of the project, students from the “Patrimoine, Visualisation et Modélisation 3D” programme at Cergy Paris Université created a visual reconstruction of musicians performing in the Salon des Nobles.