Moments of Ma Mere l’Oye (original) (raw)

a realtime music visusalization for the LA Philharmonic developed by Ars Electronica Futurelab
in collaboration with ADMAF, the Abu Dhabi Music&Arts Foundation
participating artists (Roland Aigner, Florian Berger, Horst Hörtner, Gerfried Stocker)
performed live at the Walt Disney Music Hall, Feb. 2016

generative realtime graphic
14 individual graphical pieces have been developed for a performance of Ravels “Ma Mere l’Oye” under conductor Esa-Pekka Salonen.

virtual shadow play
In addition to these graphical pieces, the four interludes of Ma Mere l’Oye are visualized with virtual shadow plays. 7 kinect cameras are positioned among the orchestra and create virtual 3D shadows of the musicians and the conductor, which are then animated by the music and projected.

spatial multi screen installation for the Walt Disney Music Hall
7 large projections are being used to display the visual interpretations. A tryptic projection in the back of the orchestra, a duo-screen in the front and two single-screens on the side.

6 graphic-pcs are generating the visuals in realtime based on parameters derived from the live music with custom made sound analysing tools.

the performance at Ars Electronica Festival 2016 in Linz the production was also performed at the Ars Electronica Festival in Linz with the Bruckner Orchestra under Dennis Russell Davies.

The general concept for the visualization

In the special version for ballet, Ma Mere L’Oye is mainly the story of Sleeping Beauty, a princess which, as we all know, was to be killed by an evil fairy and only by last minute charms of a good fairy instead was sent into a 100 years sleep from which she only could be woken up by the kiss of a prince. That’s the story line which Maurice Ravel spanned in his ballet composition, from the prelude to the apotheosis. He considered the original fairy tale episodes from the piano version, as the dreams, given by the good fairy to the sleeping princess, in order to entertain her during her 100 years of sleep.
So, what would be more appropriate than using this metaphor of dream and deep sleep as the guiding line for the visualization. We take the fairy tale sequences as the REM phases of sleep, in which abstract visualization and associative graphic elements always give some references to the fairy tales. In the “Interludi”, which Ravel composed for the ballet and inserted between the original parts, we emerge from this deep sleep, yet without quite awakening, without fully conscious to come.

The deep dream sequences consist of computer-generated graphics, which are derived and formed in real time from the sound parameters of the orchestra. The prevailing method here is a sort of sound-paninting, based on an elaborate sound analysis and algorithms which bring the actual sound into visual form. In a certain way the orchestra directly moves the digitally created brushes and pens.

The visualizations of the Interludi, on the other hand, are derived from 3D cameras directed directly at the conductor and individual orchestral musicians. Their digital images are used to create non-realistic images, but schematic representations such as shadow fissures or light-drawn outlines in the dark, which correspond in their effect to this moment, in which reality mixes with the last traces of our dream experiences.

How the story and the music come alive in the visualizations

I. Prélude “Thickets of the Inevitable”

In the Prelude, the narrative strings are introduced with their musical motifs, and immediately after the first awakening and joyfully expectant mood, it is already clear that the story also has dark sides.
The inevitably unfolding destiny of sleeping beauty announces itself and this mood takes up the visualization of the first part.
With the first sounds of the orchestra “awake” also the Screens and in a gray / green dawn dawn pull the first fine strokes, which constrict themselves and to a scrub. Again and again, like creepy bird swarms, cascades of new lines emerge, the order of which is soon overlaid by violent orchestral tones. In the quiet moments of the prelude, this thicket comes to rest, and one can trace the development of the music in the layers and traces of the lines. In the middle triptych, the undergrowth now presents itself as a wave in the surf, which, like frozen, is waiting to collapse in a foamy manner.

The harsh contrasts and matted lines that dominate the picture, especially on the side screens, deliberately contrast the overly deceptive harmony of the music. And when at the end of the prelude the effervescence of the orchestra announces the arrival of the evil fairy, black spiderlike lines like a mushroom braid overgrow the whole picture.

II. Premier tableau – Danse du rouet et scène

In the second tableau three different visual motifs are being developed, each dealing with different aspects of the storyline:
“Gambol of the Fairies – En Arriere”
“Spinning Wheel and the Red Fairy”
“Frisking Spindles”

1. “Gambol of the Fairies – En Arriere”
Insect-like geometric creatures appear on the central projection tryptichon, hunt in a flare over the picture surfaces, pursue themselves, sometimes attack themselves, in order to prance the next moment coquettishly.

2. “Spinning Wheel and the Red Fairy”
On the front screens, the graphics resembles the rotating spindles. Tens of thousands of pixels form in rotation to amorphous nuclei and in their dark red color, the moment when sleeping beauty is stinging on the spindle is picked up.

3. “Frisking Spindles”
The side screens display very minimal graphics which follow in their movements mainly the melodies of the reed instruments.

III. Deuxième tableau – Pavane de la belle au bois dormant

1. “Quiescent Flow”
While in the first two tableaus the graphics are created out of the overall spectrum of the orchestral sound, in the Pavane the solo instruments come to the fore. At first only on the central projection surface. Fine lines are being drawn, following the sounds of the clarinet, out of which figures arise reminding us of archaic animal skeletons.

2. “Ariel and Oberon”
Following the figurines of the solo instruments, fleeting air ghosts emerge on the side as well as the front screens, gliding through the air in elegant twists and leaving fleeting traces. In the end they also take over the triptych and fadewith the last sounds of the pavanne.

IV. Interlude 1

After this first dream sequence, full of abstract associations, we emerge from the deep REM sleep to the surface but without really getting awake. The conductor and individual orchestral musicians become visible as vague and moving shadow. The real world penetrates our dream world without letting us wake up completely.

V. Troisième tableau – Les entretiens de la belle et de la bête

1. “Landscapes of Ambiguity”
The contrast of beauty and the beast is not interpreted as a discriminatory delimitation but as two opposite yet connected parts of the whole. To this end, a landscape of nature is created: mountains, hills, above it a sky, sometimes gently soft green hills and valleys, then once again sharply dangerous rocks and glaciers.
Several hundred thousand pixels are grouped into 21 brushes, which in real-time, only controlled by the sounds of the orchestra, create these landscapes.
Sound-painting, which has been repeatedly chosen to describe Ravel’s music, is implemented here in the literal sense with the help of the computer.

2. “Magic Mirror”
The eyes of a mysterious being, looking out of the darkness, look at us from the side and front screens.
Particle systems, colored spray mist, which condenses and thus forms and color gradients arise and pass away.

VI. Interlude 2

Following the course of the scenic sequence of Ravels Balette-suite, we emerge from the deep dreams and the 3D infrared cameras let the orchestra appear on the screens ….

VII. Quatrième tableau – Petit Poucet

1. “Dark Forrest”
The triptych is dominated by an abstracted forest of geometrical elements, similar to tree trunks, from whose tops the higher sound parts of the music always replace fragments, which, recalling the birds of the story, surround the tree crowns.
A gloomy oppressive mood arises when real photos of a nightly forest light appear in the background, mixed in color with torches. The flurring thicket of the geometric shapes always blocks the view of the clearing behind it.

2. “Kodamas”
At the beginning we only see fine lines on the side screens. As detached branches and branches against a white sky, they dance to the deep sounds of the orchestra. Into this expanse, the forest ghosts, wounded shaped bodies from thousands of individual leaves, which in the wind of the orchestral sounds weaken and flare up. They watch over the forest and observe the tragic story that develops in it, unmoved and unimpeded.

VIII. Interlude 3

After the sad disappearance of the little thumbling in the fairy tale forest, it is absolutely dark and only with the first sounds of the third interlude the reality of the concert hall again reaches the surface. In the fine, hand-drawn lines we see the live images of the musicians and the conductor, which oscillate from the darkness.

IX. Cinquième tableau – Laideronnette, impératrice des Pagodes

1. “Laideronettes Island”
Abstract in depiction, but quite concretely referring to the story behind this part, we see inhabitants of Laideronettes island in a joyous swinging dance in honor of their empress.

Sketched, landscape motifs of the fairy tale island emerge on the side screens, mountains grow out of the blue of the sea and imaginative flowers fly through the air, until the arrival of the Green Snake with the big gong is announced. The dancing couples solidify and shed and thorn-like elements penetrate the flowing green thicket of the side screens.

With the beginning of the flute motif, movement returns to the scene, the fear goes away, and the joyful dance is resumed.
From the landscape motifs of the side screens, a swarm of colorful birds breaks loose and the orchestra’s last sounds immerse the scenery into a threatening red.

X. Interlude 4

The deep red spreads out from the center over all projection surfaces. But a dream comes to an end and we return to reality for the last interlude. Only a thin silk curtain of fine light is interspersed, and this wraps the movements of conductor and orchestra, soothing the scenery, and letting us dive for the last time into the fairy world of ma Mere L’oye.

XI. Sixième tableau – Le jardin féerique

1. “Swaying in the Wind”
With the gentle beginning of the last part, we dive into a deep and formless dream world. Only the distant light, a sun surrounded by fog, receives us and movements of flowing form and color lie down over all the picture surfaces.
Behind them emerge dusky shrubs. In reference to the Dornenhain, which the prince must overcome in order to arrive at sleeping beauty.

2. “Marangonis Tears”
On the sidescreens colorful, uplifting elements celebrate the end of one hundred years of sleep and the union of prince and princess.

3. “Prince and Princess”
On the central screens, the main motif breaks with the idyll and moves the encounter of prince and princess from the distant fairy world into the beginning of the 20th century. Plays with original photos of the bold conquerors of the air, among many young women. The precursors of a new image of women, as well as precursors of modernism in the Ravel’s music.