Inkling: A Multimedia Orchestra Experience is a unique intersection between orchestral music, abstract animation, and neuroscience. By using multi-projection, musicians surrounding the gallery space, and cutting-edge electroencephalography (EEG) technology, Inkling is an immersive representation of the neural processes during an artist’s creative state. 

​

Conductor/composer: Jaco Wong, San Francisco Conservatory of Music

Animation Artist: Evan Tedlock, University of Maryland, Baltimore County

Neuroscientist: Enrique Vargas, San Francisco Sound & Music Perception Lab

Immersive Technologist: Ana Carolina Estarita Guerrero

Music Technologist: Jay Cruz

Graphic Design: Tanya Chang

Documentation: Parsa Mirzaagha, Molly Murphy, David Tippie

Recording Engineer: Parsa Mirzaagha, David Tippie

​

The source of human consciousness and creativity has remained a mystery for much of history. Through recent neuroscientific findings and cutting-edge technology, we are finally able to approach these questions empirically. When artists enter the creative flow state, their sense of time is no longer accurate, mental distractions disappear, and mini “lightbulb moments” rapidly fire while regulated by analytical evaluations. Incorporating scientifically sound concepts and concrete brainwave datasets, Inkling poetically captures this experience in an immersive multimedia work.

​

To investigate this change from one’s default state into the mystical creative state, neuroscientist Enrique Vargas brought composer Jaco Wong into the UCSF Sound & Music Perception Lab to record his brain waves while composing. The electroencephalography (EEG) machine works by placing electrodes on the composer’s scalp to pick up neural signals from various parts of his brain. A high-resolution 64-channel EEG system was used to capture the pre-recorded data. Animation artist Evan Tedlock used this data to enhance the major design elements, including controlling the scale of the patterns according to the oscillation of alpha power and the color saturation of the imagery. Musically, the aggregate brain rhythm is reflected by the density of musical rhythm and orchestration in various states. The arrangement of the players resembles a brain as pre-recorded EEG readings and Independent Component Analysis inform where the orchestra musicians are placed spatially and when they are highlighted throughout the work. Mapping these components and their signal strength throughout the experiment allows the team to represent these unique data in a way that spans over time — and thus can be interpreted by the musicians as if it was musical notation.

 

The beginning section “Noise” portrays the chaotic and desynchronized nature of the default mode network and creates an uneasy textural experience that has no focal point. Enhanced by a liquid sound that invokes cerebrospinal fluid, the audience may experience an overload of sensory information. This transitions to “Motor”, a concerted effort to inhibit the prefrontal cortex and move towards an intentional creative flow state, takes its visual inspiration from neural networks with thin branches growing, connecting, and dissipating; the background audio is manipulated MRI scan machine sounds. Organic shapes contrasted by moments of geometric interventions slowly build up while musically employing the Risset rhythm, creating an infinitely slowing effect. Heading into “Creative Flow”, MRI scans served as a foundational visual inspiration with hints of brain fold shapes appearing in the dark areas to accompany the synchronized and bright orchestra hits augmented by high blinking electronic sounds. The imagery is symmetrical with a central object that pulses waves and metamorphosizes as ideas are generated, evaluated, and iterated upon. The “Meditation” state is the darkest in nature, expressing the feeling of loss as the ecstasy of the creative state fades. Dark hints of shapes and a haze of noisy texture appear slowly as the instruments explore their lowest range possible. An iteration of the original “Default” mode network returns, but now in a calmer and less chaotic form. High-frequency white noise gradually dominates as static takes over the image and fades to white.

​

Exhibitions

- INKLING / PICTURES AT AN EXHIBITION: Multimedia Orchestra Experience | 3 April 2022 | Fort Mason Center Gallery 308 | San Francisco | California

- INKLING: Sound Health Network Convening | 1 June 2022 | San Francisco Conservatory of Music | San Francisco | California

​

Press:

- March 28, 2022 | San Francisco Classical Voice | 'Exploring the Composer's Brain on Music with Inkling'

​