Generative Visual Music | Fort Mason Center| San Francisco, CA| 2022 - Present
– (n.) a slight knowledge or vague notion
The source of human consciousness and creativity has remained mysterious for most of history. Through recent neuroscientific findings and cutting-edge technology, we are finally able to approach these questions empirically. When artists enter the creative flow state, their sense of time is no longer accurate, mental distractions disappear, and mini “lightbulb moments” rapidly fire while regulated by analytical evaluations. Incorporating scientifically sound concepts and concrete brainwave datasets, Inkling poetically captures this experience in an immersive multimedia work.
To investigate this change from one’s default state into the mystical creative state, neuroscientist Enrique Vargas recorded composer Jaco Wong's brain waves while he composed music, in the UCSF Sound & Music Perception Lab. The electroencephalography (EEG) machine works by placing electrodes on the composer’s scalp to pick up neural signals from various parts of his brain. A high-resolution 64-channel EEG system was used for the pre-recorded data. Animation artist Evan Tedlock used the data to enhance the major design elements, including controlling the scale of the patterns according to the oscillation of alpha power and the color saturation of the imagery. Musically, the aggregate brain rhythm is reflected by the density of musical rhythm and orchestration in various states. The concert venue itself resembles a brain as pre-recorded EEG readings and Independent Component Analysis inform where the orchestra musicians are placed spatially and when they are highlighted throughout the work. Mapping these components and their signal strength throughout the experiment allows the team to represent these unique data in a way that spans over time — and thus can be interpreted by the musicians as if it was musical notation.
The beginning section “Noise” portrays the chaotic and desynchronized nature of the default mode network and creates an uneasy textural experience that has no focal point. Enhanced by a liquid sound that invokes cerebrospinal fluid, the audience may experience an overload of sensory information. This transitions to “Motor”, a concerted effort to inhibit the prefrontal cortex and move towards an intentional creative flow state, takes its visual inspiration from neural networks with thin branches growing, connecting, and dissipating; in the background is manipulated MRI scan machine sound. Organic shapes contrasted by moments of geometric interventions slowly build up while musically employing the Risset rhythm, creating an infinitely slowing effect. Heading into “Creative Flow”, MRI scans served as a foundational visual inspiration with hints of brain fold shapes appearing in the dark areas to accompany the synchronized and bright orchestra hits augmented by high blinking electronic sounds. The imagery is symmetrical with a central object that pulses waves and metamorphosizes as ideas are generated, evaluated, and iterated upon. The “Meditation” state is the darkest in nature, expressing the feeling of loss as the ecstasy of the creative state fades. Dark hints of shapes and a haze of noisy texture appear slowly as the instruments explore their lowest range possible. An iteration of the original “Default” mode network returns, but now in a calmer and less chaotic form. High-frequency white noise gradually dominates as static takes over the image and fades to white.
Composer: Jaco Wong
Animation Artist: Evan Tedlock
Neuroscientist: Enrique Vargas
Immersive Technologist: Ana Carolina Estarita Guerrero
Music Technologist: Jay Cruz
Graphic Design: Tanya Chang
Documentation: Parsa Mirzaagha, Molly Murphy, David Tippie
Recording Engineer: Parsa Mirzaagha, David Tippie
- Inkling / Pictures at an Exhibition: Multimedia Orchestra Experience | Visual Music Performance | April 3, 2022 | Fort Mason Center| San Francisco, California
- March 28, 2022 | San Francisco Classical Voice | 'Exploring the Composer's Brain on Music with Inkling'