back Back

‘Movies’ with color and music visualize brain activity data

Feb. 21, 2024.
2 min. read Interactions

Immersive audio experience helps neuroscientists explore what's happening in a brain during an experiment

About the Writer

Amara Angelica

162.39737 MPXR

Amara Angelica is Mindplex Senior Editor

Visualization of neural activity from the dorsal surface of the thinned skull cortex of the awake mouse (credit: Thibodeaux et al.)

Neuroimaging produces large quantities of data that can be difficult to intuitively explore and gain insights into the biological mechanisms behind brain activity patterns. 

So David Thibodeaux and colleagues at Columbia University have now developed a toolkit to explore this data by translating it into a video, with accompanying musical sound track in real time. The goal is to help interpret what happens in the brain when the subject is performing certain behaviors.

How it works

The toolkit was applied to previously collected WFOM data. It detected both neural activity and brain blood flow changes in mice engaging in different behaviors, such as running or grooming.

Neuronal data is represented in the recording by piano sounds. The volume of each note indicates magnitude of activity, pitch indicates the location in the brain where the activity occurred, and blood flow data is represented by violin sounds. The relationship between neuronal activity and blood flow is represented by piano and violin sounds, played in real time.

Experiments

The researchers have demonstrated the new technique in three different experimental settings, showing how audiovisual representations can be prepared with data from various brain imaging approaches. These include “2D wide-field optical mapping (WFOM)” and “3D swept confocally aligned planar excitation (SCAPE) microscopy.”

They present this technique in the open-access journal PLOS ONE today (Feb. 21, 2024).

Immersive audio experience

“It is almost impossible to watch and focus on both the time-varying [brain activity] data and the behavior video at the same time. Our eyes would need to flick back and forth to see things that happen together.

“Listening to and seeing representations of [brain activity] data is an immersive experience that can tap into this capacity of ours to recognize and interpret patterns.”

Funding: National Institutes of Health grants, Columbia ROADS grant, and the Simons Collaboration on Global Brain.

Citation: Thibodeaux, D. N., Shaik, M. A., Kim, S. H., Voleti, V., Zhao, H. T., Benezra, S. E., Nwokeabia, C. J., & C. Hillman, E. M. (2024). Audiovisualization of real-time neuroimaging data. PLOS ONE, 19(2), e0297435. https://doi.org/10.1371/journal.pone.0297435

Let us know your thoughts! Sign up for a Mindplex account now, join our Telegram, or follow us on Twitter

Comment on this content

0 Comments

0 thoughts on “‘Movies’ with color and music visualize brain activity data

Like
Dislike
Share

0

Comments
Reactions
💯 💘 😍 🎉 👏
🟨 😴 😡 🤮 💩

Here is where you pick your favorite article of the month. An article that collected the highest number of picks is dubbed "People's Choice". Our editors have their pick, and so do you. Read some of our other articles before you decide and click this button; you can only select one article every month.

People's Choice
Bookmarks