Music & Peacebuilding

Sound Connects Us: Betweenness of Sonic Experience

• Nina Kraus • Season 4 • Episode 3

Share episode

In this two-part series with Dr. Nina Kraus we examine the neuroscience of our hearing brains, exploring how we make meaning from our sonic worlds. In episode 1, we look at the afferent and efferent journeys as our brains construct meaning from sonic experience. Examining reading, we understand how reading is powered by the strength of our recognition of frequency, harmonics, FM sweeps, and other ingredients. In Episode 2, we explore the impact of musical training and bilingual experience on comprehension, synchrony, abilities to hear sounds in noise, our belonging, and our empathetic capacities to respond to affect. The two-episode series concludes with an examination of the violence of noise and sound and resulting impacts on our health and wellbeing.

The Music & Peacebuilding Podcast is hosted by Kevin Shorner-Johnson at Elizabethtown College. Join our professional development network at www.musicpeacebuilding.com - thinking deeply we reclaim space for connection and care.

Nina Kraus:

In the same way as I embrace boundaries, or you know, embrace the crossing of boundaries. I embrace paradox because a complex system that's worth studying is going to have a holistic view. And it is also going to have individual ingredients and parts. You can have both of these things and it's not a paradox. Both things are true.

Kevin Shorner-Johnson:

You were listening to season four of the music and peacebuilding podcast, a podcast season focused on multifaceted textures of belonging. Our podcast explores intersections of peacebuilding, sacredness, community, creativity and imagination through research and story. Nina Krauss is a scientist, inventor and amateur musician who studies the biology of auditory learning. Through a series of innovative studies involving 1000s of research participants from birth to age 90. Her research has found that our lives in sound shape auditory processing for better or worse, having witnessed firsthand in single neurons and humans, how hearing can change the brain and our connections. Kraus places a premium on communicating the scientific rationale for activities that strengthen the hearing brain and our sonic world. The cornerstone of her research is the ambition to improve social communication. Her book, Of Sound Mind how our brain constructs a meaningful Sonic World communicates these principles in a narrative digestible to any interested reader. Of sound mind is Kraus his love letter to sound, how sound connects us, its biological impact on making us, us, and how it affects the world we live in. This is the first in a two part series. In part one, we explore boundary crossing between this the magic of the sound brain and the role of sound in reading. I wanted to open with a question because I hadn't heard somebody. I mean, I've heard some people ask a little bit about the breadth of your expertise. But I'm fascinated by your origin story. As a literature major. I'm fascinated by the breadth of expertise diversity that you bring together in brain vaults as far as the different types of people who come together. And then the breadth of topics that you work with just the book itself has a huge amount of breadth to it. So I guess my question is, you know, if you were to start to tell me the story of boundary crossing curiosity, and how that formed within you, and where that's led you to now?

Nina Kraus:

Yeah, well, I love thinking about that. But I got to where I am today, by not by thinking about it at all, I just, I just embody boundary crossing. And that has been a strength. I think of the work it has also been a great liability, which I'll explain in a minute. But, you know, I feel like I belong at the intersection. I belong at the crossroads. You know, I grew up both in Italy and in the United States. And I don't feel as though I can can't really pass for an Italian or an American, I really feel that I belong somewhere in that intersection of worlds. In terms of, you know, my interests? Well, you know, first of all I was I was really fortunate to grow up in a household that was loaded with sound, you know, I had parents who spoke multiple languages at home, my mom was a musician, my dad also played music, and I grew up speaking these languages, and one of the languages that I grew up speaking or having, learning is harmony. You know, how to sing harmony. And I learned it as a language, even today, it's just something I can do. I have no idea, really, about the harmonic structure. And it's just something that I can I can do, I can find where there needs to be a voice. And, and so so that's fun. But yeah, I went to, you know, to school to college, and I majored in comparative literature because I knew some languages and I like to read and, and then I discovered biology and really fell in love with that and fell in love with the idea of using biology to inform my interest in sound, which first was in language and then has really blossomed into music because music is just it's just the jackpot for Our our vast hearing brain, then in terms of the disciplines, so if you go to the homepage of our website, which I hope that your listeners will do, you will see that we have panels that show the different areas that we research and will research music, rhythm. Aging concussion hearing in noise, autism. Reading. Bilingualism, Yeah, yeah, bilingualism, and you might, you know, you might ask what what are they even doing at brainvolts, but it really falls under the umbrella of sound and the brain. And I've always had, often when I speak to an audience, it's an audience of specialists. So people who are interested in research and bilingualism, for example. And, you know, I've been told so many times, oh, well, you need to focus on one or the other of these different areas, but it just doesn't work for me, because you know, if you're interested in sound, sound encompasses everything we care about. And it encompasses all of these different these different areas. And I love biology, it's a really great way of thinking about sound because our Sonic selves you know our lives and sound really make us who we are, they shape us. They shape us who we are, as individuals, they shape us who we are as partners, they shape us who we are, as members of society. And I think that it's, it's really important to be thinking about all of these different dimensions. And again, you talk about boundaries, there's the the life in the lab. And then there is the life outside the ivory tower. And I really, I'm only interested in what I do. Not only interested but I wouldn't do what I do if I didn't see that there were huge social and political and educational and medical implications.

Kevin Shorner-Johnson:

Wh.. I'm fascinated in that book about the cry, especially in this interview today at the crossing intersections between learning about musicians, brains, bilingual brains, and then also reading, I was just fascinated by the intersections there.

Nina Kraus:

Yeah, yeah, the intersections are huge. In terms of music, it's just baffling to me how I think everyone recognizes how important it is for, for for kids to be literate. And, yet, there is so little appreciation for the fact that making music can help the brain develop what we need in order to read and to make sense of, of the written word. Because, of course, you know, before we learned to write and read, we learned to talk. And it was in sound, you know, we're really just sonifying it and also you think historically, you think about the fact that we have been communicating with each other as a species for hundreds of 1000s of years, through sound. And yet, it's only been what 3000 years that there's been any writing. Literature writing is very, very, very young. Very wise people like Socrates worried once we started writing, he was afraid that people would stop that our memories would be harmed which in fact they are. Because if you have to really remember everything that you know, memory and hearing are incredibly tightly linked. And then that's just something that again, we need to, to think about we need to think about for cognitive health.

Kevin Shorner-Johnson:

Sounds are a connective force between our thoughts, memories, emotions and relations. Upon her recommendation, I read about Iain McGilchrist's sense of betweenness. Writing of music McGilchrist contrasts parts and holes to examine the study of music, psychology and literary criticism. McGilchrist uses the language of betweenness to refer to the entangled importance of relationships between things he states quote, that is to say that betweenness is more important than the things we believe we discern within it." Kraus' book opens with a fascinating account of the afferent journey of sound from the eardrum through fluid that stimulates tiny cochlear hairs that are tuned to specific frequencies. And then to the roughly 30,000 auditory nerve fibers per year as sound as mapped as electrical signal from there, the midbrain inferior colliculus to the thalamus engages in a lightning fast series of firings to determine the location of sounds and the grouping of auditory objects. A journey of accumulating insight reaches the auditory cortex, where sounds are recognized and processed. However, this seemingly one way journey is highly dialogic in back and forth communications that identify importance and bring order to sonic experience. Kraus states quote, "It is precisely this interaction of downstream and upstream influences that allows learning to occur and sculpts our sound mind." Our experiences of a vibrational universe through complex biological interactions may be the very essence of betweenness. Following my fascination with the betweenness of hearing and sound, I asked Dr. Kraus to use one example, exploring how we differentiate da and ba. To illustrate the between this of pitch, timing, harmonics, FM sweeps and other ingredients of sound. I thought it'd be really important for our musician listeners just to just to understand the complexity of the hearing brain. And I wondered if you might tell us the story of like ba, da, and gah. And what's happening as we differentiate these I wasn't, I wasn't familiar with the idea of an FM sweep, for instance, for my training, so So maybe you could tell us about like, why it's so magical that our brain can distinguish between ba da and gah. Yeah.

Nina Kraus:

And if you just if we can just stay with ba or da, we call it the mighty da. If you look at the acoustics of just the sound, da, you know, you can see the waves, and you can decompose this da sound into its component parts. So what you find is, first of all, it has a fundamental frequency - ah. But the real tricky stuff is as you go from the consonant to the vowel, du - ah, and the du, has, first of all very, very precise timing that separates the du from the ah. But you'd have to get from the du to the ah. And you do that with an FM sweep. So an FM sweep is just a change a rapid, a very rapid happens over 40 milliseconds of rapid change in frequency over time as you go from this d, which is a very acoustically dense multiple frequencies into just pulling out certain frequencies that are going to change in a systematic way, in time, your FM sweep as you connect your du to your ah. And this is happening very, very quickly. So at the same time that you are, you've got this onset response, you have the fundamental frequency of the ah and all of the harmonics that are in the ah, as well. And then you have this FM sweep, connecting it together. These components all have temporal ingredients. So things are happening very fast, but just that FM sweep is happening fast and 40 milliseconds where the difference between a T and a D is just a difference of a few milliseconds because du has is voiced and t is unvoiced. And so what they call voice onset time, again, is just a matter of 25 milliseconds. So your brain has to very, very quickly figure out, you know, and interpret these timing differences. So we get we get really good at this.

Kevin Shorner-Johnson:

And the part that blew my mind was the idea too that maybe it was between ba and da that there's been kind of controlled experiments where they've changed the gap. And it's dichotomous the Like that fascinated me, I didn't realize like at one point, the brain automatically switches that from one syllable to the other.

Nina Kraus:

Yeah. So you can, you can create a continuum and acoustic continuum, either voice onset time from da to ta or from da to ba or ga, just by changing the different acoustic elements, but you can make it continuous. So the change from one, exemplar, one unit to the next is identical in time and in frequency. But your brain because you have learned and based on your language, and this is a beautiful example of the sound to meaning connections that you have learned that are important for your language. When you cross the boundary between da and gah, you just automatically hear it as one or the other. And so our ability to categorize it's called categorical perception is really strong. And it's based on the sound to meaning connections we make. And as we know, we're born, we are citizens of all the world's languages, and all the world's music. But based on the music we make, and the languages we speak, we make these sound to meaning connections that then make us who we are, and make us able to hear these category boundaries, which is really, really useful because it helps us make sense of sound that is relevant to the language and the conversation you're having at the moment.

Kevin Shorner-Johnson:

As citizens in sound, we need to remember that our capacities of reading are fundamentally rooted in sound. Kraus notes that timing, harmonics, FM sweeps and other Sonic sensitivities are developed early in the sound minds of children. Indeed, reading problems at age eight might be predicted by sound processing struggles at age three. When students develop the sound mind through music, language learning, or even sports, they may strengthen skills of reading. What I think is powerful about what you're opening about reading is that if we start with the notion that reading is fundamentally about sound, then we asked the next question, which is, how do we develop the hearing brain and the sound brain? So would you take it from there about like, what's the evidence that shows us that reading is rooted in sound?

Nina Kraus:

I love it. So when, when we think about sound, and most people, most people really don't think about sound sound is under recognized because it's invisible. And we live in a very visually biased world. But if we think about sound, we realize that sound has many ingredients, just like a visual object, I'm holding this pen and it has a color, a size, a texture, shape. And so it has these ingredients. And if you think about sound, of course, sound also has ingredients that are very obvious to you and to the I love speaking with music instructors, because they of course, they know that there's pitch timing, timbre, phase, FM am all of these wonderful, rich ingredients that just occur even in a syllable. It's multiple elements at once. And we measure at brainvolts, we measure sound processing in the brain. And as I'm talking to you now, the neurons in your brain that respond to sound are producing electricity. And with scalp electrodes in humans, we can pick up that electricity and you know very carefully and amplify it many times but we can really get a sense of how you hear the world. And if you think about the metaphor of the brain as a mixing board, and all the different ingredients, the pitch, the timing, the timbre the FM, these are different faders on the mixing board. And so if I measure your brain's response to sound I can see what are your strengths and what are your bottlenecks in terms of the different processing of the ingredients. And we have discovered now by measuring sound processing in the brain, in 1000s of musicians from you know, birth to a very older age, and and we find that the particular ingredients that are especially strengthened are the harmonics and timing. And these are crucial ingredients. And obviously in music, the harmonics is what enables us to tell two instruments apart. So a piano, and a bassoon can be playing A flat. And you will hear the same note, but it will sound different because the harmonics are different. But the harmonics turns out to be also really important in distinguishing a D from a B that I say dad are bad. And, and timing, you know, where do things stop and start and all the timing elements in between. And the rhythm. Oh, my goodness, rhythm is you know, everybody knows, oh, yeah, of course, there's rhythm in music Well, there's so much rhythm in language. There's so much rhythm, you write this out in quarter notes and, and rests, and realize that these ingredients that are strengthened when you make music, and it's important that you make music, of course, singing counts, drumming counts, any kind of making music counts, and it will strengthen your brain's automatic processing of harmonic and timing cues. So you can be sound asleep, and I can be measuring your brain's response to sound and I will be able to see right away Oh, yeah, this is a guy who's made music. Of I can just see that because your brain has become based on your life in sound. Your brain has become automatically more sensitive and precise and accurate in coding harmonics and timing.

Kevin Shorner-Johnson:

Musical training may improve skills beyond music. Because musicians work within complex soundscapes to extract auditory information and apply skill to make responsive, creative decisions. In a 2015 longitudinal study, Slater and colleagues sought to understand if musical training might improve the ability to pick out a sentence amidst a noisy background. The ability to hear speech in a noisy soundscape is a complex auditory processing task that requires, quote, "The successful integration of cognitive, linguistic and sensory processing." This task is all the more difficult when one has bilingual superpowers. Because the individual must discriminate between enlarged sonic palettes amidst distractors. To study this skill, researchers worked with the Los Angeles harmony project, a music education program serving low income youth, over 90% of these youth identified as bilingual. After two years of training, these music students improved speech to noise threshold by 2.1 decibels, or an improvement of roughly 20 to 30% in ability. A comparison with a group that only had one year of musical training demonstrates the skill improved significantly at two years of training. How does our training and rich experiences like music, shape and mold the plasticity of our biological systems?

Nina Kraus:

Again, we can see in a child because we also can see developmentally that a child who is language delayed, has these bottlenecks, especially in harmonics, and timing, and rhythm. So so there is just as we think of it as a biological system. And what I love about these biological systems is that, you know, there's nothing that you can do if I put some electrodes on you and play some sounds to you, your brain is going to respond, doesn't matter how mad you are in the moment, it doesn't matter what's going on what we're most. I mean, yes, we're interested in some of the processing that happens in the moment, but we're especially interested in what have we become as a human being based on the languages we speak in the music we make, or the noise that we have had to endure? Or all of the various oral influences, sonic influences in our lives?

Kevin Shorner-Johnson:

I think that leads me to a question my afferent and efferent processing. So please correct me if I have a wrong understanding, but what I thought I took from the book was that maybe the methodology of using the FMRI in the past has kind of obscured the two directional way in which our hearing brain works. And you're introducing this idea of the frequency following response, because it has allowed you to really map the two way journey of the brain and am I going in the right direction? And can you talk about?

Nina Kraus:

Right? Yeah, because the FFR, frequency following responses, physiological response, we can capture a snapshot of the entire hearing brain, which goes upstairs and downstairs. So there's afferent, that's the ear to brain connection and the efferent, which is brain to ear. And, and by efferent, you know, I am including our hearing brain is vast. And again, people have, you know, hearing scientists often specialize just in the classic auditory pathway. But the auditory system, and the hearing brain has inputs, from our memory, from our feelings, from our, our guts, our movements, how we connect our other senses with sound, what we know, these are all important aspects of making sense of sound. And you can think of it as who, you know, you have this, this pathway, the automatic pathway that would respond while you're asleep to the sound of your name. But the efferent pathway, so this is carrying information from all over your brain, throughout the this vast auditory pathway network. And it changes and is informed by the sound to meaning connections, we make one of the most obvious sound to meaning connections we have learned is, we learn language. But as we make music, you're constantly making sound to meaning connections. So it's sound, the knowing it's the memory, it's the movement, it's the combining of the other senses, it's how you feel about all of these things. You know, as scientists, we love to deconstruct and parcel out try to find, you know, individual units, but life doesn't really always work that way. I mean, yes, it is helpful to look at units, but it cannot ever, ever forget the fact that the system really works as a whole. And that there are many things First of all, that there are limitations, not only in fMRI, but there are implementations in biology in general, there are limitations in the frequency following response. You know, there are that we need to acknowledge the limitations. But also, of course, learn from what we can measure, and from what we can understand biologically, because of the biology is, is really powerful to our understanding.

Kevin Shorner-Johnson:

And that's if we go back to boundary crossing, like that's why it's so important, as we open up more of a network theory of the brain and the mind, if that's why it's so much more important to see the whole, is what I understand.

Nina Kraus:

Yes, no, I think that is important, and to be seeing the whole end the parts together, you know, and I think that it's, again, in the same way as I embrace boundaries, or you know, embrace the crossing of boundaries. I embrace paradox, because a complex system that's worth studying is going to have a holistic view. And it is also going to have individual ingredients and parts you can have both, just because you know you, you have both of these things and it's not a paradox. Both things are true. These two things that seem as though they're opposing are both true.

Kevin Shorner-Johnson:

Iain McGilchrist notes the most profound truths hold paradox. profound truths hold a quote,"realization that what applies at the local level does not necessarily apply in the same way at the global level." A hearing brain is made up of parts, and it is also constructed of the network between a back and forth that organizes sonic experience. Join us in the next episode, as we examine the belonging of sound, synchrony and language, as well as an examination of violence and the harmful effects of noise. Dr. Nina Kraus' book, the sound mind is published by MIT Press. Her research lab named brainvolts can be found at brainvolts.northwestern.edu. Dr. Krauss invites you to consider donating to the work of her lab at brainvolts. She knows how much work her lab has yet to do, and the desire to advance this important work. This is the music and peacebuilding podcast hosted by Kevin Shorner-Johnson. At Elizabethtown college we host a Master of Music Education, with an emphasis in peacebuilding. thinking deeply we reclaim space for connection and care. Join us at music peacebuilding.com