Here is the final installment of our 3-part blog series coming from a conversation we had with author, sound designer, and lecturer David Sonnenschein about his latest project: 3 Deaf Mice. To learn more about 3 Deaf Mice visit http://www.3deafmice.com
3 Deaf Mice is based on David's book “Sound Design: The Expressive Power of Music, Voice and Sound Effects in Cinema”, which is #1 on Amazon and the definitive international resource on sound design for film. As founder of Sound Design for Pros, David has produced lots of great sounds, interviews and articles, and offers a 12-hour webinar series as part of the Kickstarter rewards. His experiences as audio consultant for EA's "Mass Effect 3", musician, filmmaker and neuroscientist, combine to bring this incredible sensory challenge and unique game play to everyone.
PSE: How do listening modes (source, shape and meaning) work in the real world? Doesn't our brain move among the listening modes very fluidly in our day-to-day life?
Yes, when we perceive sounds in the real world, the listening modes switch constantly and automatically in our brains. In the 3 Deaf Mice game we are creating various play stages where one specific listening mode can be focused upon at a time. This way our minds are able to understand this form of listening at a cognitive level, and our sensory mechanisms are able to perceive it and then create with it on a visceral level.
PSE: The third and final listening mode is semantic listening or listening for meaning. How is listening for meaning incorporated into the game play of 3 Deaf Mice?
At each stage of the game there will be different uses of the sound including its significance to the game play and the story. In the first stage of game play we'll want to understand the goals of the Mice, and where and why they're looking for the source of the sound. The meaning of that sound is going to have an important impact on which sounds the player will select. For example, we know that the Mice have set out to alleviate their stomach aches from the previous verse where they pigged out in the kitchen. Now when we hear sounds in the bathroom scene that have something to do with solving that problem, like finding an antacid relief, we're going to move towards that, and ultimately that will help us make decisions about which sounds we choose. That’s an example how we listen for meaning in terms of character goals and story elements.
PSE: What about the meaning in the music of Three Deaf Mice? How do we listen for meaning in music?
The song itself is a structure of sounds that has meaning beyond the individual elements. When we put together the lyrics, the instruments, and the sound effects into this song, it's going to be something that you're going to get as a gestalt or as a whole. Music in general could be defined as sounds with meaning in a structure that is recognizable by the listener. An isolated sound may have no meaning to us, just a random noise. However, that sound begins to have meaning when you add other sounds to it. In language this becomes syllables, words, sentences and ultimately communication. This is where we develop meaning through language. In music, we develop meaning by stringing notes together to create a melody, by stringing beats together to create a rhythm, by stringing different harmonic structures together to create a chord progression. These are the elements we've been listening to for centuries, and our brains are accustomed to finding meaning with traditional musical forms. However, if we listen to serial music which is a intellectual mathematical system where you must play all 12 notes of an octave before repeating any note, the organization is there but it has no meaning to our musically trained brains. It becomes quite difficult to listen to for any length of time because we don't absorb it as a musical entity and it's just too abstract. John Cage played around with a lot of these things to explore the meaning of sound and the definition of music. We obviously have found the popular meaning of music with pop music whether it's rock or hip hop, but when you move into jazz for example, the meaning of the musical phrases becomes much more complex. The listener needs to have a certain background and sophistication to appreciate jazz as much as popular music, and the same with classical music. Some Bach, for example, is very easy to listen to, whereas some of the more modern composers like Stravinsky may be much more difficult to listen to. When Stravinsky's Rite of Spring was first performed in 1907, it literally created riots because the music was so out of the ordinary and not structured like people had been used to. So this is what we're really exploring in the game. What are the components of meaning in sound, and how can they create music? What is music?
PSE: Are our interpretations of sound and music based more on cultural experiences and training or on physiological principles? For example, the sound of nails on a chalkboard is almost universally regarded as an unpleasant sound.
In the West, we are accustomed to listening to 12-tone scales like what we have on a piano, but in India they have 22-note scales. Their ears are sophisticated enough to hear those microtonal changes, whereas when we hear it, it may just sound like something is out of tune. So music itself has a certain level of training that we must have when we get to that level of sophistication. However, there are some fundamental principals that go beyond the cultural references that have to do with harmonic structure. The fact that an octave feels whole, right, simple, and grounded is not just founded on a cultural basis. It's because the actual frequencies of the octave and its fundamental physically and mathematically fit into each other, and when it hits our ears and our brains process it, we're going to feel that there's something very symmetric and balanced about it. So that in and of itself creates a kind of physical and emotional balance in our beings. You mentioned scratching on a chalkboard, and that's something more physiological because of the intense mixed frequencies in a high range that can actually irritate the hearing system. We can go into each sound that we hear and discuss it as potentially a cultural reference or a physiological stimulus, and as sound designers we want to become very aware of how to use each sound effectively for an audience. For example if you're watching children's TV, you may notice that the sounds are extremely simple and intentionally designed to be pleasant. Very few sounds will make you feel nervous or agitated. But you also need a certain amount of stimulus contrast and surprise. On the other extreme with a horror film, people go in there expecting to be scared. So you deliver to them what they want by attacking their nervous system with sound. I'm not saying just loud sounds, and sometimes it's just silence and then a shocking or weird sound to create a lot of anxiety.
PSE: Ultimately what skills do you want a player to come away with after playing 3 Deaf Mice?
Communication skills have to do with listening and projecting your ideas: it's a two way street. So first of all, mindfulness is extremely important in our educational system. This is becoming a key word for students to become better learners. Mindfulness means that they can absorb material and then create and present their own material when they become more conscious of their environment. We're becoming more aware of these different listening modes and how we hear sounds through their source, shape, and meaning. Of course when you begin to communicate and use sound effectively this way, your communication skills will obviously become stronger and the results of your communication will get you closer to your desired intention. This could apply to an academic presentation, a creative piece of music, or creating a game or a film.
PSE: How would you continue to develop 3 Deaf Mice? Any future plans for the game?
There's another goal that the 3 Deaf Mice game is aiming to accomplish, and that is developing community. Inside of this idea that we can listen and communicate with sound, we can also share our insights and our sounds with one another in very creative ways. This will become a platform in the future for people to share their sounds with one another in ways that are very expressive and creative. Specifically, the current game structure of the ten tracks and verses will be able to be unlocked and remixed, which should be lots of fun. In the future, we'll be able to allow people to share new sounds, new lyrics, and new pieces of audio clips that can then be remixed into the song structure. Ultimately, I'd like to see a game like this become an open source platform for other people to create their own songs that are very interactive. Just like Karaoke became a whole medium in music, this could become a whole new medium of creating songs and participating with other people in really fun, interactive, and dynamic ways. I see the game can also be re-purposed for specific groups in education, training and health, for example helping people with dyslexia, sight impairment or jobs requiring highly developed listening abilities like sonar operation. I’m excited to explore many new possibilities and open to unforeseen applications.
__________________________________
That's all folks! Be sure to follow the progress of 3 Deaf Mice at http://www.3deafmice.com
