Here is part 2 of our blog series coming from a conversation we had with author, sound designer, and lecturer David Sonnenschein about his latest project: 3 Deaf Mice. To learn more about 3 Deaf Mice visit the project Kickstarter page at http://www.3deafmice.com
3 Deaf Mice is based on David's book “Sound Design: The Expressive Power of Music, Voice and Sound Effects in Cinema”, which is #1 on Amazon and the definitive international resource on sound design for film. As founder of Sound Design for Pros, David has produced lots of great sounds, interviews and articles, and offers a 12-hour webinar series as part of the Kickstarter rewards. His experiences as audio consultant for EA's "Mass Effect 3", musician, filmmaker and neuroscientist, combine to bring this incredible sensory challenge and unique game play to everyone.
PSE: How do you integrate into Three Deaf Mice “causal” listening and identifying the source of the sound? Why is this type of listening important?
In our daily lives, we're surrounded by sounds that really impact our well-being. If we become aware of these sounds, we can actually live a fuller life in terms of our own health, safety and joy. For example, if a sound in your environment is making you irritable, but it has passed into your subconscious, you may not know why you are feeling that way unless you stop for a second and say, “What's going on right now?” Then all of a sudden you hear a refrigerator hum that is very dissonant and is creating this very low-level anxiety that you haven't been aware of. That's a simple example. In architecture, people speak of feng shui to get the proper energy flow. The position of the doors, furniture, and the lighting is very important for the well-being of the people that live in that home. And in that sense becoming aware of the feng shui of sound is equally important whether it's the sound of a little water fountain that gives you a sense of quiet peaceful lulling or whether you're actually putting up a sound baffle on the window that might block out the roar of the traffic. These are some areas that people specialize in, but we don't usually pay attention so much in our daily environment. Three Deaf Mice will hopefully broaden and enhance people's perception of our sonic environment.
PSE: This reminds me of a study I read recently showing that noise pollution can actually impair reading comprehension skills in children (Read study).
Good example, the other extreme is that they are using music now to help newborns with different physical ailments to actually recover quickly (Article). Talking about the power of music and health, this is not the only article of course. There are just so many examples of how we can become more aware of sound and music in health and well-being.
PSE: How will the game play be structured to guide the players to become more aware of their environments?
The game structure is based on a song called "Cheatin' the Trap," which tells the story of how the mice began their careers in psych labs and wild fields, and ultimately met up and found different objects in their environments to create music. The mice ultimately built a career in music and get to the huge stadium rock concert. This story is told in the song with 10 verses, each one at a different stage of their careers. In each of those verses, we dive into what's happening with them and discover different environmental sounds that we have to explore. There will be a series of treasure hunts for specific sounds inside the environments where they have to identify a sound and where it's coming from. When that's correctly solved, the player will move deeper into the environment seeking and finding sounds within sounds. So we're going to be employing what's known as “masking” where certain sounds will be louder and other sounds will be hidden inside them. We'll be training the player's ear to listen for sounds within sounds and to identify what makes a sound different than another sound. So by discovering what those sounds are, it's training the player in their “causal” listening skills for identifying the source.
PSE: Would you walk us through an example of one of these sonic treasure hunts?
One example from the game is in the bathroom, where we first hear the sound of water running. Once we've found that sound which is the shower, we then hear a sound inside that's kind of like a squeaky sound with a higher pitch and shorter envelope. We find that it's the medicine cabinet door squeaking open. Inside there we find some bottles, and by shaking the appropriate bottle you'll hear the sound of the pills inside of it. Once you find the pills and open the bottle, you drop the pills into a glass of water and hear the "plunk plunk". This causes the final “core sound” of the effervescent fizz of antacid in the water. So that's an example of how we will travel through an environment of sound effects that are linked to the story and song lyrics. Once you’ve found what I’m calling this “core sound”, then it becomes the focus of the next part of the game, a sonic transformation through digital sound processing.
PSE: How do you define “reduced” listening? How is this type of listening encouraged in Three Deaf Mice?
In this environment we're also going to identify what makes the sounds different by their shape or their actual physical waveform, called “reduced” listening. In other words, if we have a sound that has a higher pitch but is "inside" a louder sound, you can hear it easily because it's popping out with the different pitch. Then we can identify that as being the sound we're looking for and move into the environment where that sound is what I'm calling the "central sound." Then inside of that sound we're going to find other sounds.
Reduced listening refers to the shape of the sound waveform, and identifying its physical qualities. If we have a flute playing a melody and we have a violin playing the same melody, the pitch and rhythm might be identical, but what would be different is the timbre or frequency spectrum. Manipulating through equalization is one of the tools that we would use to shift the timbre of a musical instrument. The more complex the timbre, the more harmonics and overtones that exist in that sound and the rougher the waveform would look visually. If you go towards a sine wave, which is a pure tone, you’ll see a very smooth curve. The other extreme, white noise, is extremely complex and randomized so that it looks like a very rough, chaotic waveform. The game will have a visual equivalent of what the sound wave is doing as I just described. So if we're going to change the pitch, we'll see the waveforms getting shorter or more frequent. Another example, if we sharpen the attack by clipping or gating it, then we'll see that attack being a more sudden impact on the visual waveform, as opposed to a gradual buildup. So those are things that will help people begin to understand how sound is structured with the aid of visual stimuli.
PSE: How does this processing transform the sound effects into musical sounds? Will there be other types of processing in Three Deaf Mice?
For example, we’ll multiply tracks, shifting pitch and create harmonies. We can also create beats by looping different effects. There's a tremendous amount of creative work with rhythms being done by DJs and hip hop in general, a super creative form. This is the kind of thing that we'll make accessible in the game through the different kinds of sound processing so that people can explore these creative musical paths.
Check in later this week for Part 3 where we will discuss semantic listening, meaning and communication, and will wrap up the blog series. To learn more about 3 Deaf Mice visit the project Kickstarter page at http://www.3deafmice.com