Explore the process behind creating the detailed soundscape of The Social Dilemma.
Richard Gould is a Primetime Emmy®-winning sound editor, designer, supervisor, and a Skywalker Ranch veteran, known for his work on The Mandalorian (2019), Black Panther (2018), The Rise of Skywalker (2019), and other blockbuster films and TV shows. Throughout his career he's worn many hats as a foley editor, foley supervisor, and sound designer, amongst other roles.
Most recently Richard brought his attention to detail and creativity to the Netflix documentary The Social Dilemma (2020) as a supervising sound editor. We caught up with Richard to learn about the process creating the sound for the film.
Warning: May contain spoilers.
How did you get involved in The Social Dilemma, and what was the overall process of working on the film like
Initial conversations began around October of 2019. I got a rough cut but the drama sequences were all storyboards at that time as they hadn’t been shot yet. It was exciting to see that component and the VFX get added gradually to the cut as they were really transformative. Due to the nature of the subject matter, the documentary portions largely consist of talking heads, which while gripping and informative, really benefited I think from the dramatic component which made the film not just an important piece of documentary filmmaking but also an entertaining one to watch.
In mid-December we got our initial turnover and quickly started the design, editorial and Foley work ahead of the mix which was due to take place just a few weeks later with the Sundance Film Festival premiere following shortly after that. The core crew at Skywalker Sound consisted of myself, dialog editor James Spencer, Foley artist Andrea Gard, Foley mixer Jason Butler and our re-recording mixer Scott Lewis.
This normally wouldn’t be a point of interest, but given this past year it’s perhaps worth noting that the initial mix took place with the clients in-person and on-site at Skywalker Ranch, after which it went off to Sundance and enjoyed a successful premier.
The initial plan had been for the clients to come back once they had secured distribution to add logos and incorporate a few minor editorial changes they intended to make at the time, but given how the issues in the film were playing out with respect to the pandemic, the director Jeff Orlowski and his team at Exposure Labs realized they had to find a way to incorporate the pandemic into the film. So everything you see (and hear) in the film that concerns COVID-19 was added after it had premiered at Sundance in a second round of editorial and mixing that took place in the spring.
At that point with Netflix on as the distributor and with theaters shuttered around the world, we realized the format we needed to prioritize was the near-field, not the theatrical. There was also a sense of urgency to get the film out into the world. So with the Bay Area under a shelter in place order, we had to do everything remotely with myself and James Spencer cutting at our respective homes in Northern California, Scott Lewis in Los Angeles mixing, and the Exposure Labs crew in Colorado reviewing and giving notes. It was certainly a bit of a roller coaster and not without its challenges, but given all of that we were all very happy with how it sounded going out of the virtual door.
"For me, more than any other element, Foley gives a sense of tangibility to the world being presented."
On Skywalker's Archives Foley stage, from left to right: Richard Gould, Dave Wruck (Assistant Editor), Larissa Rhodes (Producer), Tessa Malsam (Assistant Editor), Mark Crawford (Composer), Davis Coombe (Picture Editor) and Jeff Orlowski (Director). Photo credit: John Roesch.
As far as the actual content of the film, did the directors have any creative direction for the animation or VFX sequences? How much input did you have for areas where you can do a thousand different things in terms of sound?
I think partly because Jeff and the picture department were still focusing on getting the drama and all the VFX finished, I was largely left to my own creative devices initially and had a lot of fun working with the more esoteric drama elements as the VFX came in. I started sending initial design concepts over for review and eventually larger chunks of the film till I had completed a full pass. I was anticipating an iterative creative process for a lot of the design elements, especially the sounds in the AI world, but given the mix was just a few weeks out I was thrilled to find the vast majority of the design concepts I passed over for Jeff and his team to review quickly got their stamp of approval, perhaps because they were distracted by other concerns!
Ahead of the mix, Jeff Orlowski (director), Larissa Rhodes (producer) Davis Coombe (picture editor) and Mark Crawford (composer) joined me in my editorial space to play the film down top-to-bottom to address any final editorial/design notes so that Scott Lewis our mixer could maximize his time on the stage putting his own creative signature on the project. I want to note that Mark Crawford was especially trusting and gracious with his excellent score. I’m a composer myself and Mark allowed me to essentially serve as the music editor offering up occasional editorial ideas which he was always receptive to, so it was fun to flex that creative muscle too.
That level of trust extended to the dialog track as there were a couple of moments where I made a suggestion for a subtle change in the pacing of a line when the interview subject or actor is off-screen to place greater emphasis on a word or give a little more emotional weight to a moment. I even suggested a picture change at one point during the mix to better land with the music in a given moment (thankfully it didn't change the LFOA!). All of that is testament to the working relationship and mutual respect we all had which made the experience incredibly satisfying and invigorating, at least for me.
As far as the sound design goes, there were a lot of unique elements in the drama that warranted specific sounds being designed for them, especially in the AI space where there’s the general ambience, the control console in front of the three AI characters that had both an idling sound and all the various input and UI sounds, and perhaps the most interesting element from a conceptual standpoint, the avatar of Ben which develops throughout the film which gets more detailed as they learn about Ben until it’s a 1:1 visual representation. The sound I designed for the avatar starts off very busy and active, as if the AI machine is working very hard to calculate who exactly Ben is. Then each time we return to the avatar, as the image becomes more sophisticated, the sound becomes less complex and less active. At the very end when it’s a 1:1 representation, there’s no sound ascribed to it which makes for an interesting, if subtle, contrast that perhaps highlights the avatar’s humanity in that moment.
How did your previous work as a Foley Editor/Supervisor lay the groundwork for you to be a Sound Supervisor?
In my time at Skywalker I've been fortunate to bounce around different roles, starting as an intern for Randy Thom, then working as an apprentice, assistant, supervising assistant, editor, designer and more recently as a supervisor. With each role I learn things I carry forward onto future projects. My time as a Foley editor and supervisor brought a level of confidence and expediency that was crucial on The Social Dilemma as I knew going in I only had a day to record Foley with Andrea and Jason. It was crucial that I prioritize the elements that would benefit the most from their talents and would otherwise be challenging to cut from other sources. Typically that ends up being the more specific movements, actions, handling of uncommon props or very subtle performance details that are hard to cut from a library. I sometimes refer to library Foley as “Faux-ly” as it’s really the act of performing that defines Foley more so than a categorization of sounds.
That’s not to say it’s all positives that come from previous experience. Spending so much time in the Foley world, I know I cut in a little more than was really required, but that’s partly my aesthetic at this point in time. For me, more than any other element, Foley gives a sense of tangibility to the world being presented. It’s often subtle, and if I’m leveling things myself I often say I’m looking for “just enough to tickle the ears”. In that regard (and others) Scott’s mix aesthetic really worked with my sensibilities though I think he would agree I cut in a tad too much Foley. I remember there was one moment during the mix where he was mixing some footsteps in, paused, turned from behind the desk towards the clients and said jokingly “you got your money's worth on the Foley”.
Speaking of Foley, one of my favorite moments in the film is when Tristan Harris, who’s the principal interview subject in the documentary, performs some sleight of hand tricks with a coin. Andrea and Jason added just a little coin handling texture to that shot that really brought it to life. Another favorite moment is the bedroom/bathroom sequence with Isla, the young daughter in the drama as she posts a photo of herself on an app and then grapples with her insecurities about her physical appearance that are triggered by a friend’s comment. All of those scenes in the bedroom and in the bathroom were shot MOS, with no production sound, so it’s a great example of where I knew bespoke Foley was going to add a lot with Isla’s subtle movements on the bed, handling her phone, touching her hair and face as she critically looks at herself, and as she is on the verge of tears there’s even a whimpering breath. All of that was Foley and it beautifully enhances the powerful screen performance by Sophia Hammons.
"My design approach for film differs somewhat to other mediums as I typically have to be very efficient with my time... I really enjoy trying to find creative ways to work against the structure of the story, flow of the cut, the details in a performance."
One of my favorite things to do is to add elements beyond the edge of the frame. There’s a cough right at the opening of the film as the interview subjects are sitting down that draws one of their attention off-screen. That wasn’t actually in production but something I added to justify their “performance” as it were. A similar thing happens in the drama when Ben (played by Skyler Gisondo), is walking towards a protest and he looks over his shoulder off-screen as if something caught his attention. It’s a lovely little performance detail that I decided to play off of, so as you’ll see and hear in the film, it’s in fact a police car and its siren speeding by with a doppler in the distance that drew his attention. These added sounds not only help to justify the performance in that moment but it also helps to expand the world and also forebodes the upcoming protest sequence in which he gets arrested. In fact, when we see Ben and his sister Cassandra (played by Kara Hayward) cuffed and being processed by the police later that evening, I threw a distant police helicopter into the scene to add a little to the drama of the moment. It also helps to elevate the production value of that shot. Sound is a fun (and cheap) way to do that kind of thing.
Could you talk a bit about going back and forth between the documentary portions and then the drama scripted scenes, and how you approached going in and out of them?
A lot of that flow was already in the edit. The picture editor, Davis Coombe, did a masterful job making that back-and-forth work so I only needed to worry about not messing it up, and hopefully on occasion, enhancing it a little. Sometimes the goal was a seamless transition while others were intentionally jarring. The first time we cut to the drama, I cut in a simple car passing by that anticipates the cut by a few seconds to gently wash us into the shot while also firmly setting us in suburbia. The hope is you arrive at the shot presuming the car (which doesn’t exist) just exited the frame.
That’s a simple example of one of the transitions. A more involved back-and-forth happens during Isla’s bedroom/bathroom sequence I previously mentioned. The film cuts back and forth from the drama with Isla in her bedroom then later her bathroom, and an interview with Dr. Jonathan Haidt who’s discussing the issue of rising suicide rates, particularly among young girls. Given the mood of the scene, it felt natural (though I’ll admit perhaps a bit cliche) to place a rainstorm outside Isla’s home, even though you don’t see any evidence of one. It grows in intensity through the sequence and hopefully helps to connect Isla’s shots as all taking place within one evening in the mind of the audience. I timed the thunder very carefully around the drama of the moment, hitting, anticipating or responding to specific moments in the story and also letting it roll across the transitions between the drama and documentary worlds. I also put in a very faint siren on the first shot of Isla in her bed which was a bit of an attempt to forebode where the story and characters may be heading.
"One of my favorite things to do is to add elements beyond the edge of the frame... These added sounds not only help to justify the performance in that moment but it also helps to expand the world... Sound is a fun (and cheap) way to do that kind of thing."
Credit: Exposure Labs
Do you have any plugins that you consider “go-to's”? Or are there any that you used in this film that are really a part of the sound of it?
There are certainly some I’m more fond of than others, ones I gravitate towards, but I’d hardly be the first to say that most of the time I’m using the basics; pitch shifting, EQ, dynamics and reverb. My design approach for film differs somewhat to other mediums as I typically have to be very efficient with my time. As a result I gravitate towards more of an editorial design style where I’m layering library elements (including my own) and pulling what I want out from each of them to realize the sound I’m hearing in my head as quickly as possible. When I’m generating content for an interactive title such as the VR experience Vader Immortal which I contributed to as a sound designer, I typically have a little more time to experiment with tools. In film, as may be evident in some of the answers so far, due to its linear nature, I really enjoy trying to find creative ways to work against the structure of the story, flow of the cut and the details in a performance.
In a way that the philosophy is so much more important than the actual tools.
To a degree, though some tools are undeniably indispensable so I don’t mean to minimize them in any way. I certainly value ones that speed up my workflows thereby allowing me time for creative thought elsewhere. For example I’m a big proponent of macros and wrote an article a few years back about them. They aren’t an audio tool per se, but they certainly make my work sound better as they afford me more time to think (or sleep!) which is my most valuable creative commodity.
There are some tools I use where I dutifully read the manual top-to-tail because I know in the long run it’s worth the time investment to learn. I’m currently doing this with the Avid S6 Operation Manual as I’ve started mixing some projects lately and figure that’s a tool that is going to be around a while. Then there are other tools where I’ll gleefully surf the presets knowing I’ll get something in the ballpark soon which I can then adjust to taste. But whether it’s a surgical approach or throwing the proverbial stuff at a fan to see what ends up on the wall, there’s always choices being made, even if it’s choosing one happy accident over another.
As it relates to The Social Dilemma I should mention iZotope’s Vocalsynth which I used for a subtle effect for the AI characters voices, played by Vincent Kartheiser. We had discussed the potential of processing their vocals early on but decided against it at first as we were worried it might prove distracting. Once the rest of the AI soundscape was in however, the very human performance felt a little out of place so I came up with a dozen or so treatments for the director Jeff to listen to, and we ended up going with a Vocalsynth patch I made that triplicated the voice at specific intervals, two above and one below the original pitch. Jeff loved the concept as it mirrored the three AI characters. It’s subtle but it does help to dehumanize the AI characters slightly, to make them feel somewhat artificial, and at the end of the film once the code has been rewritten and the last remaining AI utters the word “hello” to Ben as if seeing his humanity for the first time, we don’t have any of the effect on that line and it makes for a nice moment of contrast, supporting the idea that this AI is now somehow different in function than it was a moment ago.
Were there any other favorite moments of yours in the film that you’d like to highlight?
Sure, it was a fun challenge to make a range of sounds for all the character’s phones. I had a pallet for each that was unique to their character. Ben’s for example was more tech-y while others were more musical in nature say for the father or more playful ones for the youngest in the family, Isla. There’s quite a range of notifications, alerts, UI responses, photo shutters and typing sounds unique to the film which typically would take months of design, testing and review before they make it into a real-world device/service, so it was fun to see what I could get done in a day!
I wasn’t always so fast though. The sequence that took the longest to create relative to its runtime was the quick-cut Requiem For A Dream-esque montages that play between several of the interview subjects as they talk about how they themselves are addicted to these tools they’ve created. They’re very visceral, dramatic, they come out of nowhere and they were a lot of fun to intricately design as each shot is mere fractions of a second. The first one opens with the Pinterest logo which I anticipate with the classic iPhone unlock sound. The unlock sound was reminiscent of a shotgun being cocked so there’s a dramatic shotgun-esque hit that follows it on the first frame of that sequence. Then, as Aza Raskin talks about his addiction to Reddit, I used casino elements pitched to the score to tie into the addiction methodology they discuss as being similar to the slot machines in Vegas. Tristan Harris then talks of his addition to email at which point we zoom through into the interior of an eyeball. Similar to the iPhone unlock sound being contextually related to what’s being discussed at that moment (and used under fair use due to the subject matter), I elected to use the Apple Mail email sent whoosh sound effect for this moment, layered with an Formula One car pass-by to better match the velocity and intensity of the shot.
Those are just a few of the single shots in those sequences. It ended up being quite laborious relative to the runtime of all the pieces put together which couldn’t have been more than ten seconds or so in total. I think I spent the better part of a day on those shots which if I’d applied that pace to the rest of the film, I’d still be working on it today!
Thanks to Vinny Alfano for conducting this interview, and to Richard Gould for participating!
Follow Richard Gould:
Twitter: twitter.com/audioblackholes
Website: richard-gould.com
IMDb: imdb.com/name/nm5678638
Follow Vinny Alfano:
Twitter: twitter.com/vinnyalfano
Website: vinnyalfano.com
IMDb: imdb.com/name/nm6258778
Want to share this post?
