Worldbuilding: Where Gaming Meets Sound Design

by Mike Stango,  Director, Global Brand Partnerships

Ever since I first held a Super Nintendo controller, I’ve been hooked on video games. From Super Mario Bros, The Legend of Zelda and Super Smash Bros, to The Elder Scrolls, Dark Souls, Sea of Thieves and Rust. These unique worlds filled with iconic characters, brain twisting puzzles, and competitive challenges keep pulling me back in.

Why? Over 25 years of gaming experience has shown me that the difference between good games and great games is thoughtful storytelling and exceptional worldbuilding. Memorable characters, captivating visuals, and exciting rewards propel a game to the next level. But one element that is often overlooked, yet incredibly important to storytelling, is sound.

Thanks to our sonic branding research with Sentient Decision Science, we know that the emotional response to sound is 86% correlated with a consumer’s desire to engage with or avoid an experience. What this tells us, is that although the visuals of a video game can hook you, sound is key to captivating audiences — and keeping players coming back.

Below, I’ll highlight three types of sonic elements that when combined thoughtfully, provide a rich, immersive, and addictive experience for gamers like me.

Theme music

Remember the Super Mario theme song? It’s an iconic piece of pop culture. Everyone knows it, and everyone loves it. Composed by Koji Kondo, the theme instantly transports you to another world, perhaps even triggering memories of your early Super Nintendo side-scrolling days.

The most meaningful — and often nostalgic — audio element in video games is the theme music. Themes can be expressed as songs or melodies that tell an emotional story about a character, location, or game as a whole.

Memorable, unique compositions build atmosphere, drive your emotions, and pump life into each of these worlds. There’s nothing quite like the whimsical charm of Peach’s Castle in Mario 64 or the playful excitement of Kokori forest in The Legend of Zelda: Ocarina of Time (both also originally composed by Koji Kondo). In particular, I’ll never forget the first time I turned on my Xbox and heard the Halo theme song. The haunting Gregorian chant slowly swelled, sending chills down my spine, and amplifying my excitement to explore this unknown world.

Halo · Martin O’Donnell · Michael Salvatori

Sound design

In the early days of video games, Pong used single wave tones that would shift in pitch, length and timbre depending on whether the ball was hit by the player paddle, the side wall, or went in goal. These sounds, part of a game’s sound design and created by game designer Allan Alcorn, intuitively guide you throughout the experience, and ultimately shape the way you play.

Sound design encapsulates the unique suite of sounds that help define a game’s world — anything from the toy-like character expressions in Fall Guys: Ultimate Knockout (composing by Jukio Kallio and Daniel Hagström) to the eerie sounds of Doom (sound design by Ben Carney and Chad Mossholder).

Within this category of gaming audio design are reward sounds — audio cues signaling success. These sounds pop up in scenarios such as completing an action or unlocking something new. As you learn what these sounds mean, you become addicted to hearing them, which drives you to keep playing, succeeding, and advancing. The iconic Legend of Zelda chest sound is as good an example as any. Hearing that fluttering build up as the chest slowly opens, leading into the 4-note celebratory fanfare when you hoist up that hard-earned new piece of equipment is as satisfying as it gets.

Foley

Foley is the reproduction of everyday sound effects such as footsteps, breath and environment sounds. It creates a sense of realism that is essential in subconsciously tricking your brain into feeling as if you’re physically immersed in the environment of the game.

As avid gamers know, foley is absolutely essential in competitive FPS (First-Person Shooter) games. Sound is the “fastest” of our senses — meaning human reaction to sound is faster than that of visual cues. The sounds of the environment keep players in tune with what’s happening around them, and is often critical to your survival. These sounds can provide vital information such as how far away the enemy is, which direction they are running, what kind of equipment they are using, and whether they are distracted or ready to fight. Keeping your “ears to the ground” is oftentimes the difference between being crowned the winner or dubbed a loser.

The popular battle royale game Apex Legends features a unique use of foley, developed by sound designers Tyler Parsons, Brad Snyder, Jordan Harmon and Rick Hernandez. In the game, varied footstep sounds signal which characters may be in your vicinity. Thus, active listening during gameplay, recognizing the subtle differences in footsteps, and knowing how to prepare for your next fight, can give players the competitive edge.

What’s next for sound and gaming?

So what happens as we look ahead at the future of gaming, in areas of complete freedom and immersion such as VR? In these new formats, audio will become even more vital.

“Without the correct or expected audio cues to match the visuals, the brain simply will not accept the illusion of virtual reality.”

Our resident VR expert Joel Douek hits the nail on the head: “Chemically, the brain cannot tell the difference between virtual reality and reality itself. [In VR soundscapes] we seek to mimic the way we naturally perceive the world, so our brains are convincingly fooled. Without the correct or expected audio cues to match the visuals, the brain simply will not accept the illusion of virtual reality.”

A key piece of successfully achieving this illusion, is through spatialized audio technology, which provides the ability to place audio within a virtual space at relative vertical and horizontal distances, and anywhere on the 360-degree spectrum around the player. This technology is one of the best ways to organically guide VR gamers through an experience. Auditory waypoints provide navigational guidance for players, so they can intuitively decipher where to go and what to do without disrupting the immersion. Without thoughtfully designed sound, the experience — and the game world — will fall apart.

It’s no argument that music and sound are key elements in video games old and new. So as gaming technology continues to advance, consider the often-overlooked element of gaming sound design, and the immense impact it has on these experiences. The immersive worlds of gaming will only expand from here, and audio will be key to guiding its future.