VR: Needs Great Storytelling, Visuals and Sound

Why Sound Is an Immutable Ingredient for Great Storytelling

Virtual reality and augmented reality have become resident buzzwords, first in the lexicon of Fast Company and TechCrunch writers and futurists, followed by educational institutions, healthcare providers, news sources, media conglomerates, and early tech adopters—the list goes on. They are words that are rapidly gaining traction alongside old timers like the “Internet of Things,” “Thought Leader” and “Millennial.” And if you’re not tapping into virtual reality’s potential, it seems as if you’re already behind the times.

As a strategic music studio, our philosophy is based on the premise that half of storytelling is sound, because it is the strongest conveyer of emotion. We recognize that great sound is the missing link between creating reality rather than a shadow of reality. Therefore, our expertise in music and sound is a unique contribution to this buzzworthy medium. Most everyone in this field has been focused on great visuals, with sound as a mere afterthought. This lack of attention to sound negates half of the potential experience and with that goes half of a story’s emotional undercurrent. Furthermore, the tools we’ve pioneered to create positional audio allow sound to be accurately placed in space, relative to imagery, which has drastically improved our ability to create more true-to-life experiences.

With an influx of virtual reality content, we took a moment to critique three recognized VR experiences over the past year, with an eye on both sound as an emotional storytelling tool and the use of positional audio to increase realism.

Catatonic

Catatonic, a spectacular, spine-chilling immersive journey, takes you through an insane asylum, in which you are a wheelchair-bound patient.

As you exit the elevator and are wheeled through the doors of the asylum, you begin a deeply unsettling sonic journey. Turn your head to peak into the patient rooms along the corridor, and you’ll hear shrill screams, low growling and glass shattering, all of which slowly dissipate as you continue farther down the hallway. The film makes excellent use of the agonizingly slow tempo of footsteps and a squeaking wheelchair, evoking a sense of intense discomfort and underscoring the sinking feeling that escape is a fleeting dream.

In a few instances, Catatonic does fail to capitalize on opportunities for sound. Throughout the 7-minute film, you are being pushed by a large, faceless attendant. As you turn your head to face your nurse, the film doesn’t jump on this interaction to increase a sense of terror: you hear no breathing down your back, no muttering. As you glance downward, you realize you’re strapped to the wheelchair. At this point, the filmmakers might have played with the sound of shackles clinking, increasing the sensation of being trapped. In one instance, a doctor inserts a needle into your hand, but there is no sensory stimulation—the sound of the doctor flicking the needle or the push of a syringe—causing you to momentarily snap back into a passive viewing state.

Watch the experience without sound and you’ll find it half as terrifying. What would the shower scene in Psycho be without the screeching violin?

For thrill-seekers who are desensitized to horror films, try Catatonic on for size. Good luck: Catatonic

Take Flight

Take Flight is a short New York Times film that celebrated 2015’s best actors through a fantastical recreation of iconic airborne moments in cinematic history.

The film relies strongly on audio to set the stage for the story by recreating busy Los Angeles streets paired with blurry opening visuals. You hear ambulance alarms, honks, tires on wet pavement, cues that immediately alert you to your whereabouts. However the film fails to use enhanced positional audio capabilities that would more fully immerse you in the environment. As you do a 360 scan, the addition of passerby street conversations or clips of music as cars rushed past would capture a more true-to-life soundscape of Los Angeles and give greater depth to the environment as people and things move closer and further away from you.

As the film progresses, you leave the city and begin to float above the clouds. The sound of the cityscape becomes a distant memory, and a beautiful musical score begins, contrasting the bustling sounds of reality with the dream-like state of flying above the Los Angeles skyline. The music is the key emotional undercurrent that highlights the fantastical scene of floating above the clouds, childlike and playful yet transcendent. Above the clouds, you witness historic film scenes, for example E.T. and Fred Astaire’s ceiling dance during Royal Wedding. But the film experience could be enhanced by using positional audio capabilities for accompaniment. These could include sounds from E.T.’s bicycle or Fred Astaire’s tap shoes as you move closer and further away from the various floating actors, enhancing the magic of these moments.

For a dream-like escape: Take Flight

The Night Cafe

The Night Cafe, a well-received virtual reality tribute to Van Gogh, recreates the world of the artistic genius. This nod to Van Gogh, known for his expressive and spontaneous use of vivid colors and emotive subject matter, only dabbles in sound, a missed opportunity to give the piece a strong emotional undercurrent.

While the animation is stunning, you feel as if you’re viewing a tour through the artist’s home rather than actively walking through this creative space. The filmmaker could have tapped into sound and music to recreate the art scene in 19thcentury Paris or delve deeper into the mind of the bipolar painter through a variety of musical underscores to mirror the frequent mood swings the painter suffered from. As virtual reality affords us opportunities to travel to imaginary worlds, sound and music could also bring to life Van Gogh’s most famous works, Starry Night and Vase with Twelve Sunflowers, as you move in more closely to view each of the paintings.

For art aficionados: The Night Cafe

As virtual and augmented reality twist, turn and mold, we can guarantee that sound will continue to be a key storytelling tool to set the mood for the experience, give depth to this 3-D world and heighten emotion.