/ cracking the code on sound in experience design
Written by Kevin Perlmutter & Anjali Nair of Made Music Studio with research partner Cyrus McCandless, PhD, Sentient Decision Science
introduction
Every day, from the moment we wake up, we are immersed in sound. The alarm clock, which jolts you awake, the screech of vehicles in traffic, the frequent dings of smartphone alerts, the subtle hum of an air conditioner, the inevitable wail of an infant on a flight, and so on – an endless underscore that we often tune out. But whether we realize it or not, these sounds affect our state of mind, our behavior, and our ultimate perception of experiences.
Sound elicits a hyper-fast reaction time in humans – about 30 milliseconds faster than reaction times to visuals (Johnson et al., 1985). Thus, even before conscious awareness of the meaning or source of a sound, we’ve had an instinctive emotional response – either positive or negative. The input is then fully processed, leading to an evaluation of whether our instinctual reaction was appropriate to the situation. That initial split-second reaction to a sound is vital – it establishes our emotional state and colors our expectations for an experience.
Our highly emotional responses to sound have a biological basis. Researchers at Newcastle University found that the amygdala, the “emotion center” of the brain, triggers heightened activity in the auditory cortex, the part of the brain that processes auditory input, when unpleasant sounds are perceived versus soothing ones (Kumar, Kreigstein, Friston, & Griffiths, 2012.) In other words, when we hear a distressful sound, the emotional part of our brain urges our auditory sensors to perk up and pay attention.
While this may have begun as an evolutionary trait to promote the survival of our species, it has lasting implications on life today. Unpleasant noise has been linked with mental and physical stress, including anxiety, heightened blood pressure, and heart rate (Stansfield, 2016). Similarly, lack of control over one’s sonic environment can create dissatisfaction with the workplace (Lee, et al., 2015), and about 20% of us experience misophonia – involuntary reactions to certain sounds that are so extreme they cause significant interference with relationships and ability to carry out simple tasks (Wu, et al., 2014).
Yet, most are quite unaware that sound powerfully affects the way we experience the world. Oblivious to the inner processes modulating behavior, people like to believe that they’re in control of their actions. However, decision-making is not based in pure reason and logic, as we often assume. Whom we befriend, whom we vote for, and most interestingly to marketers – what we buy – are all susceptible to influence from our subconscious (Bargh, 2014).
Since responses to sound often make first impact below the threshold of awareness, it has been difficult to study its effect through conscious means traditionally used in market research. This may be one key reason why sound is often the lowest priority in the creation of products and experiences. Our modern world has too much of what we call Sonic Trash. Sonic Trash describes the harsh, grating sounds that add no value to an interaction and can cause people to tune out or avoid an experience completely.
Made Music Studio and Sentient Decision Science collaborated on an initial SonicPulse® Research study to quantify the impact of sound on people’s subconscious emotional responses as it relates to a variety of experiences. The results provide new data and insight into the key and often misunderstood relationship between sound, design, and human behavior.
Through our research with Sentient, we have tested and now are able to prove the value of making sound an early component of the design process. When we eliminate Sonic Trash, or bad and purposeless sounds, and replace it with sounds that better serve the experience, we improve outcomes for public wellness and business results.
Learnings show that as contributors to the built environment, designers and brands who neglect to consider the immense influence that sounds can have on consumers and the public will lose out on attention, loyalty, relationship and business outcomes.
Continuum has helped us elevate our sonic strategy across our social and digital platforms, and has saved our team valuable time and resources. Implementing our sonic branding strategy is now faster and easier from start to finish.”
methodology
The benchmark wave of this initial study evaluated a broad range of recognizable sounds on their emotional appeal and their effect on desire for an experience. Since we rarely experience sound in isolation, the study also analyzed the influence of visual context on the perception of sound.
Nine-hundred adult participants (US, General Population) were recruited by Sentient Decision Science to complete an online survey evaluating sounds on emotional appeal and respondent’s desire to engage or avoid an experience associated with those sounds.
As part of the study, twenty short-form sounds (:03 in length) were evaluated on key measures. The stimuli consisted of ten Naturally-Occurring Sounds, which were incidental or environmental, along with ten Designed Sounds, which were created intentionally as part of a product, brand identity, or experience. All the stimuli were sourced from existing sonic libraries and recordings of experiences.
implicit association testing
Historically, observing sub-conscious thought processes required expensive and obtrusive techniques such as Electroencephalography (EEG) or Functional Magnetic Resonance Imaging (fMRI). More recently, however, the development and refinement of a research technique called Implicit Association Testing (“IAT”) has allowed researchers to cost-effectively and accurately quantify the impact of emotion on behavior.
We chose implicit association testing (“IAT”) as the foundation for the project because it met all of the methodological requirements for the study, namely (1) cultural/linguistic agnosticism, (2) the ability to isolate true subconscious responses that are not subject to respondent control or manipulation, and (3) sufficiently cost-effective to replicate in multiple countries, each with sample sizes large enough to permit analysis on a segmented basis.
IAT is a computerized experiment – typically administered via computer, smartphone, or tablet – that scientifically measures the strength of an individual’s automatic associations between concepts (e.g., products, concepts, brands, sounds, etc..) and either attributes (e.g., energetic, innovative, trustworthy) or emotions (e.g., excitement, contentment, boredom, anger, and so on). It combines affective visual priming and response time, requiring participants to rapidly and correctly categorize target concepts with an attribute, such that easier pairings (faster responses) are interpreted as more strongly associated in memory than more difficult pairings (slower responses).
Respondents are first asked to correctly sort a series of images representing emotions into one of two categories – either Positive or Negative. Their speed and accuracy is measured as an individual cognitive baseline against which their results will be normalized – thus, data can be aggregated across individuals with differing levels of cognitive speed. There are right and wrong answers to this baseline cognitive task, so respondents must make a conscious and reflective judgment.
Respondents then complete the same task while being intermittently primed with a stimulus (audio and/or visual). Their speed and accuracy are measured again; the irrepressible subconscious response triggered by each stimulus serves as a distraction that can make it harder to complete the sorting task quickly and accurately. Differences between test and baseline metrics reflect implicit cognitive or emotional association between the stimulus and the target image or sound.
The results are scored along an index, which is scaled from 0 to 200, with the midpoint of 100 representing a neutral emotional association. Results above that point represent positive subconscious emotional associations, and those below that point represent negative ones. A deviation of +/- 5 points indicates moderate emotional impact, while a deviation of +/- 10 points indicates extreme emotional response.
emotional appeal
Emotional Appeal of sounds was tested using IAT. During this part of the experiment, respondents were asked to quickly and accurately sort emotions, primed by a short sound clip.
Later in the study, the emotional appeal of sounds was tested in-context using the same method. Two pairs of sounds were tested with corresponding images, allowing us to evaluate each sound pair against its visual context. Each pair consisted of a sound hypothesized to be pleasing and a sound hypothesized to be distressful. The results were then compared to those from the sounds in isolation to identify the effects of context.
desire to engage or avoid
Conscious desire for an experience was evaluated through a Maximum Difference Scaling Choice experiment, an established test of preference in marketing research. Participants were asked to consider the experience associated with a set of presented sounds and to select the experience they would seek out the most, as well as the one they would want to avoid the most in the future. This was repeated through several cycles to establish a hierarchy of the most desirable and most aversive sounds.
“Our reactions to Naturally Occurring Sounds provide powerfully intuitive reference points for evaluating emotional responses to Designed Sounds.”
outcomes
As a result of our research, we established the first Sonic Humanism™ Spectrum:
Some of the finest work I have ever been a part of. Strategic, smart and creative.
– Gregg Heard, VP, Brand Identity and Design, AT&T
The Sonic Humanism™ Spectrum scores sounds based on their emotional appeal to a representative sample of respondents from most appealing to most distressing. The ranking is indexed to Sentient Decision Science’s database of millions of sensory stimuli that have gone through IAT over more than a decade.
What we discovered reveals the significance of subliminal processes; there’s a dramatic 86% correlation between our subconscious emotional response to a sound and our conscious desire to engage with an associated experience or avoid it in the future.
Which means, if you’re not thoughtfully considering sound in your experience, you could be unintentionally causing a large majority of people to want to avoid the experience.
The Sonic Humanism™ Spectrum research found that a pained scream was the most unpleasant sound tested (Emotional Index = 91.5) while that of baby laughter was the most pleasant (Emotional Index = 116.8). These scores offer an objective measure of what we instinctively know: the instantaneous discomfort caused by a scream of pain, and the delight of a baby laughing.
Our reactions to Naturally Occurring Sounds provide powerfully intuitive reference points for evaluating emotional responses to Designed Sounds. Surprisingly, many sounds emitted by products fall in a territory that indicates a lack of purposeful sonic design from the companies that create them. For instance, the well-known Emergency Broadcast Weather Alert (Emotional Index = 93.2) ranks between a pained scream and nails on a chalkboard – two of the most emotionally displeasing sounds out there, statistically speaking. Several other Designed Sounds, such as the piercing beep of a microwave and the harsh signal of a credit card chip reader rank poorly in emotional appeal as well.
Some of these iconic sounds were created decades ago, when the technology or hardware of a product could not accommodate more artful sonic design. Luckily, it is now possible to break the convention of using anxiety-inducing tones to convey relatively innocuous messages and set a higher standard for Designed Sounds.
we established the strong relationship between sound & desire
We discovered that subconscious emotional appeal of a sound is 86% correlated with our conscious desire to engage with an experience or avoid it. The implication for brand and product designers is that sound can create or destroy value, depending on how effectively it’s used. No matter how small or expansive the medium, designers can bring empathy, efficiency, and emotional engagement to an experience by purposefully creating and curating emotionally appealing sounds. This in turn will drive consumers’ desire to interact with the brand or product in the future.
we found that the context in which a sound is heard amplifies its impact
In most experiences, sound is part of a larger context. To understand the way people normally experience some of the tested sounds, we paired them with an appropriate image to create a real-world context and reassessed emotional appeal.
Two variations of a Home Security Keypad Alert (Sound A and Sound B) were presented with identical images of a home security keypad to reproduce the context in which the sound would typically be heard. The intention of both sounds is to remind someone re-entering their home to turn off the security system. Results showed that the emotional valence associated with a sound amplified the relatively mundane experience of returning home and turning off an alarm.
Sound A was slightly unpleasant in isolation (Emotional Index = 96.8), but when paired with the context of the keypad alert it was perceived as even worse (Emotional Index = 94.7). Note that the sound itself is not an alarm – the emotional distress it causes is ill-suited to the context. In contrast, Sound B was emotionally neutral in isolation (Emotional Index = 100.3). Given the visual context of the keypad, it was significantly more pleasing (Emotional Index = 103.5). Thus, when Sound B delivered the message to turn off a security alarm, it was perceived as a friendly, welcome reminder rather than a harsh, anxiety-inducing alert.
Another example that illustrates the importance of context involves weather alerts. We compared the aforementioned EBS Weather alert sound (Emotional Index = 93.2) to a alternative created for The Weather Channel (Emotional Index = 107.8). Each sound was presented alongside an image of a phone screen with the weather alert “Heavy rain and thunderstorm.”
In context, the EBS Weather Alert was statistically just as unappealing as the sound in isolation (Emotional Index = 93.3). In contrast, TWC’s alert in isolation was one of the most pleasant Designed Sounds on the Sonic Humanism™ Spectrum. When presented in-context, it produced a slightly negative emotional response (Emotional Index = 98.7), successfully relaying the message it was paired with. Comparing the EBS and TWC alerts is telling; the level of distress signified by the EBS alert is disproportionate to the urgency of the message, which is non-life-threatening. TWC alert manages to deliver the same unpleasant message, without completely degrading the user experience.
Based on these outcomes it becomes clear that designers would do well to consciously evaluate the sounds they create or utilize to ensure coherence with the product or experience. Sound should not only guide the emotional responses of the user, but also convey meaning and fit to purpose.
Some of these iconic sounds were created decades ago, when the technology or hardware of a product could not accommodate more artful sonic design
implications
The new insights uncovered by Sentient Decision Science and Made Music Studio provide compelling evidence underscoring the emotional and communicative power of sound. By establishing the Sonic Humanism™ Spectrum, we now have quantitative and intuitive methods to compare and measure Naturally Occurring and Designed Sounds based on their subconscious emotional appeal. Our discovery of a high correlation between the emotional appeal of a sound and desire reveals that it’s not enough to simply create pleasing sounds. Designers should also compose sound that is context-appropriate and reflects the emotionality of the larger experience.
Instead of leaving sound as an afterthought, our research shows why designers should treat it as an integral component of their process. When used well, sound can add value to businesses. However, when used poorly, it can damage consumer loyalty and engagement.
There are larger public health considerations at stake as well. A growing body of evidence demonstrates the harm of noise related disturbances. Understanding the impact of sounds at different points along the Sonic Humanism Spectrum™ can address the problem of noise pollution and the harm caused by Sonic Trash. Life and society can be richer and simpler when sound is thoughtfully considered and designed.
works cited
Bargh, J. A. (2014). Our unconscious mind. Scientific American, 310(1), 30-37.
Johnson, R. C., & Al, E. (1985). Galton’s data a century later. American Psychologist, 40(8), 875-892. doi:10.1037//0003-066x.40.8.875
Lee, P. J., Lee, B. K., Jeon, J. Y., Zhang, M., & Kang, J. (2015). Impact of noise on self-rated job satisfaction and health in open-plan offices: A structural equation modelling approach. Ergonomics, 59(2), 222-234. doi:10.1080/00140139.2015.1066877
S. Kumar, K. von Kriegstein, K. Friston, T. D. Griffiths. Features versus Feelings: Dissociable Representations of the Acoustic Features and Valence of Aversive Sounds. Journal of Neuroscience, 2012; 32 (41): 14184 DOI: 10.1523/JNEUROSCI.1759-12.2012 Stansfeld, S. (2016, February 3). Can noise pollution damage your health? Retrieved from https://theconversation.com/can-noise-pollution-damage-your-health-54016
Wu, M. S., Lewin, A. B., Murphy, T. K., & Storch, E. A. (2014). Misophonia: Incidence, Phenomenology, and Clinical Correlates in an Undergraduate Student Sample. Journal of Clinical Psychology, 70(10), 994-1007. doi:10.1002/jclp.22098
about Made Music Studio
Made Music Studio is a global sonic studio dedicated to creating iconic and enduring music and sound to solve creative, human and business challenges. Their work began in television 20 years ago, creating themes for networks and shows (Entertainment Tonight, HBO, CBS This Morning, Super Bowl on NBC, ESPN 30 for 30) and has evolved to include pioneering musical approaches for global brands and products (Nissan, Citi, AT&T, iRobot, Deloitte).
Made Music Studio founder, Joel Beckerman, is a known thought leader it the world of sound and business, author of The Sonic Boom: How Sound Transforms the Way We Think, Feel and Buy and a Fast Company Most Creative. The Made Music Studio team have been featured on stage at major conferences including SXSW, C2 Montreal, Cannes Lions, Fast Company Innovation Festival and WSJ D.Live, as well as in major media such as Wired, AdWeek, Forbes and HBR.
about Sentient Decision Science
Sentient brings visionary advances from the behavioral sciences to the business community in practical and accessible ways. An internationally-recognized pioneer in the development of research technologies, Sentient taps the consumer subconscious and quantifies the impact of emotion and sensory input on choice. Sentient has placed in Greenbook’s “Top 50 Most Innovative Research Organizations” for 3 years running and was awarded the prestigious EXPLOR award for impact and innovation in market research in both 2011 and 2018. Their technologies are coupled with deep knowledge of behavioral economics, the essence of emotional branding, and quantitative models of the drivers of human behavior to provide deep insight and actionable results for our clients.
about SonicPulse® research
SonicPulse® Research is Made Music Studio’s proprietary capability to assess the impact and effectiveness of music and sound. With our research partners, Sentient, we utilize Implicit Association Testing to understand both subconscious and rational responses to sound. For its impact and innovation in market research, SonicPulse Research was the 2018 EXPLOR Award winner and was recognized in the 2018 GRIT 50.