Sound Follows Function

wew
Sound follows function

Sound communication and
the relevance of timbre.
Lecturer: Rainer Hirt & Kai Bronner (audio-branding.de)

wew
01 Acoustic signs and its functions

02 Overview ongoing
Functional-Timbre-Study (FTS)
wew

02 Overview ongoing
Functional-Timbre-Study (FTS)
wew

wew
A. Basics
Signs?
A. Basics B. Study C. Results

Signs!
A sign is a stimulus pattern that has a meaning.
A sign stands for something else.

Semiotics: Theory of signs
Index
An „Index“ is defined by some sensory feature
(something directly visible, audible, smellable, etc)
that is connected to it.

Icon
An icon is a pattern that physically resembles
what it „stands for“.

Symbol
Symbols are arbitrary and unmotivated, reliant on
conventional usage to determine meaning.

Acoustic signs?

Acoustic signs!
Index
Icon
Symbol
Index becomes Icon
Symbol becomes Icon

Applications of
acoustic signs
Cultural Corporate Sound
Auditory Display Audification Auditory-User-Interface

Basic design-approaches of
acoustic signs:
Auditory icons
Auditory symbols
Auditory symcons (Mixed)

wew
B. Empirical study
Ambition & research questions
1. Appropriateness of Auditory icons/symbols and symcons for the transfer of
information-functions:
–> Advice
–> Warning
–> Emergency
2. Effects of the mixture of icon and symbol („auditory symcon“)
3. Advantages of acoustic signs for „public-user-interfaces“

Analysis of a situation
– Talks with manfactures and customers (Höft-Wessel, ICA, DB)
– Secondary literature
– Preliminary study (semantic field-analysis)

Study object
Ticket machine

Study design
Study method: Descriptive study
Survey sample size : N=22

Age average: 35 years
Women/Men: 10/12

Lab set-up
> Lectern (Terminal-simulation)
> Table-PC (Touchpad)
> Headphone

Stimuli
lic nic
nic bo co
ico sy m sy m
Advice: „Ticket ready for collection“ 1 2 3
Advice: „Money ready for collection “ 1 2 3
Warning: „Timeout“ 1 2 3
Emergency: Deterrence 1 2 3
Context-Soundscape (Trainstation)

Evaluation methods
Semantic differential

Evaluation methods
Categorical scaling

Evaluation methods
Categorical scaling
Response time measurement

Evaluation methods
Categorical scaling
Participant observation

Evaluation methods
Categorical scaling
Participant observation
Association analysis

wew
C. Results
1. Appropriateness of Auditory icons/symbols and symcons for the transfer of
information-functions:
–> Advice
–> Warning
–> Emergency
2. Effects of the mixture of icon and symbol („auditory symcon“)

Auditory Icon Auditory Symbol Auditory Symcon
Advice
Warning
Emergency
Funtion(intention) partially transferred
Funtion(intention) transferred
Funtion(intention) not transferred or false interpretation


The meaning of function of acoustic signs is comprehended intuitively.

Further investigations
The study reveals amongst others, that further investigation on the acoustical
parameter „timbre“ is required.
In the literature there is nearly no advice for selecting timbre according an

information-function.
Thus, an ongoing study that addresses the perception of timbre is topic of the
further presentation.

wew
02 Timbre and function

A. Timbre: definition & properties
B. Sound perception
C. Study design
A. Timbre: definition & properties B. Sound perception C. Study design

wew
A. Timbre: definition & properties

Timbre – what? Definition
Definition (American Standards Association 1960)
Timbre is that attribute of auditory sensation in terms of which
a listener can judge that two sounds similarly presented and having
the same loudness and pitch are dissimilar.
A B

Timbre properties
Verbal descriptions
overtone constellation: bright, dark, mellow, hollow, pure
noise content: raspy, breathy, hoarse
attack: smooth, abrupt, sharp, gentle, easing
Relations
pitch - frequency
loudness - amplitude
timbre - multiple interacting acoustic factors

Timbre properties
H. Fletcher (1934)
Experiments show that a simple one-to-one relationship does not exist between
the two sets { loudness, pitch, timbre } and { sound intensity, fundamental
frequency and overtone structure }
sound intensity loudness
fundamental frequency pitch
overtone structure timbre

Timbre properties
More than one dimension:
multidimensionality
Erickson (1975)
Clearly timbre is a multidimensional stimulus: it cannot be correlated with
any single physical dimension.
Plomp (1982)
Sounds cannot be ordered on a single scale with respect to timbre.
Timbre is a multidimensional attribute of the perception of sounds.

Multi-Dimensionality-Scaling
MDS
MDS studies reveal perceptual dimensions correlated with acoustic
parameters corresponding to spectral, temporal and spectrotemporal
properties of the sound events.
– experiments with paired sound stimuli

– rating of similarity
– axes corresponding to timbre‘s main perceptual dimensions –> timbre space

Dimensions
Dimension I = brightness
Low brightness : french horn and cello
High brightness : oboe, muted trombone Dimension I
Dimension II = spectral flux

High synchronicity and low fluctuation :
clarinet, saxophone
Low synchronicity and high fluctuation :
flute, violoncello
Dimension III = attack quality

More transients : strings, flute Dimension II sion III
Dim en
Fewer transients : brass, bassoon

Timbre dimensions and
related parameters
Across studies, several acoustic/physical parameters corresponding
to timbre dimensions were found:
– Spectral centroid – Spectral flux

– Spectral deviation – Pitch strength
– Spectral density – Attack synchronity
– Attack time – Attack centroid
– Decay time – Noisiness
–Amplitude envelope ...

wew
B. Sound perception
Timbre and psychoacoustic
properties
Interview Dr. Gerhard Thoma, chief sounddesigner BMW:
(http://www.goethe.de/ges/wrt/dos/aut/sou/de2169434.htm , 10/2007)
(...) There are sets of regression equations that help calculating/predicting

a desired sound perception, for example a sporty sound:
a x sharpness + b x roughness + c x fluctuation strength, … = sportiness.!!
About 80 % of people will evaluate the sound in terms of sportiness.

But:„acid test“in driving situation (context) = proof of validity!(...)

Psychoacoustic properties and
cognitive/affective evaluation
– simple psychoacoustic parameters only refer to fixed relations of perception and
instantaneous stimuli (Haverkamp 2007)
– psychoacoustics are only capturing parts of human sound perception, largely

excluding important affective + cognitive evaluation (Vjästfall/Kleiner 2002)
– any endeavour to evaluate (product) sounds by psychoacoustics solely will fail in

the end (Blauert/Jekosch 1997)

Sound perception:
target groups/context
– individual differences of preferences of particular groups of people for
qualitatively different product sounds
– no simple „universal“ manipulation that will have the same effect on the
sound quality of products (Spence & Zampini 2006)
–> target groups (age, cultural background, milieus/lifestyle) have to be taken

into account
–> it is important to consider the kind/type of product = context

Emotional evaluation and
functional content
Activation
tense alert
Model of affective circumplex
Emergency
nervous excited
(Russell, 1980)
stressed Warning elated
upset Advice happy

Unpleasant Confirmation Pleasant
sad contented
depressed serene
bored relaxed
Deactivation

Sound perception:
evaluation of sound (timbre)
To be taken in account:
– psychoacoustic properties of sound
– learned associations
– cultural background and personal experiences (target group)
– context: mood of subjects, expectations, situation
– influence from other modalities (vision, touch,...)

wew
C. Study design
Functional Timbre Study
(„sound-colour the Gestalt“)
VARIABLES (TIGA)
Independent:
Timbre T
Gestalt G [pitch contour, rhythm, tone duration]
Dependent:
Information-function I [function, e.g. emergency, warning, advice/confirmation]
Attribution A [emotional/affective evaluation: warm, harsh, sharp,...]

Sound-colour the Gestalt
Approach:
sound-colour T the Gestalt G and examine evaluations of subjects
regarding information-function I and attribution A

Example of stimuli
Same Gestalt, different timbre
A B

Study Design
Step 1: free association task
3 Gestalt x 8 Timbre = 24 sound-stimuli are evaluated by subjects
–> pool of attributes
Step 2: rating of stimuli

Rating of 24 stimuli on Likert scales of attributes obtained from
step 1 + assessment of given information function
Step 3: analysis of results

Principal-Component-Analysis, Generalized-Linear-Models.
Examination of correlations between acoustic properties, rated attributes,
information function.
–> Findings and results serve as a basis for further investigations!

Study Design Timbre selection
Timbre differing in parameter values („timbre dimensions“)
spectral content: bright dull/dim
temporal content: static dynamic static dynamic
attack/cutoff: hard soft hard soft hard soft hard soft

Study Design Gestalt creation
Typical forms of sound-gestalt (learned associations)
descending major third of a doorbell („ding dong“) (association: welcome)
wailing siren, in germany a fourth: a-d a-d (police or ambulance)
(association: emergency - alert)
Scherer/Osinsky 1977
Tonality: major mode: indicative of pleasantness and happiness
minor mode: disgust and anger
Rhythm: rhythmic: more active, fearful, surprised

nonrhythmic: boredom
Activity = Fast tempo, high pitch level, many harmonics, large pitch variation,
sharp envelope, small amplitude variation

Fields of expertise/collaboration
Relevant fields of expertise:
music psychology, sound-/acoustic-communication, sounddesign,
sound-/music-computing, statistical methods
–> interdisciplinary team
We are not the only ones:

Product Sound Design Group TU Delft
Music Technology Group Barcelona
Music, Mind and Machine Group (MIT Media Lab , USA)
Music Information Retrieval Community
–> exchange of findings, knowledge + possible collaborations

What is the use??
Product Sounddesign
Audio-Branding
AUI (Auditory User Interfaces)
Music Information Retrieval MIR (music catalogues, sound-databases)
Mpeg7-Classification
Electronic Music Distribution
„Perceptual Software-Synthesizers“
...

wew
Thank you!

Sound Follows Function

Enviado por

Dados do documento

Título original

Direitos autorais

Formatos disponíveis

Compartilhar este documento

Compartilhar ou incorporar documento

Opções de compartilhamento

Você considera este documento útil?

Este conteúdo é inapropriado?

Direitos autorais:

Formatos disponíveis

Sound Follows Function

Enviado por

Direitos autorais:

Formatos disponíveis

wew

Sound follows function

Lecturer: Rainer Hirt & Kai Bronner (audio-branding.de)

01 Acoustic signs and its functions

01 Acoustic signs and its functions

01 Acoustic signs and its functions

A. Basics B. Study C. Results

A sign stands for something else.

A. Basics B. Study C. Results

A. Basics B. Study C. Results

A. Basics B. Study C. Results

A. Basics B. Study C. Results

A. Basics B. Study C. Results

Symbol becomes Icon

A. Basics B. Study C. Results

Cultural Corporate Sound

Auditory Display Audiﬁcation Auditory-User-Interface

A. Basics B. Study C. Results

A. Basics B. Study C. Results

2. Effects of the mixture of icon and symbol („auditory symcon“)

3. Advantages of acoustic signs for „public-user-interfaces“

A. Basics B. Study C. Results

– Preliminary study (semantic ﬁeld-analysis)

A. Basics B. Study C. Results

A. Basics B. Study C. Results

Survey sample size : N=22

A. Basics B. Study C. Results

A. Basics B. Study C. Results

Advice: „Ticket ready for collection“ 1 2 3

Advice: „Money ready for collection “ 1 2 3

A. Basics B. Study C. Results

A. Basics B. Study C. Results

A. Basics B. Study C. Results

A. Basics B. Study C. Results

A. Basics B. Study C. Results

A. Basics B. Study C. Results

2. Effects of the mixture of icon and symbol („auditory symcon“)

A. Basics B. Study C. Results

Funtion(intention) partially transferred

Funtion(intention) not transferred or false interpretation

A. Basics B. Study C. Results

3. Advantages of acoustic signs for „public-user-interfaces“

A. Basics B. Study C. Results

3. Advantages of acoustic signs for „public-user-interfaces“

The meaning of function of acoustic signs is comprehended intuitively.

A. Basics B. Study C. Results

In the literature there is nearly no advice for selecting timbre according an

A. Basics B. Study C. Results

02 Timbre and function

A. Timbre: deﬁnition & properties B. Sound perception C. Study design

A. Timbre: deﬁnition & properties

A. Timbre: deﬁnition & properties B. Sound perception C. Study design

A. Timbre: deﬁnition & properties B. Sound perception C. Study design

sound intensity loudness

fundamental frequency pitch

overtone structure timbre

A. Timbre: deﬁnition & properties B. Sound perception C. Study design

A. Timbre: deﬁnition & properties B. Sound perception C. Study design

– experiments with paired sound stimuli

A. Timbre: deﬁnition & properties B. Sound perception C. Study design

Dimension II = spectral ﬂux

Dimension III = attack quality

A. Timbre: deﬁnition & properties B. Sound perception C. Study design