Elizabeth Hellmuth Margulis∗
and Andrew P. Beatty†
∗
201 Music Building,
University of Arkansas
Fayetteville, Arkansas 72701 USA
ehm@uark.edu
†
Pioneer Hi-Bred International
7200 NW 62nd Avenue, P.O. Box 184
Johnston, Iowa 50131 USA
apbeatty@gmail.com
Information theory as formulated by Shannon (1948)
has been intermittently pursued as a potential tool
for musical analysis; however, significant problems
have prevented it from coalescing into a fruitful
theoretical approach. Papers examining music from
the perspective of information theory are widely
distributed among journals from different fields
(Nettelheim 1997) and do not show a steady, cumu-
lative trajectory in time. Rather, a cluster of papers
in the late 1950s and 1960s (Pinkerton 1956; Meyer
1957; Youngblood 1958; Krahenbuehl and Coons
1959; Cohen 1962; Hiller and Fuller 1967), another
in the 1980s (Knopoff and Hutchinson 1981, 1983;
Snyder 1990), and a few contemporary studies
(Huron 2006; Temperley 2007) represent the primary
efforts.
Three challenges have hindered the development
of information-theory applications in music: first,
obscurities in the framing of the musical questions
that information theory might answer; second,
practical obstacles to tabulating the musical entities
needed for information-theoretic analysis; and third,
uncertainties regarding the type of musical entity
that should serve as the unit of analysis.
For music, information has been understood to
refer “to the freedom of choice which a composer
has in working with his materials or to the degree of
uncertainty which a listener feels in responding to
the results of a composer’s tonal choices” (Young-
blood 1958, p. 25). This conceptual framework
has led to studies that appropriate information
theory as a compositional tool (Pinkerton 1956),
as an identifier of musical style (Youngblood
1958; Knopoff and Hutchinson 1981, 1983), and
as a mechanism for the analytical comparison of
Computer Music Journal, 32:4, pp. 64–78, Winter 2008


c2008 Massachusetts Institute of Technology.
64
Musical Style,
Psychoaesthetics,
and Prospects for Entropy
as an Analytic Tool
sections within a piece (Hiller and Fuller 1967). It
is relatively straightforward to think about entropy
as a measure of the freedom with which musical
elements are strung together in a specific style,
but it is less clear how to conceptualize entropy’s
relationship with listener perceptions. What sorts
of statistics do listeners track, and for what sorts
of items? What are the behavioral, cognitive, or
affective consequences of perceiving high or low
entropy in a piece, style, or section?
In addition to the difficulty of establishing a
conceptual framework for relating entropy to music,
the practical obstacle of extracting and tabulating
the musical items whose entropy is of interest has
stymied progress in the musical application of in-
formation theory. Early researchers, such as Young-
blood, recorded frequencies by hand—a painstaking
process that necessitated a small sample from var-
ious styles (melodies from eight songs by Schubert,
six by Mendelssohn, and six by Schumann, in the
case of his study). Yet Knopoff and Hutchinson (1983)
showed that to disambiguate noise from statistically
significant characterizations requires pools of at
least 7,900 characters. Hand tabulating samples of
this size seems a quixotic enterprise, and this practi-
cal barrier may partially account for the decline of in-
terest in information theory as a music-analytic tool.
Large corpuses of musical data exist in three
commonly available forms: graphic (musical scores),
aural (sound recordings), and symbolic (e.g., MIDI).
Despite some effort (e.g., Poliner et al. 2007), no re-
liable technology yet exists to automatically extract
pitches, rhythms, and other musical quanta from
printed scores or sound recordings. Some excellent
tools exist for the manipulation of MIDI data (e.g.,
Eerola and Toiviainen 2004), but they are not yet
sophisticated enough to extract the units desirable
for information-theoretical analysis. Only one form
Computer Music Journal
of symbolic data (scores in the ∗∗ kern format) is
associated with a set of tools, the Humdrum Toolkit
(Huron 1993), that allows for the extraction and
manipulation of the kinds of data likely to be useful
in a perception-based account of entropy. Although
the amount of music encoded in the ∗∗ kern format
(4,290,411 notes in 20,318 files) is substantially
smaller than the corpus of music available in the
MIDI format, it is still large enough to permit
information-theoretic analysis (by the standards of
Knopoff and Hutchinson 1983); moreover, unlike
MIDI, the ∗∗ kern representation preserves many
elements present in actual scores: key signatures,
staves, and stem directions, for example. Although
Huron has used Humdrum to perform many
fascinating statistical analyses (e.g., Huron 2006),
no published study has yet used the toolkit for a
dedicated information-theoretic analysis.
In addition to the challenges inherent in
constructing a conceptual framework for the
information-theoretic analysis of music and in gain-
ing access to a format that permits the automatic
computation of information-theoretic measures, a
further challenge exists in selecting an available and
appropriate unit of representation (pitches, chords,
phrase segments, etc.) to successfully answer the
question posed by the conceptual framework.
Youngblood, interested in the differentiation of
musical styles, tabulated melodic pitches with
reference to the notated key; he coded the tonic as 1,
the supertonic as 3, and the raised-first and lowered-
second scale degree as 2 (reflecting their enharmonic
equivalence). But a slightly different perspective
could reconstitute the basic alphabet from which
elements are drawn. For example, melodies might
be understood to be composed not of twelve pitch
classes oriented around a tonic, but rather of raw
pitches, or of durations, or of scale degrees with
specific durations, or of pitches at specific metric
positions. Perhaps one composer makes more liberal
use of the raised-fourth scale degree than another,
but only at certain highly restricted metric posi-
tions. An information-theoretic analysis in which
scale degree and metric position were uncoupled
might result in a misleading account. Or, for the
purposes of illustration, consider a hypothetical
(albeit implausible) composer who wrote melodies
that could have been Mozart’s except for one
quality: The composer chose freely from all playable
registers, so that where Mozart might have written
C3, B2, A2, G2 (four adjacent pitches), this composer
wrote C3, B6, A1, G4 (the same pitch classes, but
wildly distributed among registers). In this case, an
information-theoretic account eliminating register
might miss important distinctions.
In the time since information theory has fallen
out of favor as a music-analytic tool, a new approach
has emerged in psychology and the learning sciences:
statistical learning (for a review, see Saffran 2003).
Statistical learning refers to the hypothesized ability
of learners to exploit statistical regularities in
their environment to abstract structure. Infants
have been shown to exploit statistical properties
to learn about their visual (Fiser and Aslin 2002),
linguistic (Saffran, Aslin, and Newport 1996), and
musical (Saffran et al. 1999) environments. Empirical
evidence that humans can track complex statistical
properties in numerous domains makes information
theory newly interesting from the perspective of
psychology and suggests that information theory
could be useful not merely in characterizing styles,
but also in addressing questions in music cognition.
This potential link between information theory
and music perception was made in a literature
contemporaneous with but largely peripheral to
music-theoretical work on the subject: the psy-
choaesthetics literature—concerned with empirical
investigations of aesthetics—epitomized by Berlyne
(1971) and also Smets (1973). Berlyne theorizes about
a link between “quantitative uncertainty” and “pref-
erences and emotional reactions” but is careful to
note that any relationship would be mitigated by
other subjective factors (p. 46). In summarizing the
work of Attneave (1954), he proposes that geometri-
cal configurations regarded as “good,” or regular and
well-organized, tend to be those that are identifiable
from only a small part of the pattern: those, Berlyne
explains, featuring “little uncertainty” (p. 46).
But the most substantial link Berlyne offers
between information theory and perception is me-
diated by the Wundt curve (see Figure 1), first
presented in 1874 by psychologist Wilhelm Wundt.
Margulis and Beatty 65
Figure 1. The Wundt
Curve, adapted from
Wundt (1874) and Berlyne
(1960).
Wundt used the curve to describe a relationship
between stimulus intensity and perceived pleasure;
according to the figure, neither too little nor too
much intensity are pleasurable, but a “sweet spot”
between these extremes—a stimulus of moderate
intensity—affords a pleasing experience. Berlyne
adapted the curve to describe the relationship not
merely between stimulus intensity and perceived
pleasure, but more broadly between “arousal poten-
tial” and perceived pleasure. Here, arousal potential
is defined as “the ‘psychological strength’ of a stim-
ulus pattern, the degree to which it can disturb
and alert the organism,” something partially predi-
cated on “properties such as novelty, surprisingness,
and complexity” (Berlyne 1971, p. 70). Accord-
ing to Berlyne’s adaptation, people prefer stimuli
of moderate novelty and complexity. Naturally,
what constituted novelty and complexity would
vary from person to person; for example, imagine
two hypothetical individuals listening to Pierrot
Lunaire—one a non-musician and one a Schoenberg
scholar. Many things that would be new, surprising,
and complex to the first listener would be less so
to the second. Nevertheless, Berlyne would argue
that his adapted Wundt curve could apply to both
of them; it would simply shift along the x-axis to
reflect the experience level of various listeners. The
Schoenberg scholar’s curve would shift to the right,
and the neophyte’s to the left. A similar shifting
could account for the effect of other factors, such as
personality traits.
66
Fred Lerdahl (1988) makes a related connection
between complexity and aesthetic value. He offers
two aesthetic claims: first, “the best music utilizes
the full potential of our cognitive resources” (i.e.,
the best music is to a certain extent complex), but
second, “the best music arises from an alliance
of a compositional grammar with the listening
grammar” (i.e., the complexity must be understand-
able, or capable of being processed—it cannot be
too complex or complex in the wrong way). These
claims can be thought of as related to Berlyne’s
claims about aesthetic perception centered around
the Wundt curve. Information theory can be under-
stood as relevant in this context insofar as entropy
might be a valid measure of novelty, surprise, or
complexity.
Accepting for the moment an “optimal com-
plexity” theory of aesthetic value, how might it
contribute to the development of the multiplicity
of styles in the Western musical tradition? The
pursuit of an explanation for stylistic change has
been a central enterprise in music theory (e.g.,
Ratner 1980; Meyer 1989; Temperley 2004). It is
commonly thought that as expressive deviations be-
come overused and thus normative, pressure exists
for new deviations to develop, fueling the evolution
of new musical styles. If Berlyne’s interpretation of
the Wundt curve is valid, then at some perceptually
significant level of analysis, all of these styles should
facilitate some optimal level of arousal potential.
Building on this premise, this article pursues an
investigation of entropy and musical style centered
around three assumptions: (1) arousal potential can
be approximated (although primitively) by entropy;
(2) different styles encourage attention to different
musical parameters; and (3) the optimal level of
entropy (arousal potential) might accordingly reside
in different parameters for different styles. Thus,
the central hypothesis of this study is that an ex-
amination of the entropy of different parameters
in different musical styles might reveal shifts from
style to style, with the highest entropy pooling
in particular parameters for particular styles. For
example, as entropy decreased in tonal content, it
might increase in rhythmic content, reflecting a
shift in orientation from the tonal to the rhythmic
in that particular style.
Computer Music Journal
Methods
Materials
Databases of scores in the ∗∗ kern format (see
kern.humdrum.net and www.musedata.org) repre-
senting eight styles were used: 38 Bach chorales, 48
Bach preludes and fugues from the Well-Tempered
Clavier (WTC) Books 1 and 2, 33 barbershop quar-
tets, 12 Corelli trio sonatas (including 43 individual
movements), 12 Handel trio sonatas (including 49
individual movements), 12 Telemann solo sonatas
for violin (including 49 individual movements),
5 Haydn string quartets (including 20 individual
movements), and 7 Mozart string quartets (includ-
ing 22 individual movements). These styles were
selected because the encoded database for each
was large enough to permit informative measures
according to Knopoff and Hutchinson (1983).
The parameters of interest were extracted from
the ∗∗ kern files using the Humdrum toolkit (Huron
1993) in conjunction with a series of custom Perl
scripts. The Humdrum toolkit and a separate
program written in C++ read in the data and
performed the statistical calculations.
Procedures
Eight parameters automatically calculable from the
∗∗
kern representation were selected for analysis. As
described earlier, selecting the alphabet from which
musical elements should be drawn is a complex
enterprise. What constitutes a musical element?
Candidates include (among many others) pitches,
durations, pitches with durations, rhythms, metric
positions, motives, chords, phrases, keys, contra-
puntal figures, schemas, and gestures. But music
is a hierarchically structured phenomenon, and
it can be observed that rhythms are composed of
durations within meters, and chords are composed
of pitches within contrapuntal figures. A reasonable
approach might be to extract the smallest decom-
posable elements first: pitches and durations. Larger
structures, however, are not straightforwardly built
out of smaller ones. In many cases, the identity of
an element deep within the hierarchy can constrain
or reconfigure the identity of an element closer to
the surface. For example, a C sounds like the tonic
within a C-major context. But the very prevalence of
C pitches may have been instrumental in generating
that sense of key. In a D-flat context, C functions
as the leading tone. If pitch is understood to signify
an event in a network of tonal relations (so that C
in a C-major context is not the same event as C in
a D-flat major context), then key, one important ar-
biter of tonal relatedness, is not only a phenomenon
emergent from the context of pitches, but also one of
the inputs feeding into the determination of pitches.
Or consider that the endpoints of phrases are
determined by the combination of pitches and
durations. Yet these endpoints serve to sever (or at
least suppress) the relationship between pitches and
durations before and after the endpoint (Margulis
2005). Such subtleties are not incorporated into
the parameter definitions of this exploratory study.
Parameters were chosen purely pragmatically in an
effort to capture as various a snapshot of musical
dimensions as possible within the constraints of the
measures that were automatically extractable from
the ∗∗ kern representation.
Pitch
The ∗∗ kern format represents pitches using letters,
sharps, and flats. Before analyzing pitch, one ab-
straction was employed. All letters and accidentals
were converted to numbers, eliminating the dis-
tinction between enharmonic equivalents (such as
C-sharp and D-flat) that denote the same frequency
in twelve-tone equal temperament. (Performance-
based nuances in pitch, such as a slight elevation in
frequency of C-sharp in comparison to D-flat, were
necessarily excluded.)
Registrally Centered Chromatic Scale Degree
(RCCSD)
Scale degree relates pitches to the referential tonic
pitch class. The registrally centered chromatic scale
degree (RCCSD) relates pitches to a specific tonic
pitch (versus an abstract tonic pitch class). For
pieces with the tonic C (i.e., those in C major or
C minor), middle C was considered to be 0. Pitches
Margulis and Beatty 67
above it were assigned to increasingly large positive
numbers, and pitches below it to increasingly large
negative numbers, in units of semitones. For pieces
in other keys, the tonic pitch closest to middle C
was reinterpreted as 0, and the same procedure was
followed. The sum effect of this transformation was
to abstract scale degree (a representation that relates
pitches to a tonal center) from raw pitch information
without eliminating register.
Chromatic Scale Degree (CSD)
The set of values for RCCSD was reduced modulo
12, producing a representation that preserved scale
degree but eliminated register. For example, in
D major, the RCCSD for the D next to middle
C was 0, and for the D an octave above middle
C it was 12. Under the chromatic scale degree
(CSD) representation, both Ds were represented as
0s. There were twelve possible CSD values, each
encapsulating a relation to the tonic; 0 represented
the tonic, 7 the dominant, 11 the leading tone, and
so on.
Two problems affected the encoding of the
RCCSD and CSD parameters. First, no allowance
was made for the modality. Therefore, repertoires
with a more even distribution between pieces in
major and in minor may have garnered misleadingly
large information measures for RCCSD and CSD.
For example, the minor third scale degree might
occur rarely in pieces in the major mode, but often
in pieces in minor. An even distribution between
pieces in the major and minor mode would tend to
even out frequency differences for versions of the
third, sixth, and seventh scale degrees, yielding a
higher H related simply to modality distribution in
the corpus. Second, no allowance was made for key
changes across the course of a movement (because
this was not straightforwardly calculable from the
∗∗
kern representation). Accordingly, repertoires with
more key changes might possess artificially inflated
information measures for the same two parameters.
Rather than allow key changes to confound these
parameters, it would be ideal to track key changes,
constantly adjusting RCCSD and CSD measures
to the local tonic. Key (and perhaps key duration)
could then function as an additional parameter for
68
analysis. However, both of these potential confounds
were at least partially addressed in the analysis, as
described in the Results and Discussion.
Texture
The texture measure examined the ∗∗ kern score at
every moment a new event occurred to determine
how many voices were active at that moment. The
resultant representation assessed the fluctuations in
texture across the course of the music.
Directed Melodic Intervals
The distance between adjacent pitches in the melody
was calculated to produce a list of melodic interval
successions. Ascending intervals were positive, and
descending intervals were negative.
Undirected Melodic Intervals
The undirected melodic intervals measure took
the absolute value of the numbers for the directed
melodic intervals, resulting in a measure of raw
(undirected) interval size. From the perspective of
this parameter, a leap of six semitones was the same
whether it moved up or down.
Contour
The contour measure reduced all positive directed
melodic intervals to 1, all negative directed melodic
intervals to –1, and assigned 0 to lateral motion
(i.e., pitch repetitions). Contour assessed the raw
(interval-size independent) directedness of the
melodic intervals. From the perspective of this
parameter, any move up would be equivalent,
whether it ascended by a semitone or an octave.
Duration
The rhythmic values of each melodic pitch—
including rests—were treated as a sequence of
elements. However, rests of a specific duration were
counted as separate elements from pitches of that
duration.
Computer Music Journal
Statistical Measures
For each parameter in each repertoire, the first-order
entropy, conditional entropy, and normalized en-
tropy were calculated according to the formulations
of Shannon (1948). First-order entropy H is defined as

constancy likely stems in part from the fact that it
H= − pi log2 pi (1)
i of Western tonality more than a property of any
where pi signifies the probability that the ith
symbol will occur, given i unique symbols from
which to choose. H reaches a maximum value when
each symbol is equally likely to occur (i.e., all pi ’s
are equal). This implies that musical pieces with
higher entropy values are those that have more
freedom in choosing their symbols.
For two events x and y, the conditional entropy is
defined as

changes within pieces. For example, the lower en-
Hx(y) = − p(i, j) log2 pi( j) (2)
i, j
where pi( j) signifies the probability that event y will
have value j, assuming x has value i. Conditional
entropy takes into account not just the likelihood of
a particular symbol, but also its likelihood given the
identity of the preceding one. For example, D may
be rather likely in a C-major piece, but especially
likely immediately after a C-sharp.
Normalized entropy is defined as
Hactual
Hnorm = (3)
Hmax
where Hmax signifies the maximum entropy for i
unique symbols. Normalized entropy assesses the
entropy of a particular corpus as a percentage of the
maximum entropy possible for a corpus with its
number of distinct symbols.
Results
First-Order and Conditional Entropy
Pitch-Based Measures
Table 1 lists the entropy value produced by each
parameter in each repertoire. As shown in Figure 2,
the first-order entropy for CSD changes less from
repertoire to repertoire than most other measures;
the standard deviation for CSD entropies is 0.10,
versus a mean standard deviation of 0.36 for all
parameters. Across all repertoires, the first-order en-
tropy for CSD is roughly 3.3. The CSD parameter’s
measures tonal relatedness, a property of the system
particular repertoire. Additionally, the repertoires
investigated in this study stem from a relatively nar-
row time period—namely, the 18th century (except
the barbershop quartets, which are relatively tradi-
tional in terms of tonal content)—and it is possible
that 19th- and 20th-century tonal repertoires would
produce different CSD values.
Another explanation for the similarity in CSD
readings implicates statistical noise created by key
tropy for the CSD parameter in the Bach chorales
may be attributable to the fact that they are shorter
and presumably modulate (change key) less fre-
quently than the other pieces. To investigate this
hypothesis, the frequency counts for each CSD class
were tallied in each repertoire. Figure 3 illustrates
the frequency counts for each CSD in the reper-
toire of Bach chorales and the frequency counts
for each CSD in the repertoire of Haydn string
quartets, which are longer and might be presumed
to modulate more frequently. Modulating amounts
to shifting 0 to a different spot on the x-axis. Be-
cause key changes are not accounted for in the
current measures, many modulations should result
in a more uniform distribution and thus a higher
entropy value.
First-order CSD entropies constitute a measure
of chromaticism—the extent to which a composer
draws freely from all twelve possible pitch classes.
The similarity between the distribution of frequency
counts for pitch classes in the two repertoires
shown in Figure 3 suggests that differences in
modulation frequency have not distorted the
measures. Moreover, the similarity between these
distributions and the key profiles of Krumhansl and
Kessler (1982) suggests that modulations were not
frequent enough to prevent the emergence of the
frequency patterns associated with the home key.
Margulis and Beatty 69
Table 1. First-Order, Conditional, and Normalized Entropies for Each Parameter in Each Style
Directed Undirected
Melodic Melodic
Pitch RCCSD CSD Texture Interval Interval Contour Duration

Bach Chorales
First-order 3.68 3.97 3.05 0.13 2.96 2.13 1.45 1.43
Conditional 2.57 2.31 2.28 0.13 2.49 2.07 1.44 1.19
Normalized 0.85 0.83 0.85 0.13 0.64 0.74 0.91 0.45
Bach Well-Tempered Clavier
First-order 4.31 4.43 3.37 1.67 3.51 2.65 1.38 2.42
Conditional 3.37 3.10 2.86 0.66 2.98 2.54 1.36 1.51
Normalized 0.84 0.84 0.94 0.60 0.59 0.67 0.87 0.49
Barbershop Quartets
First-order 3.33 3.89 3.34 0.62 2.80 2.23 1.56 2.47
Conditional 2.53 2.40 2.44 0.43 2.60 2.13 1.53 1.93
Normalized 0.73 0.81 0.93 0.27 0.56 0.59 0.98 0.62
Corelli Trio Sonatas
First-order 4.08 4.09 3.25 1.27 3.37 2.54 1.41 2.60
Conditional 2.97 2.79 2.62 0.76 2.85 2.44 1.39 1.85
Normalized 0.83 0.80 0.91 0.55 0.59 0.64 0.89 0.62
Handel Trio Sonatas
First-order 4.19 4.35 3.32 1.30 3.76 2.92 1.42 2.69
Conditional 3.36 3.22 2.93 0.76 3.18 2.77 1.39 1.58
Normalized 0.82 0.80 0.93 0.50 0.64 0.70 0.90 0.55
Telemann Violin Sonatas
First-order 4.12 4.36 3.31 0.67 3.99 3.07 1.27 2.43
Conditional 3.54 3.43 3.00 0.54 3.25 2.85 1.24 1.57
Normalized 0.87 0.84 0.92 0.42 0.75 0.70 0.80 0.54
Haydn String Quartets
First-order 4.84 4.72 3.33 2.15 3.76 2.93 1.46 3.08
Conditional 3.48 3.25 2.99 1.17 3.20 2.72 1.40 1.55
Normalized 0.88 0.85 0.93 0.68 0.58 0.65 0.92 0.60
Mozart String Quartets
First-order 4.48 4.53 3.23 1.94 3.70 2.86 1.42 2.92
Conditional 3.29 3.07 2.78 1.25 3.10 2.66 1.37 1.82
Normalized 0.86 0.82 0.90 0.56 0.61 0.67 0.90 0.61

Furthermore, the distribution of frequency counts
for CSDs between the Bach chorale and Haydn
string quartet repertoires are significantly correlated
(r = 0.95, p < 0.001); the frequency counts for CSDs
in all repertoires, in fact, are significantly correlated
(Cronbach’s Alpha [8 items] = 0.981). Although
first-order entropy measures suggest that soprano
lines from Bach’s Well-Tempered Clavier feature the
most chromaticism, and Bach chorale melodies the
least, the difference from repertoire to repertoire is
minimal. First-order CSD entropies seem to reflect
properties of the tonal system rather than properties
of individual styles.
Styles are better distinguished from one another
by conditional entropies for CSD (see Figure 4).
In Bach chorale melodies (which were not usually
written by Bach; he set preexisting church tunes), the
conditional entropy for CSD is quite low, indicating
that given a pitch x, certain other pitches y and z are
likely to follow, and certain other pitches a and b are
unlikely to follow. Barbershop quartets also possess
a relatively low value for the conditional entropy

70 Computer Music Journal

Figure 2. First-order
entropies for chromatic
scale degree (CSD).
Figure 2
of CSD. It is noteworthy that, of the repertoires
studied here, only these two feature vocal melodies.
The voice is more limited in its range and mobility
than other instruments; it is easier to play a leap
on the violin than to sing one. This could partially
account for the comparatively low joint entropies for
Bach chorales and barbershop quartets for not only
the parameter of CSD, but also the parameters of
pitch, RCCSD, and directed and undirected melodic
intervals. Other repertoires showed freer transitions
from one note to the next, particularly Haydn string
quartets and Telemann violin sonatas.
The CSD measure differs from the pitch measure
by acknowledging octave equivalence (transforming
pitch to pitch class) and acknowledging tonal
center (transforming pitch class to scale degree).
The pitch measure, taken before any of these
transformations, reveals radically different entropies
from repertoire to repertoire that disappear when
the transformations into the CSD measure are made
(see Figure 5). As discussed earlier and as shown in
Figure 2 and Table 1, first-order entropies for CSD in
all the repertoires cluster around 3.3, but first-order
entropies for pitch range from 3.3 (in the barbershop
quartets) to 4.8 (in the Haydn string quartets).
Entropies for pitch reflect two basic characteristics:
the diversity in keys among the pieces making up
a repertoire, and the range employed. For example,
the vocal repertoires (Bach chorales and barbershop
quartets) are limited by the range of the human
voice, and both exhibit lower pitch entropies than
any other repertoire. Between those two repertoires,
Bach chorales seem to have a higher pitch entropy
because they are written in a wider variety of keys
than the barbershop quartets. As reflected by the
Figure 3. Frequency of each
CSD in the repertoire of
Bach chorale melodies and
Haydn string-quartet
melodies as a percentage
of total number of pitches.
Figure 3
CSD values (see Figure 4), when these factors are
accounted for, the barbershop quartets actually
reveal a higher entropy than the Bach chorales.
Pitch entropies for the string quartets and for the
music from the Well-Tempered Clavier are highest
because their instruments permit movement across
a wide range, and because the individual pieces
are written in a wide variety of keys. The measure
for RCCSD controls for tonal center (like the CSD
measure), but also for range. Under this measure,
the entropies for the barbershop quartets and the
Bach chorales come into much closer alignment.
Duration
Entropy measures for the parameter of duration are
lowest for the Bach chorale melodies (see Figure 6).
These melodies are largely isochronous, simplifying
synchronization for a congregation of non-expert
singers. First-order entropy for durations in Haydn
and Mozart string quartets is higher than first-order
entropy for duration in other repertoires (see Table 1)
because Haydn and Mozart string quartets use a
Margulis and Beatty 71
Figure 4. Conditional Figure 5. First-order
entropies for CSD. entropies for pitch.
Figure 4
Figure 5
Figure 6
larger repertoire of rhythmic elements, including
32nd-note rests, etc. However, once a particular
rhythmic element is employed in a Haydn string
quartet, its continuation is more closely prescribed
than in a Mozart string quartet; although Haydn
string quartets have a slightly higher first-order
entropy than Mozart string quartets, the conditional
entropy of Mozart string quartets is notably higher
than of Haydn. Melodies in the string quartets of
Mozart and Haydn, in other words, draw from a
comparably wide range of durations, but Haydn
melodies seem to use those durations in more
72
Figure 6. First-order
entropies for duration.
closely regimented, predictable ways. The entropy
values for CSD, contrastingly, are higher for Haydn
than for Mozart, suggesting that Haydn may have
been more flexible with pitch and Mozart with
rhythm in the string-quartet genre.
Contour-Based Measures
With respect to the contour parameter, Telemann’s
first-order and conditional entropies are notably
lower than other repertoires, and entropies for
barbershop quartets are notably higher. The
Computer Music Journal
Figure 7. Ratio of Figure 8. Lateral motion as
ascending to descending a percentage of all melodic
melodic motion in each motion in each repertoire.
repertoire.
Figure 7
Figure 8
contour parameter takes three possible values:
ascending, descending, and lateral. The distribution
of ascending and descending motion is remarkably
consistent from repertoire to repertoire; in every
case, ascending motion is slightly more common
than descending motion (mean ratio of ascending to
descending motion = 1.17, SD = 0.08; see Figure 7).
The differences emerge in the use of lateral motion
(see Figure 8). For all repertoires except Telemann
and barbershop quartets, between 11 percent and
15 percent of motion is lateral, but in Telemann,
only 6 percent of melodic motion is lateral, and
in barbershop quartets, fully 42 percent of it is.
The use of repeated notes, in other words, proved
a better stylistic marker for these repertoires than
other aspects of melodic contour. This is interesting
in light of the ambiguous place repeated notes have
had in theories of melody (Narmour 1990).
But contour is analyzable not only in terms of
whether the melody ascends, descends, or moves
horizontally; it is also analyzable in terms of the
size of the leaps and dips (see Figure 9). The vocal
repertoires (Bach chorales and barbershop quartets)
have lower entropies than other repertoires for
directed and undirected melodic intervals; limits of
vocal production proscribe large leaps. Telemann
violin sonatas are distinguished by the highest
entropy when interval size is taken into consider-
ation, despite the fact that they are distinguished
by the lowest entropy when only the direction of
interval is considered. Telemann violin sonatas,
in other words, freely move from one interval
size to another, even as they are relatively more
constrained in terms of whether that movement
proceeds up, down, or laterally. The relative order
of entropies in the various repertoires remained
the same (or in one case—first-order entropies for
Bach chorales and barbershop quartets—nearly the
same) whether contour was measured in terms of
undirected or directed intervals. There did not seem
Margulis and Beatty 73
Figure 9. First-order Figure 10. First-order
entropies for directed and entropies for texture.
undirected melodic
intervals.
Figure 9
Figure 10
to be a systematic difference, in terms of entropy, in
composers’ treatments of ascending intervals versus
descending ones of the same size.
Texture
Unsurprisingly, because the genre is almost com-
pletely texturally uniform, the Bach chorales possess
by far the lowest textural entropy: 98 percent of the
time, all four voices are participating, and 2 percent
of the time, all four voices are resting (see Figure 10).
The entropy for texture in the Barbershop quartets
is the next lowest: 90 percent of the time, all four
voices are participating, and 2 percent of the time,
all four voices are resting. An additional 6 percent of
the time, the texture thins to just one voice, and for
the remaining 2 percent of the time (approximately
1 percent each), the texture thins to two or three
voices. But these textures are rare and particularly
salient when they occur. The entropy for texture in
the Telemann violin sonatas is similar to that for
barbershop quartets but here reflects the relative
constancy of the violin-continuo texture—dropping
to one of the two parts only 1 percent of the time,
74
and resting in both parts less than 1 percent of the
time. However, this measure reflects only the no-
tated bass line and not the fuller texture that would
be realized by the continuo performer improvising
upon the figured bass.
Haydn string quartets, on the other hand, possess
the highest first-order textural entropy, followed
closely by Mozart string quartets. In both these
repertoires, the available textural configurations are
drawn from more evenly, with many instances where
all voices are participating, but many instances
where only some of them are. Interestingly, when
conditional entropy is considered, Mozart string
quartets supersede the Haydn string quartets for
a slightly higher textural entropy. Haydn seems
to use different textures slightly more frequently
than Mozart in his string quartets, but Haydn uses
them in more standardized ways; however, Mozart
uses different textures slightly less frequently but in
more various ways. Similarly, the first-order textural
entropy for Bach’s Well-Tempered Clavier is quite
high (although not as high as the string quartets),
but its conditional textural entropy is notably low,
dropping below the values for even other repertoires
Computer Music Journal
with substantially lower first-order entropies. This
reflects the strict treatment of texture in the fugues;
most often one voice is followed by two, which is
followed by three, and so on as the voices enter
systematically. So the progression from texture
to texture may be more predictable in the Well-
Tempered Clavier, but this is very different from the
chorales, in which no other textures aside from full
or absent were employed.
Normalized Entropy
The entropies reported in the previous section
express the freedom with which the various
repertoires move among the available options for
each parameter; however, the alphabet of available
options is not uniform from repertoire to repertoire.
For example, the largest separation between two
consecutive melodic pitches in Bach chorales is 12
semitones, but in Haydn string quartets, it is 39
semitones. The larger alphabet of available options
makes the entropies for Haydn string quartets
higher, even if the Bach chorales draw relatively
more evenly from their smaller alphabet. Normal-
ized entropy, by contrast, calculates the entropy
exhibited by a particular repertoire as a percentage
of the maximum entropy possible in a system
with that size of alphabet. In other words, given its
smaller alphabet, normalized entropy can determine
whether Bach chorales explored that alphabet more
freely than Haydn string quartets explored its
larger alphabet. Perceptually, normalized entropy
could reflect the stylistic knowledge that vocal
melodies will draw from a smaller range of pitches
than instrumental melodies and the corresponding
shift in focus to the freedom with which that
range is explored. Or, in the case of a violin sonata
and a string quartet, normalized entropy would
acknowledge the more limited textural options in
the first genre and characterize the freedom with
which available textural configurations were used.
The normalized entropies for CSD remain consis-
tent with the entropies reported above, because the
alphabet for each repertoire is the same: the twelve
available pitch classes in the chromatic scale. On
average, the repertoires use about 91 percent of the
available entropy for an alphabet of that size, again
with the notable exception of the Bach chorales,
the diatonicism of which resulted in a normalized
entropy of only 85 percent.
Although accounting for the size of the available
alphabet preserves the pattern of CSD entropies, it
alters the pattern for pitch entropies. Bach chorales
exhibit the second-lowest first-order entropy for
pitch (after barbershop quartets), but their normal-
ized entropy for pitch jumps to fourth-highest. In
other words, although the range for chorale melodies
is small, the melodies make relatively free use of
that range. The same transformation occurs for
RCCSD; Bach chorales possess a much higher en-
tropy once the measure has been normalized to
account for the small size of the alphabet.
The normalized entropy measure for texture
distinguishes better between similar repertoires than
first-order entropy. For example, the normalized
entropy for texture shows much higher textural
freedom in Haydn than in Mozart string quartets,
and higher textural freedom in Corelli than in
Handel trio sonatas. The repertoires whose standings
fell when entropy was normalized seem to have
used a unique texture rarely for special effect. The
repertoires whose standings rose used the more
common textures more evenly.
Trends for undirected and directed melodic
intervals changed similarly when the measures
were normalized. Most notably, for each of these
interval measures, normalizing them elevates Bach
chorales from one of the lowest entropies to the
highest. This change seems to reflect the particular
rarity of rare intervals in chorale melodies; for
example, in the chorale melodies there are only
two instances of intervals that appear fewer than
five times in the corpus (leaps of nine and twelve
semitones), but in Haydn string quartet melodies,
there are twelve instances (leaps of 21, 22, 23, 25,
27, 29, 31, 32, 34, 38, 39, and 41 semitones)—
mostly occurring across phrase boundaries or in
fugal textures, when the melody moves from part
to part. This reflects a problem in the measure
of normalized entropy when applied to musical
styles: If a repertoire does something ever—even
only once—this becomes a member of the alphabet
and a contributor to maximum entropy. If a style
draws freely from many other options but not from
Margulis and Beatty 75
that one, than it is punished with a low normalized
entropy, because that rare option has elevated the
maximum entropy so much higher. Rare occurrences
elevate entropy but diminish normalized entropy.
Because all repertoires had three contour options
to choose from—ascending, descending, or lateral
motion—normalizing the entropy measure made no
difference; all repertoires were measured against the
maximum entropy of a three-choice system.
The order of durational entropies roughly follows
chronology, with the two repertoires of J. S. Bach (b.
1685) the lowest, then Telemann (b. 1681), Corelli
(b. 1653) and Handel (b. 1685), and then Mozart (b.
1756) and Haydn (b. 1732), suggesting that rhythmic
freedom might be a key indicator of stylistic
evolution. But the barbershop quartet (a style
that coalesced in the middle of the 20th century)
interrupts this pattern, exhibiting a durational
entropy that places it between Telemann and
Corelli, about three hundred years too early. But
when normalized durational entropy is considered,
barbershop quartets fall into their appropriate
chronological place after Haydn and Mozart. The
chronological exception in this reading is Corelli,
whose music now supersedes both the 18th-century
string quartets and the 19th-century barbershop
quartets in normalized durational entropy. Once
again, this appears to be a consequence of Corelli’s
avoidance of rare durations; whereas Corelli uses
only one durational value fewer than five times
in the entire corpus, Haydn uses eight durational
values fewer than five times. These rare durations in
the Haydn corpus elevated the maximum possible
entropy for duration and correspondingly reduced
the normalized entropy.
General Discussion and Future Work
Entropy, when applied to individual musical pa-
rameters, can indeed capture rough characteristics
of musical styles. To a certain extent, the central
hypothesis that most styles would “specialize” in a
certain parameter, concentrating much of their en-
tropy in a particular dimension, was partially proven.
Bach chorales exhibited the highest normalized
entropy for melodic intervals; the Well-Tempered
76
Clavier showed the highest normalized entropy for
contour but low normalized entropies for other pa-
rameters. In contrast, barbershop quartets exhibited
the highest normalized entropy for duration but the
lowest normalized entropy for melodic intervals.
The string quartet genres exhibited the highest
normalized entropy for texture. However, there
seemed to be another factor at work: As time passed
between the historical origins of styles, entropy in
most parameters seemed to gradually increase. In
other words, music from the second half of the 18th
century exhibited generally higher entropies than
music from the first half of the century in most pa-
rameters. When only the repertoires with composers
born in the 17th and 18th centuries were considered
(i.e., when barbershop quartets were excluded),
the average first-order entropy for each repertoire
across all parameters is significantly correlated
with the order of composer birth dates (r = 0.862,
p < 0.05). The average normalized entropy for each
of these repertoires is also significantly correlated
with the order of composer birth dates (r = 0.757,
p < 0.05).
The finding of a gradual accrual of entropy
across time is consistent with standard accounts
of stylistic evolution (by which deviations become
gradually standardized through overuse), and the
finding regarding the concentration of entropy in
particular parameters is consistent with standard
psychoaesthetic accounts of listening, according
to which people prefer some optimal level of
complexity. This optimal level would simply be
housed in different parameters from repertoire
to repertoire. However, this model assumes that
this optimal level of complexity is a feature of a
particular parameter for an entire corpus. A more
realistic account may be that different amounts
of complexity are housed in different parameters
at different points within an individual piece, and
that listeners fluctuate their attention according
to these changes. But an investigation of this
hypothesis would require a more dynamic measure
than entropy.
As discussed in the introduction, a problem with
the use of entropy in connection with music is the
selection of the alphabet from which events are
drawn. The rare-interval problem that complicated
Computer Music Journal
measures of normalized entropy is emblematic of
this complication. It is also possible that even if a
repertoire never uses a particular token (a leap of
a major seventh, for example), that listeners still
identify this as a possibility. Entropy as defined here,
however, considers only tokens that are represented
within a particular corpus.
A related problem is whether an individual
corpus can be equated with psychological reality.
Do listeners evaluate Haydn’s choices in light of the
choices he made in his string quartets (the measure
employed here), or in all of his works, or in all
18th-century music, or in all tonal music? This
decision might depend on the prior experience and
stylistic knowledge of the listener. Cross-entropy,
which incorporates probabilities for each symbol
based not on the corpus under evaluation, but
on some kind of model (which could represent
the expectations of the listener) might serve as a
more psychologically valid measure. For example,
cross-entropy could identify the predictability of
Haydn’s music for a listener trained on 18th-century
music by other composers. Probabilistic methods—
Bayesian reasoning in particular—have recently been
applied to a range of musical problems (Sadakata,
Desain, and Honing 2006; Temperley 2007). Some
of these models provide elegant ways of modeling
cross-entropy and offer powerful potential for
measuring perceived complexity and expectation.
The multiple-viewpoints approach (Conklin and
Witten 1995; Pearce and Wiggins 2006) also possesses
potential in adapting information-theoretic ideas
to more complex and psychologically realistic
problems.
Whatever the approach, it remains challenging
to identify parameters that both have psychological
reality and are automatically analyzable from avail-
able databases. For example, it would be interesting
to study entropy in harmonic progressions, but
until better tools for automatic harmonic analysis
become available, there will be practical barriers to
implementing such a study.
Future work might concentrate on providing bet-
ter automation tools to permit more sophisticated
choices of musical parameters for analysis. It also
might focus on using categorization studies to bet-
ter understand which corpuses have psychological
reality. For example, what repertoires do listeners
perceptually link together? But in conjunction with
these technological, methodological, and theoretical
developments, it is conceivable that entropy could
reemerge as a useful tool in musical analysis.
References
Attneave, F. D. 1954. “Some Informational Aspects of Vi-
sual Perception.” Psychological Review 61(3):183–193.
Berlyne, D. E. 1960. Conflict, Arousal, and Curiosity.
New York: McGraw-Hill.
Berlyne, D. E. 1971. Aesthetics and Psychobiology. New
York: Appleton-Century-Crofts.
Cohen, J. E. 1962. “Information Theory and Music.”
Behavioral Science 7(2):137–163.
Conklin, D., and I. H. Witten. 1995. “Multiple Viewpoint
Systems for Music Prediction.” Journal of New Music
Research 24(1):51–73.
Eerola, T., and P. Toiviainen. 2004. “MIDI Toolbox:
MATLAB Tools for Music Research.” Kopijyvä,
Jyväskylä, Finland: University of Jyväskylä. Available
online at www.jyu.fi/musica/miditoolbox.
Fiser, J., and R. N. Aslin. 2002. “Statistical Learning
of New Visual Feature Combinations by Infants.”
Proceedings of the National Academy of Sciences
99(24):15822–15826.
Hiller, L., and R. Fuller. 1967. “Structure and Information
in Webern’s Symphonie, Op. 21.” Journal of Music
Theory 11(1):60–115.
Huron, D. 1993. “The Humdrum Toolkit.” Available
online at music-cog.ohio-state.edu/Humdrum/index.
html.
Huron, D. 2006. Sweet Anticipation: Music and the
Psychology of Expectation. Cambridge, Massachusetts:
MIT Press.
Knopoff, L., and W. Hutchinson. 1981. “Information
Theory for Musical Continua.” Journal of Music
Theory 25(1):17–44.
Knopoff, L., and W. Hutchinson. 1983. “Entropy as a
Measure of Style: The Influence of Sample Length.”
Journal of Music Theory 27(1):75–97.
Krahenbuehl, D., and E. Coons. 1959. “Information as a
Measure of Experience in Music.” Journal of Aesthetics
and Art Criticism 17(4):510–522.
Krumhansl, C. L., and E. J. Kessler. 1982. “Tracing the
Dynamic Changes in Perceived Tonal Organization in a
Spatial Representation of Musical Keys.” Psychological
Review 89(4):334–368.
Margulis and Beatty 77
Lerdahl, F. 1988. “Cognitive Constraints on Com-
positional Systems.” In J. Sloboda, ed. Generative
Processes in Music: The Psychology of Performance,
Improvisation, and Composition. Oxford: Oxford
University Press, pp. 231–259.
Margulis, E. H. 2005. “A Model of Melodic Expectation.”
Music Perception 22(4):663–714.
Meyer, L. 1957. “Meaning in Music and Information
Theory.” Journal of Aesthetics and Art Criticism
15(4):412–424.
Meyer, L. 1989. Style and Music: Theory, History, and Ide-
ology. Philadelphia: University of Pennsylvania Press.
Narmour, E. 1990. The Analysis and Cognition of Basic
Melodic Structures. Chicago, Illinois: University of
Chicago Press.
Nettelheim, N. 1997. “A Bibliography of Statistical
Applications in Musicology.” Musicology Australia
20:94–106.
Pearce, M. T., and G. A. Wiggins. 2006. “Expectation
in Melody: The Influence of Context and Learning.”
Music Perception 23(5):477–405.
Pinkerton, R. C. 1956. “Information Theory and Melody.”
Scientific American 194(2):77–86.
Poliner, G. E., et al. (2007). “Melody Transcription
from Music Audio: Approaches and Evaluation.”
IEEE Transactions on Audio, Speech, and Language
Processing 15:1247–1256.
Ratner, L. 1980. Classic Music: Expression, Form, and
Style. New York: Schirmer.
78
Sadakata, M., P. W. M. Desain, and H. Honing. 2006.
“The Bayesian Way to Relate Rhythm Perception and
Production.” Music Perception 23(3):267–286.
Saffran, J. 2003. “Statistical Language Learning:
Mechanisms and Constraints.” Current Directions in
Psychological Science 12(4):110–113.
Saffran, J., R. N. Aslin, and E. L. Newport. 1996.
“Statistical Learning by 8-Month-Old Infants.” Science
274:1926–1928.
Saffran, J., et al. 1999. “Statistical Learning of Tone
Sequences by Human Infants and Adults.” Cognition
70(1):27–52.
Shannon, C. E. 1948. “A Mathematical Theory of Commu-
nication.” Bell Systems Technical Journal 27:379–423.
Smets, G. 1973. Aesthetic Judgment and Arousal. Leuven,
Belgium: Leuven University Press.
Snyder, J. L. 1990. “Entropy as a Measure of Musical
Style: The Influence of A Priori Assumptions.” Music
Theory Spectrum 12(1):121–160.
Temperley, D. 2004. “Communicative Pressure and
the Evolution of Musical Styles.” Music Perception
21(3):313–337.
Temperley, D. 2007. Music and Probability. Cambridge,
Massachusetts: MIT Press.
Wundt, W. M. 1874. Grundzüge der Physiologischen
Psychologie [Foundations of Physiological Psychology].
Leipzig: Engelmann.
Youngblood, J. E. 1958. “Style as Information.” Journal of
Music Theory 2(1):24–35.
Computer Music Journal