Escolar Documentos
Profissional Documentos
Cultura Documentos
net/publication/275144327
The Music Therapy Communication and Social Interaction Scale (MTCSI): Developing
a New Nordoff-Robbins Scale and Examining Interrater Reliability
CITATIONS READS
5 302
4 authors, including:
SEE PROFILE
Some of the authors of this publication are also working on these related projects:
writing a book on music making and learning with DIY recording studios View project
All content following this page was uploaded by Adam Patrick Bell on 20 December 2015.
ABSTRACT: Building on the Musical Communicativeness scale Building upon this foundation, NR clinicians and research-
(Nordoff & Robbins, 1977, 2007), the Music Therapy Communication ers identified a need to develop a new scale with better-
and Social Interaction scale (MTCSI; Guerrero et al., 2014), was devel- defined constructs that expand upon the core concepts of the
Table 1
Observable Behaviors of Engagement and Emotional Communication Measured by the MTCSI
Emotional
Engagement communication
Engagement (Socially Interactive Affective Physical
Preengagement or Communicative) Responses Responses
Attention Vocalization Instrument Transitions Joining Turn Reciprocal Musical Reciprocal Verbal
Use In Taking Communication Communication
Note. Transitions can be coded as Preengagement or Engagement. Emotional Communication is conceptualized as having two distinct compo-
nents, affective and physical responses.
organizational structure of the MTCSI. For a detailed descrip- pilot study, Guerrero & Turry (2013) reported promising pre-
tion of each construct measured by the MTCSI, please refer to liminary findings of the correlations between the domains of
Appendix A. the newly created MTCSI and those of the well-established
The first category is Engagement, or in its absence, Vineland-II, but no preliminary data on any form of reliability
Preengagement. Engagement is exhibited by communica- was reported. The purpose of this paper is to detail the devel-
tion and/or social interaction with peers or the therapists. To opment of the MTCSI and report the results of a recently com-
accompanied by their regular teachers and teaching assistants. observation exhibited the same quantity of Engagement behav-
Sessions were video-recorded from two different perspectives iors, Affective Responses, and Physical Responses on a minute-
in the room by the research assistants at the beginning, mid- by-minute basis. In the second phase, the “agreement” intervals
dle, and end of the 12-week-long therapy course. from the first phase (categorical) were further analyzed, again
Music therapy was provided in weekly 30-min group ses- as either “agreement” or “no agreement” on the basis of level.
sions. A typical TOTS music therapy session commenced with a This phase tested whether or not raters agree on the degree of
hello song and concluded with a goodbye song. Kern, Wolery, Engagement, Affective Response, and Physical Response exhib-
and Aldridge (2007) explain: “In music therapy, ‘hello’ and ited by the child in the music therapy session. For both phases
‘good-bye’ songs are used frequently to establish predictable of analysis, agreement was calculated and reported as a coef-
routines and structure, provide undivided attention, and com- ficient, as were the means and standard deviations of each cate-
municate a welcome” (p. 1265). These bookends are inter- gory to examine the consistency of the raters. A caveat of utiliz-
spersed with improvised “joining-in” and “turn-taking” activi- ing a percentage-based assessment of interrater reliability is that
ties. Kim, Wigram, and Gold (2009) posit: “Improvisational the scores are less robust than a kappa approach (Gwet, 2010),
music therapy is just such an individualized intervention that which accounts for agreement that may occur due to chance.
facilitates moment-by-moment motivational and interpersonal However, being that this was a preliminary investigation of
responses in children with autism” (p. 391). As a result, no interrater reliability with the aim of improving the functional-
two music therapy sessions at TOTS featured the same con- ity of the MTCSI, the simpler statistical procedure was chosen.
tent. While the overarching session structures were similar and Even with its inherent inflations, the data serves its purpose in
Results
Procedure Tables 2 and 3 present the agreement coefficients and
The MTCSI was designed with the intent that nonmusic accompanying means and standard deviations for Rater Team
therapists could understand and execute it with the goal that A and Rater Team B respectively.
its utility would extend into interdisciplinary collaborations For both teams, it is evident that categorical agreement coeffi-
and studies. To test how the constructs defined in the MTCSI cients were very high. The mean categorical coefficient scores for
are interpreted and scored by raters from different academic Team A were 0.93 (SD = 0.05) for Engagement, 0.88 (SD = 0.24)
fields, a preliminary examination of interrater reliability was for Affective Responses, and 0.91 (SD = 0.08) for Physical
undertaken. In the interest of fostering an interdisciplinary Responses. The mean categorical coefficient scores for Team
perspective on the constructs measured by the MTCSI, four B were 0.99 (SD = 0.02) for Engagement, 0.94 (SD = 0.09) for
raters from four different academic backgrounds (music edu- Affective Responses, and 0.97 (SD = 0.04) for Physical Responses.
cation, music therapy, psychology, & medicine) participated Despite these high coefficients, it should be noted that the stand-
as coders. In total, 24 video-recorded music therapy sessions ard deviation for Team A’s Affective Responses agreement coef-
were coded using the MTCSI and split evenly amongst two ficient was high, reflecting their inconsistent ratings for Session
teams of coders, Team A and Team B. Both coding teams were 2 of Child 4 (0.13). Both teams struggled to produce consistent
assigned four different children and tasked with coding three results for Engagement Level, with comparatively lower coeffi-
music therapy sessions per child. The eight TOTS students cients and higher standard deviations than the other constructs
assigned to the coding teams were selected randomly from assessed. The Engagement Level agreement coefficients for Child
the original pool of 36 participants. Coders were trained by an 2 were especially low (0.22, 0.38, & 0.29). In contrast, both rat-
experienced tutor before commencing the interrater reliability ing teams produced high agreement coefficients for Affective
study. The prestudy tutoring phase consisted of 3 hr of training Responses Level and Physical Responses Level. With regard to
in which the coder and tutor watched music therapy sessions Affective Response Level, Team A’s agreement coefficient was
filmed at TOTS and discussed the constructs the MTCSI aims 0.91 (SD = 0.08) and Team B’s agreement coefficient was 0.97
to measure: Engagement (or in its absence, Preengagement), (SD = 0.04). With regard to Physical Response Level, Team A’s
Affective Responses, and Physical Responses. agreement coefficient was 0.89 (SD = 0.10) and Team B’s agree-
Coding a 30-min TOTS music therapy session with the MTCSI ment coefficient was 0.99 (SD = 0.02).
is a labor-intensive process that typically requires 1 to 2 hr to
complete. Sessions are microcoded, requiring the rater to assign Discussion
a score for each minute of activity. After watching each minute With the exception of Engagement Level, interrater reliabil-
of video, raters assign scores in the appropriate categories. ity of the MTCSI’s categories are consistently high. The large
To assess interrater reliability, the rating teams watched the standard deviations reported for subspecific scales are indica-
video-recorded music therapy sessions together, but coded the tive of the fact that the sample size is small and one outlying
behaviors of the children independently. Raters were not per- result skews this statistic.
mitted to dialogue with each other while coding, other than to As a clinical tool, the MTCSI remains in a state of gestation in
request re-watching an interval or indicate that they were ready which it requires further testing and refinement. Including the
to move on with the next interval. Each of the three categories findings of this report and the earlier cited report of Guerrero
was analyzed in a two-phase process. In the first phase, cat- and Turry (2013), the MTCSI has shown promise to be both a
egorical agreement was examined as either “agreement” or “no reliable and valid measure of social interaction and communi-
agreement.” In this phase, the MTCSI was tested to see if two cation. Despite the authors’ optimism, there are a number of
independent raters using this instrument agree if the child under pragmatic concerns that make utilizing the MTCSI problematic.
64 Music Therapy Perspectives (2014), Vol. 32
Table 2
Agreement Coefficients for Rater Team A
Agreement coefficient
Engagement Affective Physical
Child Session Categorical Level Categorical Level Categorical Level
1 1 0.93 0.36 0.93 0.93 0.82 0.82
2 0.92 0.67 0.92 0.92 0.92 0.92
3 0.95 0.55 0.95 0.95 0.82 0.82
2 1 0.91 0.22 0.96 0.87 1.00 1.00
2 1.00 0.38 0.93 0.93 0.93 0.93
3 0.90 0.29 1.00 0.95 0.95 0.95
3 1 0.96 0.86 1.00 1.00 0.86 0.86
2 1.00 0.84 1.00 1.00 1.00 1.00
3 0.96 0.88 1.00 1.00 1.00 1.00
4 1 0.96 0.61 0.93 0.93 0.89 0.89
2 0.86 0.45 0.13 0.90 0.97 0.69
3 0.82 0.55 0.77 0.77 0.77 0.77
Table 3
Agreement Coefficients for Rater Team B
Agreement coefficient
Engagement Affective Physical
Child Session Categorical Level Categorical Level Categorical Level
5 1 1.00 0.93 1.00 1.00 1.00 1.00
2 1.00 0.97 1.00 1.00 1.00 1.00
3 0.95 0.90 0.95 1.00 1.00 1.00
6 1 1.00 0.89 0.93 1.00 0.93 1.00
2 0.96 0.71 0.96 0.96 0.96 1.00
3 1.00 0.92 0.96 0.96 0.96 1.00
7 1 1.00 0.67 1.00 0.92 1.00 1.00
2 1.00 0.66 0.92 1.00 1.00 1.00
3 0.96 0.58 0.96 1.00 0.92 1.00
8 1 1.00 0.73 1.00 1.00 1.00 1.00
2 1.00 0.71 0.69 0.83 1.00 0.92
3 1.00 0.48 0.95 1.00 0.91 1.00
Mean 0.99 0.76 0.94 0.97 0.97 0.99
SD 0.02 0.16 0.09 0.05 0.04 0.02
First, the training required to employ the MTCSI is very time- may result in a dearth in data generation from the indexing pro-
intensive, typically requiring trainees to spend 3 hr with an cess. As one trainee remarked, “the MTCSI forces you to recog-
experienced coder to guide them through the process. Second, nize that there is always something, even when it seems there is
once trained, the coding time required to completely analyze a nothing.” While not expected, the MTCSI has the added benefit
single session is approximately twice the real time of the video of challenging researchers and practitioners alike to consider
excerpt. Experienced coders can complete a session in real the seemingly diminutive events as well as the glaringly salient
time plus one-third to one-half of the video excerpt’s duration. events that occur within a music therapy session in order to
For example, the coders involved in the interrater reliability develop a comprehensive clinical picture of the client.
study typically completed a 20-min session in 30 min. The “agreement” versus “no agreement” approach utilized
The microcoding process contributes a systematic dimen- in this analysis is conservative, but eschews an analysis of rater
sion to the analysis of NR music therapy sessions, wherein each tendencies to be “hard” or “easy” coders. Therefore, an alter-
single minute necessitates analysis—this is not the case with native statistical approach such as calculating Cohen’s kappa
the qualitative indexing method, where subjective decisions could produce a more authoritative result for this preliminary
determine which parts of a video excerpt should be analyzed analysis of the interrater reliability of the MTCSI. Further, in
more closely. Marcus (personal communication, December an effort to improve the interrater reliability of the construct
15, 2013) suggests that because indexing is intended to probe Engagement Level in the MTCSI, heeding the feedback of
for clinical implications, a client’s unresponsiveness to therapy the raters, the instructional materials will be further refined
Music Therapy Communication and Social Interaction Scale 65
to remove ambiguities and provide more specific examples Kim, J., Wigram, T., & Gold, C. (2009). Emotional, motivational and interpersonal
of what to observe in an activity to make the coding process responsiveness of children with autism in improvisational music therapy.
Autism, 13(4), 389–409.
more concrete and less interpretive.
Mahoney, J. (2010). Interrater agreement on the Nordoff-Robbins Evaluation Scale
This quantitative approach to analysis is a double-edged 1: Client-therapist relationship in musical activity. Music and Medicine, 2(1),
sword of sorts, as despite its aforementioned advantages, it 23–38.
presents some philosophical problems for NRMT therapists. Nordoff, P., & Robbins, C. (1965). Improvised music for autistic children. Music
For example, the construct under most scrutiny and criticism Journal, 23(8), 39 & 67.
amongst the MTCSI’s authors is emotional communication, Nordoff, P., & Robbins, C. (1968). Improvised music as therapy for autistic children.
In E. T. Gaston (Ed.), Music in therapy (pp. 191–193). New York: MacMillan.
because it frames the acts of smiling or laughing as evidence of
Nordoff, P. & Robbins, C. (1977). Creative music therapy. New York: Harper & Row.
“positive” affect. Clients of NRMT display a broad range of emo- Nordoff, P., & Robbins, C. (2007). Creative music therapy: A guide to fostering clini-
tions that could be perceived as “positive,” and are not limited cal musicianship (2nd ed.). Gilsum, NH: Barcelona.
to smiles and laughs. The challenge for a non-NRMT-trained TOTS (n.d.). Retrieved from: http://www.theseourtreasures.com/services.html
coder is that these other displays of affect were often subtle and Trevarthen, C., Aitken, K., Papoudi, D., & Robarts, J. (1998). Children with autism:
required in-depth acquaintance with the client that only the Diagnosis and interventions to meet their needs (2nd ed.). London: Jessica
therapist possesses. The MTCSI was not designed to supplant Kingsley Publishers.
Tsatsanis, K. (2005). Neuropsychological characteristics in autism and related condi-
the indexing process, and should not be regarded as its quanti-
tions. In F. Volkmar, R. Paul, A. Klin, & D. Cohen (Eds.), Handbook of autism and
tative analog. Delimiting the affective behaviors accounted for pervasive developmental disorders: Vol. 1 (3rd ed., pp. 365–381). Hoboken,
by the “MTCSI – Group” confines coders, which for the purpose NJ: Wiley.
The behaviors coded in this section are typically seen early Vocalization codes begin with nonverbal sounds (e.g.,
in the course of music therapy. The coded behaviors reflect grunts, shrieks, vowel or consonant sounds), speech-
that the child is not yet actively participating in the session or related sounds (e.g., la la la), or words that are not in
is just beginning to participate in and attend to the session. At response to or directed toward peer(s), therapist, or ther-
times during the course of therapy, a child may revert to the apy objects (Code 1). The sounds are not in reaction to the
behaviors of this section. content of the session. Codes 2 and 3 represent the emer-
The second section of the observation scale is Section B: gence of vocalizations that are in response to peer(s), ther-
Engagement – Communication and Social Interaction with apist, or music, but are not apparently communicative.
Therapist or Peer(s). Categories in this section include: The behavior does not meet criteria for Joining In (Section
B1), although it may be a direct precursor to Joining In.
1. Joining In an Activity with Therapist or Peer(s)
2. Turn Taking Code Vocalization as:
3. Reciprocal Musical Communication with Therapist or
Peer(s) 1 – Makes sound or verbalization with no relation to ses-
4. Reciprocal Verbal Communication with Therapist or sion context.
Peer(s) 2 – Makes or alters sound or verbalization partially or
intermittently in response to session context.
The behaviors coded in this section indicate that the child 3 – Makes sound or verbalization in response to session
is engaged in the session, although the level of engagement context, but with no direct communication or social
1 –Inattention to session, e.g., paces around, rocks back Codes 2 and 3 of Joining In represent higher levels
and forth (not in relation to music); does not alter behav- or more frequent occurrences of communication and
ior in response to music; does not look at therapist or social interaction. Communication and social inter-
peer(s); plays with nonmusical object; does not relate action may be demonstrated by any of the examples
his/her activity to therapist, peer(s), or session context; of behaviors listed above. Codes 2 and 3 are applied
actively avoids therapy session. Self-soothing or self- depending on the context of the situation. If the child
stimulating behavior. has a brief moment of highly connected and commu-
2 –Intermittent attention to therapist, peer(s), or therapy nicative behavior within an interval but is otherwise
activities; attention alternating with inattention. Briefly merely attentive to the situation, coding may be divided
approaches therapist or peer(s) or orients body in between Code 3 of Joining In and Code 3 of Section
response to the session. A3: Attention. However, if the child’s communication
3 –Sustained attention to therapist, peer(s), and therapy or interaction is intermittent throughout the interval,
activities. Approaches or orients towards therapist or a code of 2 for Joining In can account fully for this
peer(s), but with no direct communication or social behavior, without any need to code under Section A3:
interaction. Attention.
Sidebar – Note persistent unresponsiveness of child. Code Joining In an Activity as:
Indicate any therapist or peer attempts to elicit response
from child. 1 –Simultaneous participation in an activity (i.e., musi-
cal, dramatic, or movement) along with therapist or
B. Engagement – Communication and Social peer(s), parallel play, with minimal communication
Interaction with Therapist or Peer(s) or social interaction. The child demonstrates minimal
In Engagement, the child is interacting or communicating affective response, physical response, or musical con-
with therapist or peer(s). The child may be interacting only nection (in dynamics, tempo, or melody) with therapist
with peer(s), only with therapist, or with both peer(s) and ther- or peer(s).
apist. If the child disengages from the interaction, communi- 2 –Simultaneous participation in an activity (i.e., musical,
cation, or session activity, indicate inattention using Code 1 dramatic, or movement) along with therapist or peer(s)
under Section A3: Attention. In children with developmental with intermittent communication or interaction. Child
disabilities, it may be common to alternate frequently between shows some affective response, physical response, joint
participation and disengagement. attention, or musical connection (in dynamics, tempo,
or melody) with therapist or peer(s).
1. Joining In an Activity with Therapist or Peer(s) 3 –Simultaneous participation in an activity (i.e., musi-
Joining In an Activity with Therapist or Peer(s) is often cal, dramatic, or movement) along with therapist or
the first actual sign of communication and social inter- peer(s) with strong or sustained communication or
action in a session, and is the usual entry point into interaction. Child shows strong affective response,
engagement behavior. Joining In codes begin with the physical response, joint attention, or musical connec-
child’s simultaneous participation in an activity (musi- tion (in dynamics, tempo, or melody) with therapist or
cal, dramatic, or movement) with therapist or peer(s). peer(s).
Such behavior is contrasted to performing an activity on Sidebar – Note attempts to join in, even if unsuccessful.
one’s own, which is a behavioral marker of Section A: Note therapist or peer assistance with play.
68 Music Therapy Perspectives (2014), Vol. 32
1 –Child does not respond, responds infrequently, or Sidebar – Note repetitious or habitual musical state-
responds only upon explicit prompting when it is his/ ments that do not invite any particular response and
her turn, OR does not wait for peer(s) to take a turn, are inflexible, uncommunicative, and unchanging in
plays out of turn, and may be distracted or disruptive response to session context.
(e.g., grabs instrument during peer[s]’ turn). The child
may appear to show attention to the group most of the 4. Reciprocal Verbal Communication with
time, but does not actively participate. Therapist or Peer(s)
2 –Child intermittently responds when it is his/her turn,
while at other times does not respond. Additionally, or Conversation that occurs within a therapy session.
alternatively, the child may at times be distracted or dis- Code 1 responses may include gesturing, and intelli-
ruptive. The child may appear to show attention to the gible or unintelligible words that are spoken or sung;
group most of the time and intermittently participate however, in Codes 2 and 3, comments or questions
actively. (spoken or sung) must be intelligible. If spoken or sung
3 –Child responds all the time or almost all the time words are not intelligible to the coder, the child can-
when it is his/her turn, AND waits for peer(s) to take not be coded 2 or 3. Instead, the behavior should be
a turn, does not play out of turn, and is not disruptive. described in a Sidebar under this category, with a note
The child shows attention to the group and actively of whether the verbalization appears to be directed at
participates. therapist or peer(s). Reciprocal Verbal Communication
is distinct from Vocalization (Section A1), which does
If there is a difference in the level of disruptiveness of not involve direct communication or social interaction
a child’s behavior depending on whether he is wait- with therapist or peer(s). Note: If the therapist is pro-
ing for his turn or taking his turn, code for the more viding a musical framework for response and the child
disruptive behaviors. For example, if the child takes responds verbally, code using B3: Reciprocal Musical
his turn at the proper times but is highly disruptive or Communication.
distracted between his turns, he would receive a code
of 1. Code Reciprocal Verbal Communication as follows:
Sidebar – Note nature and degree of therapist assistance 1 –Responds to question or comment by therapist or
or cueing (e.g., physical assistance—indicating it is the peer(s). Response may be through gestures (e.g., hand
child’s turn to play the drum by presenting the drum to wave or head nod), or intelligible or unintelligible spo-
the child or tapping the drum; or verbal assistance— ken or sung words.
“Leo, now it’s your turn!”). Indicate nature of any dis- 2 –Initiates intelligible comment or question in spoken or
ruptive behavior. sung words directed toward therapist or peer(s). Usually
Music Therapy Communication and Social Interaction Scale 69
implies a request for a verbal, musical, or behavioral by body movement, e.g., jumping up and down or clap-
response. ping hands in apparent happiness or excitement.
3 –Sustains conversation in spoken or sung words for at 3 –Shared or reciprocal display of affective response with
least two intelligible exchanges. the therapist or peer(s). Possibly accompanied by body
movement such as clapping along with a peer’s music
Sidebar – Note any disruptive, impulsive, perseverative, making.
or tangential verbalizations.
Sidebar – Describe nature and intensity of affective
C. Transition response. Note a child’s absence of affective response,
or flat affect.
A transition is the period after one music therapy activity has
ended and another has yet to begin. A transition is marked by
the clear stopping of one activity, followed by a period without 2. Physical Response to Therapist, Peer(s), or
organized activity (the transition), and the starting of another Therapy Session
activity. During a transition there may be no explicit directions A socially appropriate or positive physical response to
to the children as to what they should do. The therapist might therapy session, therapist, or peer(s). A behavior may
be silent, express reactions to the previous activity, query start out as socially appropriate but may be sustained
as to what the group would like to do next, sing about the in an inappropriate manner. In this case, the behavior
transition, or combine some of these behaviors. Code under would be noted as a Sidebar.
1. Affective Response The child’s overall affect during the therapy session was
Visible or audible display of emotion in response to ses- 1. Distressed, fearful
sion, therapist, or peer(s) – e.g., facial expressions indi- 2. Neutral, with little visible affect
cating joy, sadness, or anger; laughing, crying, or shout- 3. Occasional moments of positive affect
ing. Upon reviewing intervals of the session, the coder 4. Frequent moments of positive affect
should make a point of searching for affective responses, 5. Sustained positive affect
as they may go unnoticed in the initial viewing of an
interval. The child’s overall relationship with the therapist was
Code Affective Response as follows: 1. Detached, disengaged
2. Detached, with occasional engagement
1 –Displays affect, but not in response to therapy session 3. Intermittently engaged
and not directed at therapist or peer(s). 4. Frequently engaged, with occasional detached or dis-
2 –Demonstrates affective response to therapy session, but tracted behavior
not directed at therapist or peer(s). Possibly accompanied 5. Consistently engaged
70 Music Therapy Perspectives (2014), Vol. 32
The child’s overall relationship with peer(s) was la), jargon, or humming. (J. Birnbaum, personal communica-
1. Detached, disengaged tion, May, 2008).
2. Detached, with occasional engagement Perseveration 1. In general, persistence in doing something
3. Intermittently engaged to an exceptional level or beyond an appropriate point. 2. In
4. Frequently engaged, with occasional detached or dis- neuropsychology, the inappropriate repetition of behavior
tracted behavior that is often associated with damage to the Frontal Lobe of
5. Consistently engaged the brain. 3. An inability to interrupt a task or to shift from
one strategy or procedure to another. (VandenBos, 2007,
Definition of Terms p. 687–688).
Attention A state of awareness in which the senses are Reciprocal Musical Conversation A musical exchange that
focused selectively on aspects of the environment and the occurs between a child and the therapist or between a child
central nervous system is in a state of readiness to respond to and peer(s) with the following element of “conversation”: a
stimuli. (VandenBos, 2007, p. 82). directed musical statement (melodic or rhythmic sound or
Communication The transmission of information, which series of sounds, played or sung, with a clear beginning and
may be by verbal (oral or written) or nonverbal means. Humans end) that is answered with another musical statement.
communicate to relate and exchange ideas, knowledge, feel- Social Interaction Any process that involves reciprocal
ings, and experiences, and for many other interpersonal and stimulation of response between two or more individuals….
social purposes. (VandenBos, 2007, p. 200). Social interaction includes the development of cooperation