Psychotherapy © 2012 American Psychological Association

2012, Vol. 49, No. 3, 276 –290 0033-3204/12/$12.00 DOI: 10.1037/a0029564

A Meta-Analysis of Psychodynamic Psychotherapy Outcomes:

Evaluating the Effects of Research-Specific Procedures

Joel M. Town Marc J. Diener

Dalhousie University Long Island University-CW Post

Allan Abbass Falk Leichsenring

Dalhousie University University of Giessen

Ellen Driessen Sven Rabung

VU University of Amsterdam University Medical Center Hamburg-Eppendorf, and
Alpen-Adria-Universität

The aim of this research was to examine the extent to which the use of research-specific procedures in
psychodynamic psychotherapy impacts upon treatment effectiveness and which variables moderate this
potential relationship. Effects of audio/video recording of sessions, use of treatment manuals, and checks
of treatment fidelity were examined. A meta-analysis was conducted on randomized controlled trials of
psychodynamic psychotherapy. Forty-six independent treatment samples totaling 1615 patients were included.
The magnitude of change between pretreatment and posttreatment aggregated across all studies (45 treatment
samples) for overall outcome was large (d៮ ⫽ 1.01), and further improvement was observed between
posttreatment and an average 12.8-month follow-up (d៮ ⫽ 0.18). Subgroup analyses comparing studies that
used research-specific procedures and those that did not revealed that for posttreatment data no differences in
treatment effects were found. However, the use of treatment manuals and fidelity checks were significantly
associated with improvement between the end of treatment and follow-up assessment. Within the limitations
of analyses, this data offered preliminary evidence that use of research-specific procedures does not contribute
in a negative manner to posttreatment outcomes in psychodynamic psychotherapy, and their use contributes
to positive differences that emerge with time. These findings, although observational in nature, make a case
for reconsidering how dimensions of clinical utility and experimental control may be integrated in psychody-
namic psychotherapy to enable further elucidation of principles that evidently work.

Keywords: psychodynamic, psychotherapy, meta-analysis, research, moderator

Supplemental materials: http://dx.doi.org/10.1037/a0029564.supp

Psychotherapy outcome research has evolved to regard the need for
adequate specificity and standardization of psychotherapy treatments
as essential, despite limitations in the generalizability of the controlled
trial methodology (e.g., Blatt & Zuroff, 2005; Roth & Fonagy, 2005).
Nevertheless, having a true and accurate picture of the nature of a
treatment delivered is relevant for controlled research, naturalistic
studies, and practice-based evidence more generally to enable a valid
assessment of effectiveness. Psychotherapy research and to a lesser
extent clinical and training facilities, therefore, now commonly use
manual-based treatments, audio/video recording of sessions, and for-
mal checks on treatment fidelity. From here on, audio/video record-
ing, treatment manuals, and fidelity checks, as a collective are com-
monly described as research-specific procedures.
Changes in clinical practice and psychotherapy training (Crits-
Christoph, Frank, Chambless, Brody, & Karp, 1995) can be traced to
efforts to incorporate these research methods more systematically,
moving away from the historical position of less structured methods
and theoretical texts lent from psychoanalysis (Matarazzo & Garner,
1992). However, psychodynamic clinicians and teachers, past and
present, continue to vary widely in both their attitudes toward these

Joel M. Town and Allan Abbass, Department of Psychiatry, Dalhousie
University, Halifax, Nova Scotia, Canada; Marc J. Diener, Clinical Psychol-
ogy Doctoral Program, Long Island University—CW Post Brookville, NY,
USA; Falk Leichsenring, Clinic of Psychosomatics and Psychotherapy, Uni-
versity of Giessen, Giessen, Germany; Ellen Driessen, Department of Clinical
Psychology, Faculty of Psychology and Education, VU University of Amster-
dam, Amsterdam, The Netherlands; Sven Rabung, Department of Medical
Psychology, University Medical Center Hamburg-Eppendorf, Hamburg, Ger-
many, and University of Klagenfurt, Klagenfurt, Austria.
The authors would like to dedicate this article to the memory of Dr.
Leigh McCullough: With grateful appreciation for Dr. McCullough’s pas-
sionate support, collaboration, and a lifetime’s achievements in advancing
the field of psychotherapy research and clinical theory, practice, and
training.
Correspondence concerning this article should be addressed to Joel M.
Town, Abbie J. Lane Building, 7th Floor, Room 7516, 5909 Veterans’
Memorial Lane, Halifax, Nova Scotia B3H 2N1, Canada. E-mail:
joel.town@dal.ca

276
 RESEARCH PROCEDURES META-ANALYSIS 277

Studies included in the 6 select reviews of Psychodynamic therapy:

Abbass (2006) - 23 studies
Abbass (2009) - 14 studies
Diener (2007) - 10 studies
Driessen (2010) - 23 studies
Leichsenring (2011) - 10 studies
Town (2011) - 7 studies

71 independent studies iden
fied through a manual review of 6 prior

reviews (excluding 16 duplicate studies included in 2 or more reviews)

30 excluded studies- reasons for exclusion:

23 studies- Criterion of randomized design not fulfilled
7 studies- Study data unavailable

41 independent studies consis
ng of 46 treatment arms met inclusion

criteria and were included in meta-analysis

Figure 1. Process of selection of trials.

methods and their use of them. Further, the empirical question of
whether the requisites of experimental research diverge from the
requisites of good treatment remains unclear. This article examines
the impact on clinical outcome, if any, of using audio/video recording,
treatment manuals, and treatment fidelity checks in psychodynamic
psychotherapy.
Background
It has been well-documented that psychodynamic therapists
have historically tended to be antipathetic toward scientific inves-
tigation (Parry, Roth, & Fonagy, 2005). One view is that what can
be learnt from current paradigms of research is of limited signif-
icance because of over simplistic attempts to quantify the complex
mental activity presumed within psychoanalytic theory (Green,
1996). Further objections lie in theoretically based assumptions
that research methods confound treatment objectives. For example,
traditional psychoanalytic theory might assume that audio/video
recording compromises the neutrality of the therapist, arguably by
contributing to the existing therapist–patient power imbalance: the
sanctity of the therapeutic relationship is sullied, therefore distort-
ing the transference– countertransference matrix. Theoretically,
this could harm the therapeutic process, enhance patient resistance,
and impede treatment progress. Assuming iatrogenic effects such
as this can and do regularly occur also with use of treatment
manuals and fidelity checks (which empirical data do not currently
substantiate), more widespread use of psychodynamic treatment
protocols incorporating these research-specific components is
unlikely.
Audio/Video Recording in Psychodynamic
Psychotherapy
A number of recent articles from psychodynamically trained
clinicians have highlighted the utility of, and applications for, this
technology (Abbass et al., 2011; McCullough, Bhatia, Ulvenes,
Berggraf, & Osborn, 2011; Manring, Greenberg, Gregory, & Gall-
inger, 2011), prompting the statement that such innovations, “will
take psychotherapy training, research, supervision, and treatment
forward toward increased effectiveness” (Barnett, 2011, p. 103).
However, variation in the use of audio/video equipment by psy-
chodynamic therapists exists, and it would be overly simplistic to
suggest that this can be wholly attributed to theories, like that
described earlier, that have since been updated and revised. Other
possible contributory factors relevant to the wider psychotherapy
field include (a) therapists’ and/or supervisors’ anxiety, (b) strain
and pressures that come with set-up and time needed to review
tapes, (c) resource limitations preventing access to technology, (d)
difficulties guaranteeing confidentiality and/or security of tapes,
and (e) training deficits/limited knowledge about the technology.
Despite the aforementioned possible limitations or “barriers” to
audio/video recording, psychodynamically orientated researchers
have described the advantages of using video recording for psy-
chotherapy training purposes (Abbass, 2004; Binder, 1993, 1999;
Hilsenroth, Defife, Blagys, & Ackerman, 2006; Levenson &
Strupp, 1999), including clinical supervision within psychoana-
lytic programs (Haggerty & Hilsenroth, 2011). As an educational
experience, video playback enables microanalysis of key in-
session events (Aveline, 1992; Binder, 1993; Levenson, 2006), and
the opportunity to expose less experienced therapists to treatment
nuances. Using recordings, therefore, may help trainees overcome
rigid adherence to technique, which can occur at the expense of the
therapeutic alliance (Strupp, Butler, & Rosser, 1988). Video re-
cording in particular allows the detection and analysis of subtle
nonverbal cues communicated between patient and therapist that
would otherwise go unnoticed. Furthermore, the use of video
playback can enable greater specificity in guiding the timing and
application of therapeutic technique, providing anchored instruc-
tion (Binder, 2004). This is consistent with the finding that use of
278 TOWN ET AL.
video recording within a psychotherapy training program enabled
acquisition of psychodynamic techniques (Hilsenroth et al., 2006).
Haggerty and Hilsenroth (2011) argued that recording therapy
sessions can also minimize the effect of memory limitations (e.g.,
forgetting, misattribution, absent mindedness, bias) that restrict
how accurately session content can be examined using more indi-
rect methods.
Beyond providing a resource that can supplement and facilitate
the supervisory process, session recording also facilitates the abil-
ity of therapists to conduct self-analysis (Alpert, 1996; Aveline,
1992; Huhra, Yamokoski-Maynhart, & Prieto, 2008; Wolberg,
1954). More generally, therapists’ exposure to videotapes of their
own sessions and those of others promotes increased self-
awareness and improved anxiety tolerance (Abbass, 2004). Al-
though few studies have examined the impact of using recordings,
a recent meta-analysis on the utility of affect-focused techniques in
psychodynamic therapies found that the most robust moderator
variable was the presence of use of audio/videotapes within su-
pervision (r ⫽ .29; Diener, Hilsenroth, & Weinberger, 2007).
Despite this accumulation of evidence, to our knowledge, the
empirical question of whether audio/video technology contributes
positively or negatively to clinical outcomes has not been formally
answered (Brown, 1990; Friedmann, Yamamoto, Wolkon, & Da-
vis, 1978).
Treatment Manuals in Psychodynamic Psychotherapy
Psychotherapy treatment manuals provide information on the
nature of the disorder being treated and the specific treatment
strategies and technique, duration, and format to be delivered
(Clarkin, 1998). Early psychodynamic psychotherapy manuals re-
ported time-limited, focal forms of treatment (e.g., Supportive-
Expressive Psychotherapy, Luborsky, 1984; Time-Limited Dy-
namic Psychotherapy, Strupp & Binder, 1984), and more recently,
contemporary longer-term treatments have been manualized (e.g.,
Transference focused psychotherapy, Clarkin, Yeomans, & Kern-
berg, 2006; Mentalization-based treatment, Bateman & Fonagy,
2004). The introduction of manuals, primarily as a research tool
for operationalizing interventions under empirical investigation,
appeared to also offer the potential for improving outcomes
through delivering adherent empirically based treatments (Wilson,
1995, 1996), superior to reliance on the clinical judgment of
therapists (Drozd & Goldfried, 1996; Wilson, 1998). However,
based on national practitioner surveys (e.g., Addis & Krasnow,
2000), a reluctance to incorporate manualized methods is not
unique to psychodynamically informed therapists. Although the
merit in common practitioner concerns about manualized ap-
proaches (Addis, Wade, & Hatgis, 1999) may be questioned (Fon-
agy, 1999), the empirical literature indicates that treatment man-
uals do not ensure effective delivery of therapy (Binder, 1993;
1999; Butler & Strupp, 1993). However, based on a small number
of studies that produced conflicting results, and given the limita-
tions of study methodology, it is less clear whether that which is
gained in adherence to specific techniques through manuals (Crits-
Christoph et al., 1998; Henry, Strupp, Butler, Schacht, & Binder,
1993; Hilsenroth, Defife, Blagys, & Ackerman, 2006; Multon et
al., 1996) may be at the expense of other therapeutic factors and
therapeutic progress (Strupp & Anderson, 1997).
Conclusions about the value of treatment manuals note their
limitations but overall support their clinical and training utility as
a source of “conceptual support” for defining patient problems and
guiding the content of interventions (Binder et al., 1993; Strupp &
Anderson, 1997). Furthermore, manuals have aided psychotherapy
research efforts to better understand mechanisms of therapeutic
change. Treatment manuals have almost become a necessary ele-
ment in psychotherapy outcome studies (Kazdin, 1994; Kiesler,
1994; Lambert & Bergin, 1994); thus, they offer an accepted
medium for operationalizing the independent variable and as such,
a framework for assessing fidelity.
Treatment Fidelity in Psychodynamic Psychotherapy
Treatment fidelity (also commonly referred to as treatment
integrity) refers to the extent to which therapy is delivered as
intended (Kazdin, 1994). Adherence considers whether the core
components or techniques, typically described in a treatment man-
ual, are implemented, and treatment competence is the skill or
accuracy with which these interventions are delivered: both com-
ponents define treatment fidelity but may be measured separately.
Although treatment manuals offer direction for standardizing how
therapy can be delivered, formal assessment of treatment imple-
mentation usually involves objective review of audio/video record-
ings of sessions: methods may include use of a reliable scale of
measurement, independent trained raters, and interrater reliability
calculations.
Empirical research measuring treatment adherence and compe-
tence has overall demonstrated no consistent relationship with
outcome, irrespective of the adequacy of the methods used. The
lack of effect may be due to the methods used having been
inadequate to detect an association, or that other treatment char-
acteristics, therapist, or patient factors, confound a possible rela-
tionship with outcome (Perepletchikova & Kazdin, 2005). Leich-
senring et al. (2011, p. 319) concluded that the lack of association
could be because fidelity plays a more complex role in the out-
come of psychotherapy.
According to psychological theory, psychotherapy is intended to
relate to therapeutic change. Thus, independent of the result,
treatment fidelity is relevant to any discussion on outcome because
it can point to the likely validity of statements about effectiveness/
efficacy (Kazdin, 1994, p. 38). For example, treatment fidelity is
relevant for determining whether (a) a treatment delivered is
representative of the theoretical constructs and mechanisms pre-
sumed to underpin its purpose, (b) the extent to which treatment
effects are causally attributed to the treatment applied, and (c)
whether these methods are generalizable to clinical practice
(Leichsenring et al., 2011).
Preliminary results from a meta-analysis, albeit with a small
sample (k ⫽ 9) of studies comparing psychodynamic psychother-
apy and CBT, found a positive relationship between psychody-
namic studies reporting treatment fidelity procedures and outcome
(Leichsenring et al., 2011). This was true, however, for only a few
variables (Leichsenring et al., 2011). Although nonsignificant,
some of the correlations were substantial, corresponding to
medium-large effect sizes. This was interpreted as possible evi-
dence that the implementation of fidelity checks may offer a small
effect on the outcome for psychodynamic treatment studies. Re-
 RESEARCH PROCEDURES META-ANALYSIS 279

search was recommended to confirm a result that could potentially
have practice implications for psychodynamic psychotherapy.
Current Study
Psychodynamic psychotherapy clinicians, researchers, and
stakeholders alike continue to look for methods, techniques,
and strategies for titrating and improving the size, longevity, and
transmissibility of treatment effects. Progress is necessary to in-
crease the clinical utility of empirical findings and reduce the
researcher– clinician gap. Given the aforementioned wide-ranging
relevance of research-specific procedures, we propose that dis-
missing their potential application requires demonstrable evidence
that they are associated with significantly worse outcomes. Pro-
ponents committed to psychodynamic principles regularly use
audio/video recording, treatment manuals, and fidelity checks in
clinical trials, though, to our knowledge, no well-controlled clin-
ical trials exist that used a dismantling procedure to examine the
possible effects of their use within any psychotherapy modality,
psychodynamic, or otherwise. This gap in the literature may con-
tribute to variation in attitudes toward their use and also limits
conclusions on the effects of such use.
This review will therefore examine empirically the extent to
which use of research-specific procedures in psychodynamic psy-
chotherapy effect outcomes. As a secondary research question, we
sought to identify which variables moderate the potential relation-
ship between the use of these research methods in psychodynamic
psychotherapy and outcome.
Method
Selection of Studies
Using extensive searches previously conducted, we selected arti-
cles using the list of studies included in the six most recently pub-
lished reviews on psychodynamic psychotherapy conducted by the
coauthors (Abbass, Hancock, Henderson, & Kisely, 2006; Abbass,
Kisely, Kroenke, 2009; Diener et al., 2007; Driessen et al., 2010;
Leichsenring & Rabung, 2011; Town, Abbass, & Hardy, 2011) (Ta-
ble 1 Characteristics of reviews). Criteria used for selecting studies
differed between reviews: One review selected studies with a parallel
group design consisting of a nonactive control arm (Abbass et al.,
2006), whereas others included active treatment comparisons (Dries-
sen et al., 2010; Leichsenring & Rabung, 2011; Town et al., 2011).
Four of these reviews selected only Short-Term Psychodynamic Psy-
chotherapy (STPP), excluding studies where the average treatment
length was greater than 40 therapy sessions (Abbass et al., 2006;
Abbass, Kisely, & Kroenke, 2009; Driessen et al., 2010; Town et al.,
2011), the fifth selected only Long-Term Psychodynamic Psychother-
apies (LTPP) (Leichsenring & Rabung, 2011), and the sixth described
studies of psychodynamic psychotherapy that presented data relevant
to therapist facilitation of patient affect experiencing/expression (Die-
ner et al., 2007). We also noted that the original searches were
conducted at different times, so additional studies may have since
been published. However, the literature from which data were se-
lected is representative of a large body of studies (k ⫽ 71) commonly
cited on the overall effectiveness of short- and long-term psychody-
namic psychotherapy. We felt this provided a valid rationale for using
studies from this list of previously published and recent meta-
analyses.
Inclusion Criteria
Having identified studies of interest based on their inclusion in
one of the six published reviews on psychodynamic psychother-
apy, a second screening process was used to confer eligibility. We
included studies in which the psychotherapy treatment (a) was
described by the authors as psychodynamic or psychoanalytic in
nature, (b) was provided in an individual or group format (e.g., not
Internet delivered or self-help), and (c) applied verbal techniques
(e.g., treatments using art as a form of expression were excluded).
All participants had to be at least 18 years old and considered to
have a common mental disorder. The latter included anxiety dis-
orders, depressive disorders, stress-related physical conditions, and
interpersonal or personality problems mixed with symptom disor-
ders. Psychotic disorders were excluded. We only included ran-
domized controlled trials (RCTs) in which psychodynamic psy-
chotherapy was implemented as an active treatment arm and

Table 1
Characteristics of Psychodynamic Therapy Reviews From Which Studies Were Selected

Psychodynamic Date search Studies

Review therapy format Study design Population Databases searched completed included

Abbass et al., 2006 STPP; ⱕ40 sessions; RCT Common mental disorders PsycINFO, CENTRAL, April 2005 23
individual tx MEDLINE, CINAHL,
EMBASE
Abbass et al., 2009 STPP; ⱕ40 sessions; RCT, non-RCT Somatic disorders PsycINFO, MEDLINE, 2007 14
individual tx Cochrane Library
Diener et al., 2007 No restrictions RCT, non-RCT Mixed PsycINFO February 2006 10
Driessen et al., STPP; ⱕ40 sessions; RCT, non-RCT Depressive disorders PsycINFO, PubMed, Cochrane November 2007 23
2010 individual/group Library, EMBASE, Web of
Science
Leichsenring & LTPP; ⬎1 year or RCT, non-RCT Common mental disorders PsycINFO, MEDLINE, Current April 2010 10
Rabung, 2011 50 sessions; Contents
individual/group
Town et al., 2011 STPP; ⱕ40 sessions; RCT Personality disorders PsycINFO, MEDLINE, January 2008 7
individual CINAHL, EMBASE

Note. STPP ⫽ Short-Term Psychodynamic Psychotherapy; LTPP ⫽ Long-Term Psychodynamic Psychotherapy; RCT ⫽ Randomized Controlled Trial.
280 TOWN ET AL.
participants were randomly assigned to treatment. Studies were
excluded if there was insufficient data to enable calculation of
within-group effect sizes.
Data Extraction
Using a structured and standardized coding scheme, written to
specify study characteristics of interest, we (JMT, MJD, AA, ED,
SR) independently extracted the necessary data for calculation of
effect sizes alongside the following study information: full study
citation, short- versus long-term treatment length, mean number of
treatment sessions, primary patient diagnostic group, treatment
modality, use of treatment manuals, treatment fidelity checked,
outcome informant, use of audio/video recording, and use of
pharmacotherapy. For primary diagnostic group, we included de-
pressive disorders, anxiety disorders, somatoform disorders, and
personality disorders consistent with Diagnostic and Statistical
Manual (4th ed.; DSM–IV; American Psychiatric Association,
1994) categories. A fifth group, “mixed-other disorders,” was
added to ensure that other patient groups from eligible studies were
captured within the dataset.
When necessary, coding ambiguities were discussed and the
coding manual was revised/clarified as needed. Given the potential
ambiguity in coding decisions for the “Outcome type” moderator
variable (e.g., “General psychiatric symptoms” vs. “depressive
symptoms,” etc.), the first (JMT) and third author (AA) recoded all
data for this variable. Kappa coefficients calculated to check
interrater reliability between raters were excellent (Cohen’s ␬ ⫽
0.95). Any discrepancies between these ratings and the original
ones were resolved via consensus discussion.
In the case of nine papers (Cooper, Murray, Wilson, & Roma-
niuk, 2003; de Jonghe et al., 2004; Hamilton et al., 2000; Linnet
and Jemec, 2001; Maina et al., 2005; Monsen & Monsen, 2000;
Shapiro et al., 1994/Hardy et al., 1995; Svartberg, Stiles, & Seltzer,
2004; Winston et al., 1994), ratings were completed by two coders
because of overlap of studies in the authors’ respective databases.
Interrater reliability was computed for categorical data with the
kappa coefficient, and all values (range ⫽ 0.84 to 1.0) were in the
excellent range (⬎0.74; Fleiss, 1981). Intraclass correlation coef-
ficients (ICC) for ratings of mean treatment length also demon-
strated excellent agreement between raters (ICC[2,1] ⫽ 1.0;
Shrout & Fleiss, 1979). Discrepancies in ratings were resolved via
consensus discussion.
Effect size calculation consisted of within-group standardized
mean difference scores using Equations 4.18 – 4.19 from Boren-
stein, Hedges, Higgins, and Rothstein (2009; Abbass, Town, &
Driessen, 2012; cf., Dunlap, Cortina, Vaslow, & Burke, 1996), and
the standard error was calculated using Equations 4.20 – 4.21 from
Borenstein et al. (2009).
Effect sizes were calculated for both pre–post outcomes and
posttreatment-follow up outcomes (when available). In cases in
which multiple follow-ups were reported, the longest time period
for follow-up was selected, with one exception: The follow-up
period selected for Bateman and Fonagy (2001) was the 18-month
follow-up. Although there is an additional 8-year follow-up avail-
able for this study (Bateman & Fonagy, 2008), the 18-month
follow-up was selected because this interval is more commonly
used to evaluate treatment effects than the longer, 8-year alterna-
tive. Effect sizes were coded as positive if the data indicated
improvement in the treatment group or negative if the data indi-
cated deterioration in the treatment group. When studies reported
multiple effect sizes that were relevant (e.g., multiple outcome
measures were used), these effect sizes were averaged, following
standard meta-analytic convention (Horvath & Symonds, 1991;
Martin, Garske, & Davis, 2000).
The original coding manual distinguished between eight types
of outcome measures. To limit the number of analyses, however,
only the following outcomes were examined in the analyses: (a)
Overall outcome (aggregate of all eight original outcome types),
(b) General psychiatric symptoms, (c) Anxiety symptoms, (d)
Depressive symptoms, and (e) Personality, that is, a condensed
outcome variable consisting of the original “Personality function-
ing/traits” outcomes as well as outcome data based on the Inven-
tory of Interpersonal Problems (Horowitz, Alden, Wiggins, &
Pincus, 2000).
Quantitative Data Synthesis
Effect sizes were aggregated across studies using the random-
effects method of Hedges and colleagues (Hedges & Vevea, 1998).
Random-effects methods are considered to be more representative
of real-world data (National Research Council, 1992) and yield
results that are more generalizable than their fixed-effect counter-
parts (Hedges & Vevea, 1998). These calculations were performed
using version 2 of the Comprehensive Meta-Analysis (CMA;
Borenstein et al., 2005) software. Cohen’s (1988) benchmarks for
the magnitude of effect sizes were selected to aid in interpreting
the results. Homogeneity tests and related analyses (e.g., calcula-
tion of I2) were conducted to examine the degree of variation
between effect sizes (Borenstein et al., 2009). All reported p values
in the present study are two-tailed unless otherwise noted.
Moderator Analyses
Because the primary analyses of the present study involved
outcome comparisons between studies that did versus did not use
a particular methodological variable (e.g., recordings), all moder-
ator analyses were conducted by comparing outcomes for studies
that did and did not use that methodological variable separately for
each of the relevant moderator variable levels. Thus, for example,
in examining the moderator impact of use of treatment manuals on
the use of recordings, outcomes for studies that used recordings
were compared with outcomes for studies that did not use record-
ings, first for studies that did use treatment manuals and then for
studies that did not use such manuals.
Continuous moderator analyses were conducted using mixed-
effects (method of moments) meta-regression analyses for effect
size data aggregated across all outcomes only and for pre-post data
only, with the average effect size for each study serving as the
dependent variable and each continuous moderator variable serv-
ing as a covariate. Because CMA software will not conduct a
multiple meta-regression analysis, each covariate was examined
using a separate meta-regression. Meta-regression analyses were
performed using the following moderators as predictor variables of
effect size: (a) Publication year, and (b) Mean number of treatment
sessions. When this latter information was not explicitly reported
or when data were not provided to allow calculation of mean, the
 RESEARCH PROCEDURES META-ANALYSIS
planned/estimated number of treatment sessions based on treat-
ment completion was used.
For all categorical moderator analyses, Q tests, analogous to
analysis of variance in primary research (Borenstein et al., 2009;
Lipsey & Wilson, 2001), were calculated to determine whether the
various levels of the moderator variable differed significantly from
each other. When all subgroups in a particular analysis had at least
six studies, estimates of the variance of true effect sizes were not
pooled; when at least one subgroup, however, had fewer than six
studies, estimates of the variance of true effect sizes were pooled
because the accuracy yielded by pooling is likely to be greater than
any real differences between subgroups (Borenstein et al., 2009).
When examining the impact of audio/video recordings, categorical
analyses were conducted for each of the following moderator vari-
ables: (a) Use of pharmacotherapy (e.g., antidepressant medication):
yes (medication use was allowed during treatment and/or follow-up),
no (medication use was not permitted during treatment or follow-up),
or unclear (it was unclear whether or not medication use was permit-
ted during treatment and/or follow-up); (b) Use of a treatment manual:
yes or no (regarding use of treatment manual reported in methodol-
ogy); (c) Treatment fidelity: None (i.e., fidelity checks of treatment
were reported, and psychodynamic treatment was shown to not be
delivered as required by the study; note that no studies in the present
meta-analysis were given this rating), Unknown (i.e., fidelity checks
were not reported on the psychodynamic treatment), Confirmed (i.e.,
fidelity checks of treatment were reported, and psychodynamic treat-
ment was shown to be delivered as required by the study); (d)
Treatment length: Short-term (ⱕ40 sessions) versus long-term (⬎40
sessions). When examining the impact of use of a treatment manual,
categorical analyses were conducted for the following moderators: (a)
Use of audio/video recording; (b) Use of pharmacotherapy; (c) Treat-
ment fidelity; (d) Treatment length. Finally, when examining the
impact of treatment fidelity, categorical analyses were conducted for
the following moderators: (a) Use of audio/video recording; (b) Use
of pharmacotherapy; (c) Use of a treatment manual; (d) Treatment
length.
Publication Bias
Potential publication bias of the overall meta-analysis was as-
sessed in multiple ways including (a) Begg and Mazumdar’s
(1994) rank correlation, (b) Egger’s regression intercept (Egger,
Davey Smith, Schneider, & Minder, 1997), and (c) Duval and
Tweedie’s (2000a, 2000b) trim and fill procedure.
Results
Inclusion of Studies
The initial search, screening studies included in six previously
conducted reviews of the psychodynamic psychotherapy literature,
resulted in the identification of 71 independent trials (this did not
include 16 duplicate studies included in one or more of the 6
original reviews). In the second phase, the full-text of each trial
was screened by one of the authors for the presence of this
review’s inclusion and exclusion criteria. Twenty-three studies
were excluded because the criterion of randomized design was not
fulfilled. Seven studies were excluded because the necessary data
to enable effect size calculation were not available (Baldoni et al.,
281
1995; Creed et al., 2003; Guthrie, Creed, Dawson, & Tomenson,
1993; Morris, 1975; Piper, Azim, McCallum, & Joyce, 1990;
Sjodin, 1983; Svedlund, Sjodin, Ottosson, & Dotevall, 1983).
Hardy et al. (1995) was excluded because data were reported on a
subsample of Shapiro et al. (1994). This process yielded 41 inde-
pendent RCTs of psychodynamic psychotherapy suitable for in-
clusion and meta-analysis. Within this literature, five studies (Clar-
kin, Levy, Lenzenweger, & Kernberg, 2007; Huber and Klug,
2006; Knekt et al., 2008; Vinnars, Barber, Noren, Gallop, &
Weinryb, 2005; Winston et al., 1994) reported two psychotherapy
arms described as psychodynamic or psychoanalytic in origin and
thus eligible for inclusion as forms of psychodynamic psychother-
apy. In total, 46 psychodynamic psychotherapy treatment groups
were available for analysis. (A complete reference list for all
meta-analysed studies is available in a data supplement that ac-
companies the online version of this article.)
Study characteristics. The 46 independent psychodynamic
psychotherapy treatment samples consisted of 1,615 subjects. The
mean number of subjects per treatment sample was 35 (SD ⫽ 28),
and the range was 8 to 128.1 The included studies reported on
subjects displaying a range of common mental health presenta-
tions. The primary diagnostic groups treated were categorized as
Depressive disorders (k ⫽ 15), Anxiety disorders (k ⫽ 4), Soma-
toform disorders (k ⫽ 5), Personality disorders (k ⫽ 13), and
mixed/other disorder (k ⫽ 9).
The majority of studies (k ⫽ 38) reported psychodynamic psy-
chotherapy with an average treatment length of ⱕ40 sessions
(short-term), whereas eight studies averaged ⬎40 sessions (long-
term) in treatment duration. One study described a group-based
psychodynamic intervention (Liberman and Eckman, 1981), and
all other treatments were delivered in an individual format. Audio
or video recording of psychodynamic treatment sessions was im-
plemented in 24 studies, and not used or not reported in 22 studies.
Manualized psychodynamic treatments were used in 33 studies,
whereas 13 did not report the use of a treatment manual. Treatment
fidelity checks were conducted in more than half of the included
studies (k ⫽ 25) and not checked or reported in others (k ⫽ 21).
Although a standardized method for evaluating study quality
was not used here, several points on the methodology of the studies
can be highlighted. Because only RCTs were included, all studies
used random assignment to allocate subjects to condition. The
assessment of treatment outcome at follow-up was present in 31
studies (average follow-up on overall outcome ⫽ 12.8 months,
SD ⫽ 9.2). Pharmacotherapy (e.g., use of antidepressants) was
1
Not every effect size or every study included in the present meta-
analytic review was included in each analysis since, for example, separate
analyses were conducted for pre–post versus posttreatment/follow-up ef-
fect sizes and Cooper et al. (2003) provided only posttreatment/follow-up
data. The descriptive data reported in the main body of the article provide
the numbers aggregated across every study and every effect size, and so
they are not indicative of the actual sample sizes for particular analyses in
the present study. The analysis with the largest sample size was the overall
meta-analysis for psychodynamic psychotherapy using pre–post effect
sizes. This analysis included 45 independent treatment samples from 42
peer-reviewed journal articles as well as one book chapter (plus unpub-
lished data for that book chapter) made up of 40 RCTs. The total number
of participants for this specific analysis was 1,613, with a range of sample
sizes from 13 to 116, and a mean sample size of 36 (SD ⫽ 29).
282 TOWN ET AL.
permitted in 23 studies, not permitted in 13, and it was unclear
whether medication was monitored in the remaining 10 studies.
Quantitative Data Synthesis
Effect sizes of psychodynamic psychotherapy. Including all
of the 45 conditions of psychodynamic psychotherapy, results
indicated a large magnitude of change in outcome from pretreat-
ment to posttreatment when all outcome measures were aggre-
gated, and that this change was statistically significant (d៮ ⫽ 1.01,
95% confidence interval [CI] ⫽ 0.86 –1.16, p ⬍ .001). Results also
indicated demonstrable heterogeneity across effect sizes (Q[44] ⫽
157.51, p[one-tailed] ⬍ .001), and the degree of variation that
could be attributed to true differences between effect sizes fell in
the medium to large range (I2 ⫽ 72.07, ␶2 ⫽ 0.17, ␶ ⫽ 0.42;
Higgins, Thompson, Deeks, & Altman, 2003). This heterogeneity
is examined further by the attempt described later to identify
moderator variables that could account for observed differences
between effect sizes.
For outcomes measured between posttreatment and follow-up,
the effect size was 0.18 (95% CI ⫽ 0.07– 0.29), indicating a
statistically significant improvement in overall outcome (p ⫽
.002). There was no demonstrable heterogeneity across effect sizes
(Q[29] ⫽ 37.40, p[one-tailed] ⫽ .14), and the degree of variation
that could be attributed to true differences between effect sizes fell
slightly below Higgins et al.’s (2003) benchmark for small (I2 ⫽
22.46, ␶2 ⫽ 0.02, ␶ ⫽ 0.15).
The treatment effects in psychodynamic psychotherapy seen
between pre- and posttreatment (see Table 2) were also calculated
specifically for different outcome domains, with results ranging
from 0.75 to 1.20, all ps ⬍ .001. These analyses found statistically
significant change in outcome scores that were in the large range,
and approaching the conventional 0.80 cutoff for personality func-
tioning. At pretreatment to follow-up, change in personality func-
tioning was large (d៮ ⫽ 0.96, 95% CI ⫽ 0.60 –1.27, p ⬍ .001). We
calculated changes in outcomes from posttreatment to follow-up.
Effect sizes ranged from 0.22 to 0.50, all ps ⬍ .05, indicating
small, but significant, improvement across all outcome domains.
Publication bias for overall effect sizes. Results indicated no
demonstrable evidence of publication bias for the overall meta-
analysis of pre–post outcome using either Begg and Mazumdar’s
(1994) rank correlation method (Kendall’s tau [with continuity cor-
rection, Borenstein, Hedges, Higgins, & Rothstein, 2005] ⫽ 0.13,
p[one-tailed] ⫽ .10) or Egger’s (Egger et al., 1997) regression inter-
cept method (intercept ⫽ 0.79, p[one-tailed] ⫽ .16). In addition,
Table 2
Random Effects Meta-Analysis of Psychodynamic Psychotherapy: Pre- to Posttreatment and Posttreatment to Follow-Up Change
Pre–post treatment
k d៮ 95% CI Z value
Overall 45 1.01 0.86–1.16 13.12
Depression 28 1.20 1.0–1.40 11.76
Anxiety 18 0.87 0.67–1.07 8.41
Personality functioning 20 0.75 0.50–0.99 6.02
General psychiatric 31 1.07 0.87–1.27 10.4
ⴱ ⴱⴱ
p ⬍ .01. p ⬍ .05.
results from Duval and Tweedie’s (2000a, 2000b) trim and fill pro-
cedure suggested that the impact of potential publication bias would
likely be minimal even if it did exist (zero studies trimmed; adjusted
and observed estimates of effect size were identical).
For posttreatment to follow-up change, publication bias results
suggested some potential for publication bias using Egger’s (Egger
et al., 1997) intercept method (intercept ⫽ 1.11, p[one-tailed] ⫽
.05), and only a trend toward significance for Begg and Mazum-
dar’s (1994) approach (Kendall’s tau [with continuity correction] ⫽
0.17, p[one-tailed] ⫽ .099). Results from Duval and Tweedie’s
(2000a, 2000b) trim and fill procedure indicated that the impact of
any potential publication bias would likely be minimal even if it
did exist (zero studies trimmed; adjusted and observed estimates of
effect size were identical).
Utility of audio/video recordings by outcome type. Table 3
summarizes all effect sizes and test statistics for each outcome
category and type of contrast for pre–post change. The pooled
effect size indicating the size of the change in overall outcome in
studies with audio/video recording was 0.92 (95% CI ⫽ 0.71–1.14,
p ⬍ .001) and 1.11 (95% CI ⫽ 0.90 –1.32, p ⬍ .001) when
recording was absent. Across each of the four individual outcome
categories, the effect size for pre–post change was also large (d៮ ⬘s
ranging from 0.85 to 1.32) and statistically significant both in
studies that utilized audio/video recordings as well as studies that
did not; the one exception was personality functioning, for which
studies that used audio/video recording demonstrated a statistically
significant effect in the medium–large range (d៮ ⫽ 0.63, 95% CI ⫽
0.38 – 0.89, p ⬍ .001).
The comparison for effect size between studies that did use record-
ings and those that did not was not statistically significant for any
outcome type, though a small effect indicating potentially better
outcomes was found in studies that did not use audio/video recording
for depressive symptoms (⌬d៮ ⫽ 0.22) and personality functioning
(⌬d៮ ⫽ 0.30).
Table 4 summarizes all effect sizes and test statistics based on
posttreatment to follow-up contrasts for individual outcome types. A
comparison for anxiety symptoms was not conducted because only a
single study was coded as using audio/video recording. Comparisons
of within-group contrasts at posttreatment to follow-up revealed sta-
tistically significant improvement with and without audio/video re-
cording when all outcomes were aggregated together (d៮ ⫽ 0.21, 95%
CI ⫽ 0.01– 0.41, p ⫽ .042; and d៮ ⫽ 0.16, 95% CI ⫽ 0.02– 0.31, p ⫽
.030; respectively). For individual outcome domains analyzed, results
ranged from 0.09 to 0.42; see Table 4 for more details).
Post/follow-up
p k d៮ 95% CI Z value p
ⴱ
⬍.001 30 0.18 0.07–0.29 3.11 .002ⴱ
⬍.001ⴱ 14 0.24 0.04–0.44 2.39 .017ⴱⴱ
⬍.001ⴱ 11 0.50 0.04–1.0 2.11 .035ⴱⴱ
⬍.001ⴱ 12 0.25 0.01–0.48 2.04 .041ⴱⴱ
⬍.001ⴱ 19 0.22 0.07–0.36 2.96 .003ⴱ
 RESEARCH PROCEDURES META-ANALYSIS 283

Table 3
Random Effects Meta-Analysis of Psychodynamic Psychotherapy by Use of Audio/Video Recordings: Within-Group Change and
Between-Group Differences for Pre- to Posttreatment Outcomes

Within-group Between-group

Subgroup k d៮ 95% CI Z-value p ⌬d a Q-value p

Overall
A/V 24 0.92 0.71–1.14 8.38 ⬍.001ⴱ ⫺0.19 1.45 .229
No A/V 21 1.11 0.90–1.32 10.32 ⬍.001ⴱ
Depression
A/V 15 1.10 0.82–1.37 7.79 ⬍.001ⴱ ⫺0.22 1.19 .275
No A/V 13 1.32 1.03–1.61 8.80 ⬍.001ⴱ
Anxiety
A/V 6 0.93 0.56–1.29 5.01 ⬍.001ⴱ 0.08 0.12 .732
No A/V 12 0.85 0.59–1.10 6.51 ⬍.001ⴱ
Personality functioning
A/V 13 0.63 0.38–0.89 4.85 ⬍.001ⴱ ⫺0.30 1.20 .274
No A/V 7 0.93 0.46–1.39 3.91 ⬍.001ⴱ
General psychiatric
A/V 16 0.98 0.64–1.31 5.64 ⬍.001ⴱ ⫺0.17 0.70 .402
No A/V 15 1.15 0.92–1.38 9.93 ⬍.001ⴱ

Note. ⌬d ⫽ the magnitude of difference in outcome between studies that did use audio-visual recording and studies that did not do so.
a
Positive effect sizes indicate differences in favor of use of research-specific procedure.
ⴱ
p ⬍ .001.

Based on between-groups comparisons at posttreatment to follow-
up, a small effect size was found for personality functioning favoring
use of audio/video recording (⌬d៮ ⫽ 0.33). All between-groups sta-
tistical comparisons were, however, nonsignificant.
Utility of treatment manuals by outcome type. Table 5
summarizes all effect sizes and test statistics for each outcome cate-
gory and type of contrast for pre–post change. The pooled effect size
for overall outcome in studies with treatment manuals was 0.98 (95%
CI ⫽ 0.79 –1.17, p ⬍ .001) and 1.08 (95% CI ⫽ 0.82–1.35, p ⬍ .001)
when treatment manuals were absent. Across each of the four indi-
vidual outcome categories, the effect size for pre–post change was
also large (d៮ ⬘s ranging from 0.82 to 1.28) and statistically significant
both in studies that used treatment manuals as well as studies that did
not; the one exception was personality functioning, for which studies
that used treatment manuals (d៮ ⫽ 0.74, 95% CI ⫽ 0.41–1.06, p ⬍
.001) as well as studies that did not use treatment manuals (d៮ ⫽ 0.78,
95% CI ⫽ 0.48 –1.09, p ⬍ .001) both demonstrated a statistically
significant effect in the medium–large range.

Table 4
Random Effects Meta-Analysis of Psychodynamic Psychotherapy by Use of Audio/Video Recordings: Within-Group Change and
Between-Group Differences for Posttreatment to Follow-Up Outcomes

Within-group Between-group

Subgroup k d៮ 95% CI Z-value p ⌬d a Q-value p

Overall
A/V 14 0.21 0.01 to 0.41 2.04 .042ⴱ 0.05 0.14 .707
No A/V 16 0.16 0.02 to 0.31 2.17 .030ⴱ
Depression
A/V 6 0.27 ⫺0.13 to 0.67 1.33 .185 0.06 0.06 .808
No A/V 8 0.21 0.01 to 0.41 2.08 .037ⴱ
Anxietyb
A/V — — — — — — — —
No A/V — — — — — — — —
Personality functioning
A/V 6 0.42 ⫺0.04 to 0.88 1.78 .076 0.33 1.61 .205
No A/V 6 0.09 ⫺0.12 to 0.30 0.84 .403
General psychiatric
A/V 8 0.25 ⫺0.06 to 0.56 1.58 .115 0.03 0.02 .878
No A/V 11 0.22 0.05 to 0.39 2.56 .010ⴱ

Note. ⌬d ⫽ the magnitude of difference in outcome between studies that did use audio-visual recording and studies that did not do so.
a
Positive effect sizes indicate differences in favor of use of research-specific procedure. b The comparison for anxiety symptoms was not conducted
because only a single study was coded as using audio/video recording.
ⴱ
p ⬍ .05.
284 TOWN ET AL.

Table 5
Random Effects Meta-Analysis of Psychodynamic Psychotherapy by Use of Treatment Manual: Within-Group Change and
Between-Group Differences for Pre- to Posttreatment Outcomes

Within-group Between-group

Subgroup k d៮ 95% CI Z-value p ⌬d a Q-value p

Overall
Manual 32 0.98 0.79–1.17 10.28 ⬍.001ⴱ ⫺0.10 0.39 .533
No manual 13 1.08 0.82–1.35 8.03 ⬍.001ⴱ
Depression
Manual 18 1.16 0.90–1.41 8.87 ⬍.001ⴱ ⫺0.12 0.33 .565
No manual 10 1.28 0.94–1.62 7.41 ⬍.001ⴱ
Anxiety
Manual 11 0.98 0.64–1.31 5.74 ⬍.001ⴱ 0.16 0.63 .426
No manual 7 0.82 0.61–1.02 7.75 ⬍.001ⴱ
Personality functioning
Manual 14 0.74 0.41–1.06 4.47 ⬍.001ⴱ ⫺0.04 0.04 .838
No manual 6 0.78 0.48–1.09 4.97 ⬍.001ⴱ
General psychiatric
Manual 24 1.04 0.78–1.31 7.72 ⬍.001ⴱ ⫺0.09 0.20 .656
No manual 7 1.13 0.86–1.39 8.42 ⬍.001ⴱ

Note. ⌬d ⫽ the magnitude of difference in outcome between studies that did use treatment manuals and studies that did not do so.
a
Positive effect sizes indicate differences in favor of use of research-specific procedure.
ⴱ
p ⬍ .001.

All comparisons for effect size between studies that did use
treatment manuals and those that did not were not statistically
significant for any outcome type, and effects fell short of the
conventional benchmark for a small effect (i.e., all ⌬ds ⬍ 0.20).
Table 6 summarizes all effect sizes and test statistics based on
posttreatment to follow-up contrasts for individual outcome types.
Within-group contrasts revealed statistically significant improve-
ment but only for studies that used treatment manuals (d៮ ⫽ 0.22,
95% CI ⫽ 0.06 – 0.37, p ⫽ .007) for overall outcome. Similarly,
effect sizes for the individual outcome domains in studies that used
treatment manuals ranged from 0.25 to 0.71, all ps ⬍ .05 (see
Table 6 for details). All between-groups statistical comparisons
were nonsignificant, although a medium effect size for anxiety
symptoms was obtained favoring use of treatment manuals (⌬d៮ ⫽
0.56).
Utility of fidelity checks by outcome type. Table 7 sum-
marizes all effect sizes and test statistics for each outcome
category and type of contrast for pre–post change. The pooled
effect size indicating the size of the change in overall outcome
in studies with confirmed fidelity was 1.03 (95% CI ⫽ 0.81–

Table 6
Random Effects Meta-Analysis of Psychodynamic Psychotherapy by Use of Treatment Manuals: Within-Group Change and
Between-Group Differences for Posttreatment to Follow-Up Outcomes

Within-group Between-group

Subgroup k d៮ 95% CI Z-value p ⌬d a Q-value p

Overall
Manual 21 0.22 0.06 to 0.37 2.71 .007ⴱ 0.10 0.59 .441
No manual 9 0.12 ⫺0.08 to 0.31 1.20 .230
Depression
Manual 10 0.26 0.02 to 0.50 2.16 .031ⴱⴱ 0.07 0.11 .744
No manual 4 0.19 ⫺0.21 to 0.58 0.91 .363
Anxiety
Manual 7 0.71 0.11 to 1.31 2.33 .020ⴱⴱ 0.56 1.26 .261
No manual 4 0.15 ⫺0.64 to 0.93 0.36 .717
Personality functioning
Manual 7 0.33 0.01 to 0.66 2.02 .043ⴱⴱ 0.19 0.60 .440
No manual 5 0.14 ⫺0.25 to 0.52 0.70 .484
General psychiatric
Manual 14 0.25 0.07 to 0.43 2.74 .006ⴱ 0.09 0.29 .593
No manual 5 0.16 ⫺0.12 to 0.44 1.11 .027

Note. ⌬d ⫽ the magnitude of difference in outcome between studies that did use treatment manuals and studies that did not do so.
a
Positive effect sizes indicate differences in favor of use of research-specific procedure.
ⴱ
p ⬍ .001. ⴱⴱ p ⬍ .05.
 RESEARCH PROCEDURES META-ANALYSIS 285

Table 7
Random Effects Meta-Analysis of Psychodynamic Psychotherapy by Fidelity: Within-Group Change and Between-Group Differences
for Pre- to Posttreatment Outcomes

Within-group Between-group

Subgroup k d៮ 95% CI Z-value p ⌬d a Q-value p

Overall
Fidelityb 24 1.03 0.81–1.25 9.05 ⬍.001ⴱ 0.04 0.07 .796
No fidelityc 21 0.99 0.79–1.20 9.48 ⬍.001ⴱ
Depression
Fidelityb 13 1.14 0.83–1.44 7.29 ⬍.001ⴱ ⫺0.12 0.33 .567
No fidelityc 15 1.26 0.99–1.52 9.21 ⬍.001ⴱ
Anxiety
Fidelityb 5 1.14 0.74–1.54 5.63 ⬍.001ⴱ 0.35 2.36 .124
No fidelityc 13 0.79 0.54–1.01 6.51 ⬍.001ⴱ
Personality functioning
Fidelityb 11 0.68 0.40–0.96 4.76 ⬍.001ⴱ ⫺0.13 0.27 .607
No fidelityc 9 0.81 0.39–1.22 3.81 ⬍.001ⴱ
General psychiatric
Fidelityb 16 1.02 0.67–1.37 5.75 ⬍.001ⴱ ⫺0.09 0.18 .675
No fidelityc 15 1.11 0.89–1.33 10.04 ⬍.001ⴱ

Note. ⌬d ⫽ the magnitude of difference in outcome between studies with confirmed treatment fidelity and studies without reported treatment fidelity
checks.
a
Positive effect sizes indicate differences in favor of use of research-specific procedure. b Indicates studies that conducted treatment fidelity
checks. c Indicates studies that did not report treatment fidelity checks.
ⴱ
p ⬍ .001.

1.25, p ⬍ .001) and 0.99 (95% CI ⫽ 0.79 –1.20, p ⬍ .001) when
treatment fidelity checks were not reported. Effect sizes for
individual outcome domains ranged from 0.68 to 1.26, all of
which were statistically significant (all ps ⬍ .001; see Table 7
for details).
The comparison for effect size between studies with confirmed
fidelity versus studies that did not report fidelity checks was not
statistically significant for any outcome type. Effect size calcula-
tion of the magnitude of between-groups comparisons found a
small effect for anxiety symptoms (⌬d ⫽ 0.35) favoring studies
with confirmed fidelity.
Table 8 summarizes all effect sizes and test statistics based on
posttreatment to follow-up contrasts for individual outcome types.
Comparison of within-group contrasts at posttreatment to follow-up
revealed statistically significant improvement in four of five analyses
for studies with confirmed fidelity (see table for specifics). Statisti-
cally significant within-group contrasts at posttreatment to follow-up
in studies where fidelity checks were not conducted were only present
for general psychiatric symptoms (d៮ ⫽ 0.20, 95% CI ⫽ 0.02– 0.38,
p ⫽ .031) and depressive symptoms (d៮ ⫽ 0.23, 95% CI ⫽ 0.01– 0.46,
p ⫽ .045).
Based on between-groups comparisons at posttreatment to
follow-up, a statistically significant difference was found for anx-
iety symptoms with a magnitude of a large effect size favoring
studies with confirmed fidelity (⌬d៮ ⫽ 1.40, p ⫽ .004). All remain-
ing between-groups statistical comparisons were, however, non-
significant.
Moderator analyses of the impact of research-specific pro-
cedures. We conducted moderator analyses (for each respective
research procedure, we considered the effect of (a) audio-video re-
cording/treatment manual/treatment fidelity checks [where applica-
ble]; (b) pharmacotherapy use; (c) treatment length; (d) mean number
of treatment sessions), including all studies, with outcome defined as
change in pretreatment to posttreatment scores for overall outcome.
Only a single study was coded as not using a treatment manual that
had confirmed fidelity. Similarly, only a single study was coded for an
absence of treatment manual but presence of audio/video recording.
These sets of analyses were therefore not conducted. Moderator
results showed that no categorical analyses yielded a medium effect
size (i.e., all ⌬d៮ ⬘s ⬍ 0.50; no p values could be computed for these
analyses because the meta-analytic software cannot calculate moder-
ator analyses for subgroup data; the ⌬d៮ ⬘s were calculated by subtract-
ing the relevant difference scores).2 In addition, all meta-regressions
using the mean number of treatment sessions or the year of publica-
tion did not significantly predict outcomes for studies that did have
any of the research-specific variables (all ps ⬎ .05); the same was true
for all meta-regressions for studies that did not have any of the
research-specific variables.
Discussion
There is mounting evidence from recent meta-analyses that dem-
onstrate the effectiveness of psychodynamic psychotherapy (Abbass
et al., 2006; Abbass et al., 2009; Driessen et al., 2010; Leichsenring &
2
For example, to examine the moderating role of treatment length on the
impact of audio/video recording, studies that did not use audio/video
recording (d៮ ⫽ 1.00) were compared with studies that did use audio/video
recording (d៮ ⫽ 0.92) for the group of studies that had short-term treatment.
The difference associated with use of recording is therefore 1.00 ⫺ 0.92 ⫽
0.08 for studies with short-term treatments. Next, studies that did not use
audio/video recordings (d៮ ⫽ 1.38) were compared with studies that did use
audio/video recordings (d៮ ⫽ 0.94) for the long-term treatments only. These
yields a difference score of 1.38 ⫺ 0.94 ⫽ 0.44. When the previously
mentioned difference score of 0.08 is subtracted from this difference score
of 0.44, the resulting effect size is 0.36.
286 TOWN ET AL.

Table 8
Random Effects Meta-Analysis of Psychodynamic Psychotherapy by Fidelity: Within-Group Change and Between-Group Differences
for Posttreatment to Follow-Up Outcomes

Within-group Between-group

Subgroup k d៮ 95% CI Z-value p ⌬d a Q-value p

Overall
Fidelityb 17 0.22 0.04 to 0.40 2.41 .016ⴱⴱ 0.08 0.39 .535
No fidelityc 13 0.14 ⫺0.02 to 0.31 1.70 .089
Depression
Fidelityb 8 0.27 ⫺0.06 to 0.59 1.58 .113 0.04 0.02 .880
No fidelityc 6 0.23 0.01 to 0.46 2.00 .045ⴱⴱ
Anxiety
Fidelityb 3 1.54 0.72 to 2.35 3.69 ⬍.001ⴱ 1.40 8.50 .004ⴱ
No fidelityc 8 0.14 ⫺0.34 to 0.61 0.56 .574
Personality functioning
Fidelityb 7 0.33 0.01 to 0.66 2.02 .043ⴱⴱ 0.19 0.60 .440
No fidelityc 5 0.14 ⫺0.25 to 0.52 0.70 .484
General psychiatric
Fidelityb 10 0.27 0.00 to 0.54 1.98 .047ⴱⴱ 0.07 0.18 .669
No fidelityc 9 0.20 0.02 to 0.38 2.15 .031ⴱⴱ

Note. ⌬d ⫽ the magnitude of difference in outcome between studies with confirmed treatment fidelity and studies without reported treatment fidelity
checks.
a
Positive effect sizes indicate differences in favor of use of research-specific procedure. b Indicates studies that conducted treatment fidelity
checks. c Indicates studies that did not report treatment fidelity checks.
ⴱ
p ⬍ .01. ⴱⴱ p ⬍ .05.

Rabung, 2011; Town et al., 2011). The reoccurring finding that
therapeutic gains may not only be maintained after psychodynamic
treatment but they continue to improve with time is especially note-
worthy (Shedler, 2010). Based on the results from the current meta-
analysis of only randomized controlled trials including 46 psychody-
namic psychotherapy treatment samples (d៮ ⫽ 1.01 for pre–post
overall outcome; d៮ ⫽ .18 for overall outcome between posttreatment
and an average of 12.8 months of follow-up), we now believe there is
unambiguous empirical support for these findings. Evidence of con-
tinued gains posttreatment is of particular importance because it
suggests that under the right circumstances, psychotherapy may fa-
cilitate changes in the underlying psychological structures and intra-
psychic processes presumed to mediate psychopathology, thus en-
abling long-lasting benefits that translate to the real world, far beyond
and after therapy is complete. But, crucially much remains unknown
about the precise mechanisms of psychotherapeutic change (Kazdin,
2007). Toward this goal, treatment protocols may use procedures such
as audio/video recording, treatment manuals, and fidelity checks that
allow specification of what is actually delivered in psychotherapy. To
examine the utility and impact of these methods on treatment effec-
tiveness in psychodynamic psychotherapy, in this meta-analysis, we
conducted individual subgroup analyses to compare treatment effects
in studies using recording equipment, treatment manuals, and fidelity
checks versus psychodynamic psychotherapy in which each respec-
tive procedure was absent. Within the limitations of these analyses,
which require results to be considered observational in nature, all
comparisons between groups across different outcome domains indi-
cated no statistically significant differences in treatment effects at
posttreatment. The presence/absence of significant within-group dif-
ferences in treatment effects between posttreatment and follow-up do,
however, provide preliminary evidence that using research-specific
procedures may in fact bring an advantage, particularly on measures
of anxiety (⌬ds ranging from .56 to 1.40) and, to a lesser extent,
personality functioning (⌬ds ranging from .19 to .33), which appear
after treatment is completed.
The use of audio/video recording, treatment manuals, and fidelity
checks in psychodynamic psychotherapy can be controversial. Based
on some proponents of psychoanalytic theory, everything else being
equal, one might suppose negative effects. The limitations of this
conceptual argument may be questioned, given other proponents
clearly committed to psychodynamic principles regularly incorporate
audio/video recording, manualized treatments, and fidelity checks in
clinical trials. Given the absence of statistically significant differences
in posttreatment effects, overall, the current findings suggest the use
of these methods, at this time, is not demonstrably associated with
either better or worse outcomes at the end of therapy in efficacy trials.
If you consider the magnitude of the difference in treatment effects
between studies rather than rely solely on the criteria of whether they
reached statistical significance, this revealed some limited (nonsignif-
icant) differences based on the use of audio/video equipment. Of the
five pre- to posttreatment between-group comparisons, two fell in the
range of a small effect indicating better outcomes for the studies that
did not use audio/video recording (⌬d ⫽ .22 for depressive symp-
toms; ⌬d ⫽ .30 for personality functioning). At follow-up, however,
this trend was not observed; by contrast, studies that used audio/video
recording demonstrated better outcomes for personality functioning
(⌬d ⫽ .33).
The potential benefits of recording psychotherapy sessions for
purposes of retrospective review (Abbass, 2004; Alpert, 1996;
Binder, 1993a, 1999) and supervision (Aveline, 1992; Haggerty &
Hilsenroth, 2011) find only limited support from the current find-
ings. This is perhaps unsurprising given that we might expect not
just the occurrence of recordings but also the type, quality, and
perceived helpfulness of the review/supervision process received
to impact on treatment outcome. Research efforts should pay
attention to the longer-term effects of audio/video review both for
 RESEARCH PROCEDURES META-ANALYSIS
treatment outcome as well as therapist training and development of
treatment competencies.
Furthermore, moderator analysis examining the magnitude of
difference in pre- to posttreatment outcome between those studies
that used recordings and those that did not, revealed a small effect
(d៮ ⫽ 0.36), favoring long-term treatment (⬎40 sessions). Closer
inspection of the pooled mean effect size within studies of long- or
short-term treatment with or without audio/video recording
showed that the larger treatment effects seen in studies not using
recording was in fact driven by five studies that provided treat-
ment, with an average duration of approximately 3 years (145
sessions); d៮ ⫽ 1.40. The three studies of long-term psychodynamic
psychotherapy that used audio/video recording, in contrast, pro-
vided on average only 59 sessions with a pooled posttreatment
effect of d៮ ⫽ .95, comparable with short-term psychodynamic
psychotherapy (ⱕ40 sessions) outcomes (k ⫽ 38), with or without
recording (d៮ ⬘s ranging from .92–.99). Thus, the observation of
larger effects in psychodynamic psychotherapy without audio/
video recording may be better accounted for by treatment dose
rather than the impact of recording equipment.
The lack of a significant association between outcome posttreat-
ment and the use of either treatment manuals or fidelity checks is
consistent with findings from the common factors literature that
outcome variance due to treatment differences is minor (Lambert,
1992; Wampold et al., 1997). A recent meta-analysis (k ⫽ 9) that
included five overlapping studies analyzed here, also found no sig-
nificant correlations between effects sizes and treatment integrity
procedures comparing psychodynamic psychotherapy with cognitive–
behavioral therapy. Although effect sizes at follow-up were not ex-
amined, the magnitude of correlations at posttreatment did offer
tentative evidence that the attention given to implementing procedural
interventions for monitoring fidelity may particularly benefit outcome
in psychodynamic psychotherapy (Leichsenring et al., 2011). The
failure to replicate some of these posttreatment finding in the current
study could be accounted for by a methodological difference between
studies: here study differences were assessed based on a categorical
distinction of the presence/absence of research procedures, which
ignores possible variation in the nature and quality of the procedures
used. However, a potentially new emerging finding from this study is
evidence that continued therapeutic improvement occurring after psy-
chodynamic psychotherapy may be associated to the use of research-
specific procedural factors. We were stuck to find that nine of a
possible 10 within-group analyses (90%), examining posttreatment to
follow-up outcome change in studies according to the presence of
treatment manuals or fidelity checks, were statistically significant
when the same was true in only two of 10 analyses (20%) for studies
excluding these procedures. Between-groups statistical comparisons
were nonsignificant in all but one case; however, of the 10 analyses,
two fell narrowly below the benchmark for a small effect (⌬d ⫽ .19)
and two were in the small/medium to large range (⌬d ’s ⫽ .33–1.40).
As such, this offers further evidence that research procedural inter-
ventions connected to assessment of how treatment is delivered, may
moderate outcome. These research procedures may also just be a
proxy variable for other influences that account for therapeutic change
(Perepletchikova & Kazdin, 2005). We do, however, recognize that
the between-group analyses showed largely small effects and included
a small number of studies, meaning the addition of new studies may
change the results. We advise caution when interpreting these find-
ings, and further studies are required to confirm the results.
287
An alternative interpretation of these results is that treatment
effects present in psychodynamic psychotherapy studies that used
research-specific procedures translate largely only to improve-
ments on measures of anxiety and personality functioning. By
statistical convention, small but consistently positive effects (⌬d ⫽
.19 for use of manual; ⌬d ⫽ .19 for use of fidelity checks; ⌬d ⫽
.33 for use of audio/video recording) were present on personality
functioning scores in follow-up. Results at 18-month and 8-year
follow-up, in one included study of manualized long-term psy-
chodynamic psychotherapy, specifically illustrate that continued
improvement and wide-ranging changes in functioning can occur
at long-term outcome in a personality disordered sample (Bateman
& Fonagy, 2001, 2008). More attention should be paid to the
importance of efforts to pursue a “goal-orientated structure to
treatment” (Critchfield & Benjamin, 2006, p. 62) and therapist
consistency (Livesley, 2007) for treating personality problems.
The magnitude of improvement in anxiety symptoms, however,
was more pronounced, both between pre- and posttreatment and at
follow-up assessment. Most notably, a very large statistically
significant treatment effect (d៮ ⫽ 1.40, p ⫽ .004) was found
indicating further improvement after therapy was completed in
studies that checked fidelity. This finding, when taken alongside
the generally good evidence for the efficacy of manualized behav-
ioral and cognitive– behavioral approaches with anxiety disorders,
may indicate that prioritizing the delivery of specific, concrete
techniques is important. This is consistent with symptoms of
anxiety being potentially more tractable to strategies because they
are less likely to represent a disorder of the self (Roth & Fonagy,
2005).
It remains to be seen whether these observations reflect chance
findings. Having not accounted for the possible moderating effects
of therapist experience and level of training, this may represent a
confound within the present study: we might predict larger effects
and less variation between studies regardless of the use of research
procedures in more experienced and well-trained therapists. Future
research using larger sample sizes and greater power is necessary
to address these issues more definitively.
Critical Analysis
The results of this meta-analysis should be considered alongside
its limitations. First, this studies’ reliance on subgroup analyses to
detect differences associated to different research-specific proce-
dures cannot be used to prove causality: it is possible that differ-
ences in effect sizes may be attributed to differences between the
sets of studies. Second, although a large number of studies were
included in analyses, some controlled studies that may have met
the inclusion were missed because they were not included in past
publications from which eligible studies were selected. Further-
more, the original searches were conducted at different times
(between 2005 and 2010) using different criteria so additional
studies may have since been published. Third, the possibility of
publication bias stemming from a reliance on reviews of the
published outcome literature, which may fail to detect trials with
null findings, should be considered: publication bias analyses
conducted suggest this is, however, unlikely. Fourth, the quality of
included studies was variable (e.g., medication was allowed along-
side psychotherapy in 23 studies). Although the results of suba-
nalyses to examine the moderating effect of use of treatment
288 TOWN ET AL.
manual, treatment fidelity, and medication use showed that these
variables (pharmacotherapy use, treatment length, mean number
of sessions, year of publication) did not account for a significant
proportion of variance in treatment effects, power for testing
moderators is often low (Borenstien et al., 2009; Hedges & Pigott,
2004). Additional points for consideration on methodological
study quality such as blinding of observer-rated outcomes, report-
ing of intention to treat analyses, and the reliability of random
allocation procedure may, however, have had an influence on
treatment effects. The study can point to the inclusion of a sizable
body of literature, including only controlled trials, using random
allocation to group, predefined selection criteria for patient partic-
ipation, and the use of reliable outcome measures, to state the case
for the validity of the reported findings.
Conclusion
The current findings confirm that psychodynamic psychother-
apy produces large overall treatment effects that continue to im-
prove in long-term follow-up. Based on subgroup analysis, we can
say that there is provisional evidence indicating that using
research-specific treatment procedures is not associated with spe-
cifically better or worse outcomes posttreatment. However, signif-
icant treatment gains occurring after therapy appear more likely
when these research-specific procedures are used. Although well-
controlled research is ultimately necessary to infer causality and
the mechanisms of change contributing to outcome are unclear, the
current findings makes a case for reconsidering how the benefits of
integrating research-specific procedures in psychodynamic psy-
chotherapy can be maximized to outweigh perceived limitations.
References
Abbass, A., Arthey, S., Elliott, J., Fedak, T., Nowoweiski, D., Markovski,
J., & Nowoweiski, S. (2011). Web-conference supervision for advanced
psychotherapy training: A practical guide. Psychotherapy, 48, 109 –118.
doi:10.1037/a0022427
Abbass, A., Kisely, S., & Kroenke, K. (2009). Short-term psychodynamic
psychotherapy for somatic disorders. Systematic review and meta-
analysis of clinical trials. Psychotherapy and Psychosomatics, 78, 265–
274. doi:10.1159/000228247
Abbass, A., Town, J. M., & Driessen, E. (2012). Intensive short-term
dynamic psychotherapy: A systematic review and meta-analysis of out-
come research. Harvard Review of Psychiatry, 20, 97–108. doi:10.3109/
10673229.2012.677347
Abbass, A. A., Hancock, J. T., Henderson, J., & Kisely, S. (2006). Short-
term psychodynamic psychotherapies for common mental disorders.
Cochrane Database of Systematic Reviews, CD004687. doi:10.1002/
14651858.CD004687.pub3
Abbass, A. A. (2004). Small-group videotape training for psychotherapy
skills development. Academic Psychiatry, 28, 151–155. doi:10.1093/
brief-treatment/mhn001
Addis, M. E., & Krasnow, A. D. (2000). A national survey of practicing
psychologists’ attitudes towards psychotherapy treatment manuals.
Journal of Consulting and Clinical Psychology, 68, 331–339. doi:
10.1037//0022-006X.68.2
Addis, M. E., Wade, W. A., & Hatgis, C. (1999). Barriers to dissemination
of evidence based practices: Addressing practitioners concerns about
manual-based psychotherapies. Clinical Psychology: Science and Prac-
tice, 6, 430 – 441.
Alpert, M. C. (1996). Videotaping psychotherapy. Journal of Psychother-
apy Practice and Research, 5, 93–104.
American Psychiatric Association. (1994). Diagnostic and statistical man-
ual of mental disorders (4th ed.). Washington, DC: American Psychiat-
ric Association.
Aveline, M. (1992). The use of audio and videotape recordings of therapy
sessions in the supervision and practice of dynamic psychotherapy.
British Journal of Psychotherapy, 8, 347–358. doi:10.1111/j.1752-
0118.1992.tb01198.x
Baldoni, F., Baldaro, B., & Trombini, G. (1995). Psychotherapeutic per-
spectives in urethral syndrome. Stress Medicine, 11, 79 – 84. doi:
10.1002/smi.2460110115
Barnett, J. E. (2011). Utilizing technological innovations to enhance psy-
chotherapy supervision, training, and outcomes. Psychotherapy, 48,
103–108. doi:10.1037/a0023381
Bateman, A., & Fonagy, P. (2001). Treatment of borderline personality
disorder with psychoanalytically oriented partial hospitalization: An
18-month follow-up. American Journal of Psychiatry, 158, 36 – 42.
Bateman, A., & Fonagy, P. (2008). Eight-year follow-up of patients treated
for borderline personality disorder: Mentalization-based treatment ver-
sus treatment as usual. The American Journal of Psychiatry, 165, 631–
638. doi:10.1176/appi.ajp.2007.07040636
Bateman, A. F., & Fonagy, P. (2004). Psychotherapy for borderline per-
sonality disorder: Mentalization-based treatment. New York: Oxford
University Press.
Begg, C. B., & Mazumdar, M. (1994). Operating characteristics of a rank
correlation test for publication bias. Biometrics, 50, 1088 –1101. Re-
trieved from http://www.biometrics.tibs.org/
Binder, J. L., Strupp, H. H., Bongar, B., Lee, S. S., Messer, S., & Peake,
T. H. (1993). Recommendations for improving psychotherapy training
based on experiences with manual guided training and research: Epi-
logue. Psychotherapy, 30, 599 – 600.
Binder, J. L. (1993). Is it time to improve psychotherapy training? Clinical
Psychology Review, 12, 301–318. doi.org/10.1016/0272-
7358(93)90015-E
Binder, J. L. (1999). Issues in teaching and learning time-limited psy-
chodynamic psychotherapy. Clinical Psychology Review, 19, 705–719.
doi.org/10.1016/S0272-7358(98)00078-6
Blatt, S. J., & Zuroff, D. C. (2005). Empirical evaluations of the assump-
tions in identifying evidence based treatments in mental health. Clinical
Psychology Review, 25, 459 – 486.
Borenstein, M., Hedges, L., Higgins, J., & Rothstein, H. (2005). Compre-
hensive meta- analysis version 2. Englewood, NJ: Biostat.
Borenstein, M., Hedges, L. V., Higgins, J. P., & Rothstein. H. R. (2009).
Introduction to meta-analysis. Chichester, UK: Wiley, Ltd.
Brown, E. (1990). Problems of the use of video recordings in training for
psychotherapy. Psychotherapy and Psychosomatics, 53, 139 –141. doi:
10.1159/000288356
Butler, S. F., & Strupp, H. H. (1993). Effects of training experienced
dynamic therapists to use a psychotherapy manual. In N. E. Miller, L.
Luborsky, J. P. Barber, & J. P. Docherty (Eds.), Psychodynamic treat-
ment research: A handbook for clinical practice (pp. 191–210). New
York: Basic Books.
Clarkin, J. F., Levy, K. N., Lenzenweger, M. F., & Kernberg, O. F. (2007).
Evaluating three treatments for borderline personality disorder: A mul-
tiwave study. American Journal of Psychiatry, 164, 922–928.
Clarkin, J. F., Yeomans, F., & Kernberg, O. F. (2006). Psychotherapy of
borderline personality: Focusing on object relations. Washington, DC:
American Psychiatric Publishing, Inc.
Clarkin, J. F. (1998). Intervention research: Development and manualiza-
tion. In A. S. Bellack & M. Hersen (Eds.), Comprehensive clinical
psychology (Vol. 3). New York: Pergamon Press.
Cohen, J. (1988). Statistical power analysis for the behavioral sciences
(2nd ed.). Hillsdale, NJ: Erlbaum.
Cooper, P. J., Murray, L., Wilson, A., & Romaniuk, H. (2003). Controlled
trial of the short- and long-term effect of psychological treatment of
 RESEARCH PROCEDURES META-ANALYSIS
post-partum depression: Part I. Impact on maternal mood. British Jour-
nal of Psychiatry, 182, 412– 419.
Creed, F., Fernandes, L., Guthrie, E., Palmer, S., Ratcliffe, J., Read, N., . . .
Tomenson, B. (2003). The cost-effectiveness of psychotherapy and
paroxetine for severe irritable bowel syndrome. Gastroenterology, 124,
303–317. doi:10.1053/gast.2003.50055
Critchfield, K. L., & Benjamin, L. S. (2006). Integration of therapeutic
factors in treating personality disorders. In L. E. Castonguay & L. E.
Beutler (Eds.), Principles of therapeutic change that work (pp. 253–
271). New York: Oxford University Press.
Crits-Christoph, P., Frank, E., Chambless, D. L., Brody, C., & Karp, J. F.
(1995). Training in empirically validated treatments: What are clinical
psychology students learning. Professional Psychology: Research and
Practice, 26, 514 –522.
Crits-Christoph, P., Siqueland, L., Chittams, J., Barber, J. P., Beck, A. T.,
Liese, B., . . . Woody, G. (1998). Training in cognitive, supportive-
expressive, and drug counseling therapies for cocaine dependence. Jour-
nal of Consulting and Clinical Psychology, 66, 484 – 492.
de Jonghe, F., Hendriksen, M., van Aalst, G., Kool, S., Peen, J., Van, R.,
. . . Dekker, J. (2004). Psychotherapy alone and combined with pharma-
cotherapy in the treatment of depression. British Journal of Psychiatry,
185, 37– 45. doi:10.1192/bjp.185.1.37
Diener, M. J., Hilsenroth, M. J., & Weinberger, J. (2007). Therapist affect
focus and patient outcomes in psychodynamic psychotherapy: A meta-
analysis. American Journal of Psychiatry, 164, 936 –941. doi:10.1176/
appi.ajp.164.6.936
Driessen, E., Cuijpers, P., de Maat, S., Abbass, A., de Jonghe, F., &
Dekker, J. (2010). The efficacy of short-term psychodynamic psycho-
therapy for depression: A meta-analysis. Clinical Psychology Review,
30, 25–36. doi.org/10.1016/j.cpr.2009.08.010
Drozd, J. F., & Goldfried, M. R. (1996). A critical evaluation on the
state-of-the-art in psychotherapy outcome research. Psychotherapy, 33,
171–180.
Dunlap, W. P., Cortina, J. M., Vaslow, J. B., & Burke, M. J. (1996).
Meta-analysis of experiments with matched groups or repeated measures
designs. Psychological Methods, 1, 170 –177. doi:10.1037/1082-
989X.1.2.170
Duval, S., & Tweedie, R. (2000a). A nonparametric “trim and fill” method
of accounting for publication bias in meta-analysis. Journal of the
American Statistical Association, 95, 89 –98. Retrieved from http://
www.amstat.org/publications/jasa.cfm
Duval, S., & Tweedie, R. (2000b). Trim and fill: A simple funnel-plot-
based method of testing and adjusting for publication bias in meta-
analysis. Biometrics, 56, 455– 463. Retrieved from http://www
.biometrics.tibs.org/
Egger, M., Davey Smith, G., Schneider, M., & Minder, C. (1997). Bias in
meta-analysis detected by a simple graphical test. British Medical Jour-
nal, 315, 629 – 634. Retrieved from http://www.British Medical Journal
.com/
Fleiss, J. L. (1981). Statistical methods for rates and proportions. New
York: Wiley.
Fonagy, P. (1999). Achieving evidence-based psychotherapy practice: A
psychodynamic perspective on the general acceptance of treatment man-
uals. Clinical Psychology: Science and Practice, 6, 442– 444.
Friedmann, C., Yamamoto, J., Wolkon, G., & Davis, L. (1978). Videotaped
recordings of dynamic psychotherapy: Supervisory tool or hindrance.
American Journal of Psychiatry, 135, 1388 –1391.
Guthrie, E., Creed, F., Dawson, D., & Tomenson, B. (1993). A randomized
controlled trial of psychotherapy in patients with refractory irritable
bowel syndrome. British Journal of Psychiatry, 163, 315–321.
Haggerty, G., & Hilsenroth, M. J. (2011). The use of video in psychother-
apy supervision. British Journal of Psychotherapy, 27, 193–210. doi:
10.1111/j.1752-0118.2011.01232.x
Hamilton, J., Guthrie, E., Creed, F., Thompson, D., Tomenson, B., Bennett,
289
R., . . . Liston, R. (2000). A randomized controlled trial of psychother-
apy in patients with chronic functional dyspepsia. Gastroenterology,
119, 661– 669.
Hardy, G. E., Barkham, M., Shapiro, D. A., Stiles, W. B., Rees, A., &
Reynolds, S. (1995). Impact of Cluster C personality disorders on
outcomes of contrasting brief psychotherapies for depression. Journal of
Consulting and Clinical Psychology, 63, 997–1004.
Hedges, L. V., & Pigott, T. D. (2004). The power of statistical tests for
moderators in meta-analysis. Psychological Methods, 9, 426 – 445. doi:
10.1037/1082-989X.9.4.426
Hedges, L. V., & Vevea, J. L. (1998). Fixed- and random-effects models in
meta-analysis. Psychological Methods, 3, 486 –504. doi:10.1037/1082-
989X.3.4.486
Henry, W. P., Strupp, H. H., Butler, S. F., Schacht, T. E., & Binder, J. L.
(1993). Effects of training in time-limited dynamic psychotherapy:
Changes in therapist behavior. Journal of Consulting and Clinical Psy-
chology, 61, 434 – 440.
Higgins, J. P., Thompson, S. G., Deeks, J. J., & Altman, D. G. (2003).
Measuring inconsistency in meta-analyses. British Medical Journal,
327, 557–560. doi:10.1136/bmj.327.7414.557
Hilsenroth, M. J., Defife, J., Blagys, M., & Ackerman, S. (2006). Effects
of training in short-term psychodynamic psychotherapy: Changes in
graduate clinician technique. Psychotherapy Research, 16, 292–303.
Horowitz, L. M., Alden, L. E., Wiggins, J. S., & Pincus, A. L. (2000). IIP:
Inventory of Interpersonal Problems Manual. The Psychological
Corporation.
Huber, D., & Klug, G. (2006). Munich psychotherapy study (mps): The
effectiveness of psychoanalytic long-term psychotherapy for depression.
In Society for Psychotherapy Research (Ed.). Book of abstracts. From
research to practice (p. 154). Ulmer Textbank.
Huhra, R. L., Yamokoski-Maynhart, C. A., & Prieto, L. R. (2008). Re-
viewing videotape in supervision: A developmental approach. Journal of
Counseling and Development, 86, 412– 418. doi:10.1002/j.1556-
6678.2008.tb00529.x
Kazdin, A. E. (1994). Methodology, design, and evaluation in psychother-
apy research. In A. E. Bergin & S. L. Garfield (Eds.), Handbook of
psychotherapy and behavior change (4th ed., pp. 19 –71). New York:
Wiley.
Kazdin, A. E. (2007). Mediators and mechanisms of change in psycho-
therapy research. Annual Review of Clinical Psychology, 3, 1–27. doi:
10.1146/annurev.clinpsy.3.022806.091432
Kiesler, D. J. (1994). Standardization of intervention: The tie that binds
psychotherapy research and practice. In P. F. Talley, H. H. Strupp, S. F.
Butler (Eds.), Psychotherapy research and practice: Bridging the gap
(pp. 143–153). New York: Basic Books.
Knekt, P., Lindfors, O., Harkanen, T., Valikoski, M., Virtala, E., Laak-
sonen, M. A., . . . Renlund, C. (2008). Randomized trial on the effec-
tiveness of long-and short-term psychodynamic psychotherapy and
solution-focused therapy on psychiatric symptoms during a 3-year
follow-up. Psychological Medicine, 38, 689 –703. http://dx.doi.org/
10.1017/S003329170700164X;/Border doi:http://dx.doi.org/10.1017/
S003329170700164X
Lambert, M. J. (1992). Psychotherapy outcome research: Implications for
integrative and eclectic therapists. In J. C. Norcross & M. R. Goldfried
(Eds.), Handbook of psychotherapy integration (pp. 94 –129). New
York: Basic Books.
Leichsenring, F., & Rabung, S. (2011). Long-term psychodynamic psy-
chotherapy in complex mental disorders: Update of a meta-analysis.
British Journal of Psychiatry, 199, 15–22. doi:10.1192/
bjp.bp.110.08277
Leichsenring, F., Salzer, S., Hilsenroth, M., Leibing, E., Leweke, F., &
Rabung, S. (2011). Treatment integrity: An unresolved issue in psycho-
therapy research. Current Psychiatry Reviews, 3, 313–321.
Levenson, H., & Strupp, H. H. (1999). Recommendations for the future of
290 TOWN ET AL.
training in brief dynamic psychotherapy. Journal of Clinical Psychol-
ogy, 55, 385–391.
Levenson, H. (2006). Time-limited dynamic psychotherapy. In A. B.
Rochlen (Ed.), Applying counseling theories: An outline case-based
approach (pp. 75–90). Upper Saddle River, NJ: Pearson.
Liberman, R. P., & Eckman, T. (1981). Behavior therapy versus insight-
oriented therapy for repeated suicide attempters. Archives of General
Psychiatry, 38, 1126 –1130.
Linnet, J., & Jemec, G. (2001). Anxiety level and severity of skin condition
predicts outcome of psychotherapy in atopic dermatitis patients. Inter-
national Journal of Dermatology, 40, 632– 636.
Lipsey, M. W., & Wilson, D. B. (2001). Practical meta-analysis. Thousand
Oaks, CA: Sage.
Livesley, W. J. (2007). Integrated therapy for complex cases of personality
disorder. Journal of Clinical Psychology: In Session, 64, 207–221.
Luborsky, L. (1984). Principles of psychoanalytic psychotherapy: A man-
ual for supportive/expressive treatment. New York: Basic Books.
Maina, G., Forner, F., & Bogetto, F. (2005). Randomized controlled trial
comparing brief dynamic and supportive therapy with waiting list con-
dition in minor depressive disorders. Psychotherapy and Psychosomat-
ics, 74, 43–50. doi:10.1159/000082026
Manring, J., Greenberg, R. P., Gregory, R., & Gallinger, L. (2011).
Learning psychotherapy in the digital age. Psychotherapy, 48, 119 –126.
doi:10.1037/a0022185
Martarazzo, R. G., & Garner, A. M. (1992). Research on training for
psychotherapy. In D. K. Freedheim, H. J. Freudenberger, J. W. Kessler,
S. Messer, D. R. Petevsm, H. H. Strupp, P. L. Watchel (Eds.), History of
psychotherapy: A century of change (pp. 850 – 877). Washington, DC:
American Psychological Association.
McCullough, L., Bhatia, M., Ulvenes, P., Berggraf, L., & Osborn, K.
(2011). Learning how to rate video-recorded therapy sessions: A prac-
tical guide for trainees and advances clinicians. Psychotherapy, 48,
127–137. doi:10.1037/a0023131
Monsen, K., & Monsen, J. T. (2000). Chronic pain and psychodynamic
body therapy: A controlled outcome study. Psychotherapy, 37, 257–269.
doi:10.1037/h0087658
Morris, N. E. (1975). A group self-instruction method for the treatment of
depressed outpatients (Doctoral dissertation). Toronto, Canada: Univer-
sity of Toronto.
Multon, K., Kivlighan, D., & Gold, P. (1996). Changes in counselor
adherence over the course of training. Journal of Counseling Psychol-
ogy, 43, 356 –363.
National Research Council. (1992). Combining information: Statistical
issues and opportunities for research. Washington, DC: National Acad-
emy Press.
Parry, G., Roth, T., & Fonagy, P. (2005). Psychotherapy research, health
policy, and service provision. In A. Roth & P. Fonagy (Eds.), What
works for whom: A critical review of psychotherapy research (2nd ed.,
pp. 43– 65). London: The Guilford Press.
Piper, W. E., Azim, H. F., McCallum, M., & Joyce, A. S. (1990). Patient
suitability and outcome in short-term individual psychotherapy. Journal
of Consulting and Clinical Psychology, 58, 475– 481.
Roth, A., & Fonagy, P. (2005). What works for whom: A critical review of
psychotherapy research (2nd ed.). New York: Guilford Press.
Shapiro, D. A., Barkham, M., Rees, A., Hardy, G. E., Reynolds, S., &
Startup, M. (1994). Effects of treatment duration and severity of depres-
sion on the effectiveness of cognitive-behavioral and psychodynamic-
interpersonal psychotherapy. Journal of Consulting and Clinical Psy-
chology, 62, 522–534. doi:10.1037/0022-006X.62.3.522
Shedler, J. (2010). The efficacy of psychodynamic psychotherapy. Amer-
ican Psychologist, 65, 98 –109. doi:10.1037/a0018378
Shrout, P. E., & Fleiss, J. L. (1979). Intraclass correlations: Uses in
assessing rater reliability. Psychological Bulletin, 86, 420 – 428. doi:
10.1037/0033-2909.86.2.420
Sjodin, I. (1983). Psychotherapy in peptic ulcer disease. A controlled
outcome study. Acta Psychiatrica Scandinavica, 67(Suppl 307), 1–90.
Strupp, H. H., & Anderson, T. (1997). On the limitations of therapy
manuals. Clinical Psychology: Science and Practice, 4, 76 – 82.
Strupp, H. H., & Binder, J. L. (1984). Psychotherapy in a new key: A guide
to time-limited dynamic psychotherapy. New York: Basic Books.
Strupp, H. H., Butler, S. F., & Rosser, C. L. (1988). Training in psychody-
namic psychotherapy. Journal of Consulting and Clinical Psychology,
56, 689 – 695.
Svartberg, M., Stiles, T. C., & Seltzer, M. H. (2004). Randomized con-
trolled trial of the effectiveness of short-term dynamic psychotherapy
and cognitive therapy for cluster C personality disorders. American
Journal of Psychiatry, 5, 810 – 817. doi:10.1176/appi.ajp.161.5.810
Svedlund, J., Sjodin, I., Ottosson, J. O., & Dotevall, G. (1983). Controlled
study of psychotherapy in irritable bowel syndrome. Lancet, 2, 589 –592.
Town, J. M., Abbass, A., & Hardy, G. (2011). Short-term psychodynamic
psychotherapy for personality disorders: A critical review of randomized
controlled trials. Journal of Personality Disorders, 25, 723–740. doi:
10.1521/pedi.2011.25.6.723
Vinnars, B., Barber, J. P., Noren, K., Gallop, R., & Weinryb, R. M. (2005).
Manualized supportive-expressive psychotherapy versus nonmanualized
community-delivered psychodynamic therapy for patients with person-
ality disorders: Bridging efficacy and effectiveness. American Journal of
Psychiatry, 10, 1933–1940. doi:10.1176/appi.ajp.162.10.1933
Wampold, B. E., Mondin, G. W., Moody, M., Stich, F., Benson, K., &
Ahn, H. (1997). A meta-analysis of outcome studies comparing bona
fide psychotherapies: Empirically, “All must have prizes”. Psychologi-
cal Bulletin, 122, 203–215.
Wilson, G. T. (1995). Empirically validated treatments as a basis for
clinical practice: Problems and prospects. In S. C. Hayes, V. M. Follette,
R. M. Dawes, & K. E. Grady (Eds.), Scientific standards of psycholog-
ical practice: Issues and recommendations (pp. 163–196). Reno, NV:
Context Press.
Wilson, G. T. (1996). Manual-based treatments: The clinical application of
research findings. Behavior Research and Therapy, 34, 295–314.
Wilson, G. T. (1998). Manual-based treatment and clinical practice. Clin-
ical Psychology: Science and Practice, 5, 363–375.
Winston, A., Laikin, M., Pollack, J., Samstag, L. W., McCullough, L., &
Muran, C. (1994). Short-term psychotherapy of personality disorders.
American Journal of Psychiatry, 151, 190 –194.
Wolberg, L. R. (1954). The technique of psychotherapy (4th ed.). New
York: Grune & Stratton.
Received June 21, 2012
Accepted June 25, 2012 䡲