Brain Imaging and Behavior (2017) 11:1862–1872

DOI 10.1007/s11682-016-9660-0

ORIGINAL RESEARCH

Reversal learning reveals cognitive deficits and altered prediction

error encoding in the ventral striatum in Huntington’s disease
Katharina Nickchen 1,2,3 & Rebecca Boehme 2,4 & Maria del Mar Amador 1 &
Thomas D. Hälbig 1 & Katharina Dehnicke 1,5 & Patricia Panneck 1,2 & Joachim Behr 2,6 &
Konstantin Prass 5 & Andreas Heinz 2 & Lorenz Deserno 2,7 & Florian Schlagenhauf 2,7 &
Josef Priller 1,2,8

Published online: 5 December 2016

# Springer Science+Business Media New York 2016

Abstract Huntington’s disease (HD) is an autosomal
dominant neurodegenerative condition characterized by
a triad of movement disorder, neuropsychiatric symp-
toms and cognitive deficits. The striatum is particularly
vulnerable to the effects of mutant huntingtin, and cell
loss can already be found in presymptomatic stages.
Since the striatum is well known for its role in rein-
forcement learning, we hypothesized to find altered be-
havioral and neural responses in HD patients in a prob-
abilistic reinforcement learning task performed during
Katharina Nickchen and Rebecca Boehme contributed equally to this
work.
* Josef Priller
josef.priller@charite.de
functional magnetic resonance imaging. We studied 24
HD patients without central nervous system (CNS)-ac-
tive medication and 25 healthy controls. Twenty HD pa-
tients and 24 healthy controls were able to complete the task.
Computational modeling was used to calculate prediction error
values and estimate individual parameters. We observed that
gray matter density and prediction error signals during the learn-
ing task were related to disease stage. HD patients in advanced
disease stages appear to use a less complex strategy in the rever-
sal learning task. In contrast, HD patients in early disease stages
show intact encoding of learning signals in the degenerating left
ventral striatum. This effect appears to be lost with disease
progression.
Keywords Huntington’s disease . Reinforcement learning .
Ventral striatum . Gray matter density

1
Department of Neuropsychiatry, Charité –Universitätsmedizin Berlin
(Charité Campus Mitte), Charitéplatz 1, 10117 Berlin, Germany
Introduction
2
Department of Psychiatry and Psychotherapy,
Charité –Universitätsmedizin Berlin (Charité Campus Mitte),
Huntington’s disease (HD) is a rare autosomal dominant neu-
10117 Berlin, Germany rodegenerative condition caused by a CAG trinucleotide re-
3
Fliedner Klinik Berlin, 10117 Berlin, Germany
peat expansion in the Huntingtin gene located on the short arm
4
of chromosome 4 (4p16.3). HD commonly presents in midlife
Center for Social and Affective Neuroscience, Linköping University,
58245 Linköping, Sweden
with a combination of a progressive movement disorder, neu-
5
ropsychiatric symptoms, e.g. depression, irritability, apathy, as
Department of Neurology, Helios Klinikum Bad Saarow, 15526 Bad
Saarow, Germany
well as cognitive deficits with executive dysfunctions and
6
dementia (Barker and Priller 2013). HD is characterized by
Department of Psychiatry, Psychotherapy and Psychosomatics,
Medical School Brandenburg - Campus Neuruppin,
neuronal dysfunction, shrinkage and ultimately neuronal loss
16816 Neuruppin, Germany in a wide range of brain structures. The caudate and putamen
7
Max-Planck-Fellow-Group Cognitive and Affective Control of
appear to be particularly vulnerable and show significant at-
Behavioral Adaptation, Max-Planck-Institute for Human Cognitive rophy even in presymptomatic stages (Aylward et al. 2011;
and Brain Sciences, 04103 Leipzig, Germany Kipps et al. 2005; Weir et al. 2011).
8
Cluster of Excellence NeuroCure, BIH and DZNE, Substantial evidence has implicated frontostriatal circuits
10117 Berlin, Germany in reinforcement learning. Activity of midbrain dopaminergic
Brain Imaging and Behavior (2017) 11:1862–1872
neurons was shown to encode the difference between received
and expected outcome (Montague et al. 1996; Schultz et al.
1997), termed the ‘reward prediction error’ (RPE). This is in
line with findings in humans using functional magnetic reso-
nance imaging (fMRI). Coding of such RPEs was repeatedly
shown in the striatum, which receives dopaminergic input from
the midbrain (D’Ardenne et al. 2008; O’Doherty et al. 2003).
Given this involvement of the striatum in reinforcement
learning and its early dysfunction in HD, deficits in rein-
forcement learning in HD patients are to be expected.
Indeed, a behavioral study showed deficits in probabilistic
discrimination and reversal learning in HD patients com-
pared to controls (Lawrence et al. 1999). However, to date
only few studies have investigated reinforcement learning
in HD patients in relation to neurobiological measures.
Using a Monetary Incentive Delay task, Enzi et al. reported
an impaired association of activity in the left ventral stria-
tum with outcome anticipation in pre-HD subjects near to
disease onset (Enzi et al. 2012). Recently, another study
using the Monetary Incentive task found evidence for dif-
ferentially enhanced reward-related fMRI signaling in the
ventral striatum and orbitofrontal cortex/anterior insula in
prodromal HD (Malejko et al. 2014).
Palminteri et al. used an instrumental learning paradigm
and combined this behavioral task with structural measures
in HD patients (Palminteri et al. 2012). Presymptomatic HD
subjects with accentuated dorsal striatal atrophy were im-
paired in punishment learning but not reward learning. In con-
trast, symptomatic patients (mostly medicated with antipsy-
chotic drugs) with atrophy in ventral and dorsal striatum
showed impairments in learning from reward and punishment
compared with presymptomatic HD subjects and healthy con-
trols. The behavioral deficits were associated with striatal de-
generation following a dorsoventral gradient pattern of gray
matter loss.
So far, neuronal representation of learning signals like
RPEs has not been investigated in HD. Here, we investigated
24 symptomatic HD patients without CNS-active medication
and 25 healthy controls in a reversal learning task using fMRI.
We used computational modeling of reinforcement learning to
describe changes in learning dynamically, and to provide
quantitative fits and parameters for individual behavior. We
hypothesized that ventral striatal RPEs signals are reduced in
HD patients, and that this reduction is associated with the
stage of disease and the degree of ventral striatal atrophy.
Material and methods
Subjects
Twenty-four HD patients and 25 age-/sex-matched healthy
controls (HC) were included in this study. One patient could
1863
not finish the fMRI task because of a panic attack, and did not
undergo further physical and neuropsychological examina-
tion. Three HD patients and one HC were not able to complete
the fMRI task.
In the end, 20 HD patients and 24 HC were included in the
final analysis. The mean age of this final HD cohort was
47.5 years ± 10.3 years. The median number of CAG-repeats
was 43 (range 40–50). The median Unified Huntington’s dis-
ease rating scale (UHDRS) motor score was 18 (range 1–58).
The median total cognitive score was 206 (range 120–319).
Twelve HD patients had a total functional capacity (TFC)
between 11 and 13, eight patients had a TFC ≤ 10. The mean
age in HC was 46.8 years ± 11.3 years. The median total cog-
nitive score in HC was 316.5 (range 265–406) (Table 1).
None of the investigated subjects showed clinical signs of
depression or psychosis. The median mini-mental state exam-
ination (MMSE) scores were 28.5 (range 22–30) in HD pa-
tients, and 29 (range 26–30) in HC. One HD patient showed
mild signs of dementia reaching 22 points on the MMSE.
Overall, HD patients had received a median 14 years of edu-
cation (range 8–18), whereas HC had received a median
17 years of education (range 12–20) (Table 1).
Two patients had UHDRS motor scores < 5 and were there-
fore classified as pre-manifest HD mutation carriers with dis-
ease burden (CAG index) scores of 210 and 264, respectively.
One HD patient and 5 HC consumed ten or more cigarettes
per day. Alcohol use disorder was an exclusion criterion.
Neuropsychological testing
After fMRI, patients and controls performed four neuropsy-
chological tests for reaction time and attentional function
based on the Test battery for Attentional Performance (TAP,
Zimmermann and Fimm, Psychologische Testsysteme,
Herzogenrath, Germany):
1) Alertness without warning tone (simple reaction to a vi-
sual stimulus): In this test, the average time a subject
needs to press a button after a white cross (x) appears
on a black computer screen is measured.
2) Alertness with warning tone (reaction to a visual stimulus
after a warning tone): This test includes the same task as
in 1), but a warning tone appears before the cross is shown
on the screen.
3) Divided attention (reaction to simultaneous visual and
auditory stimuli): In this task, white crosses and points
move in a four-by-four pattern while a sequence of tones
in two different frequencies is played. Subjects are
instructed to press a button whenever four crosses appear
in the form of a square on the screen and whenever two
tones of the same frequency follow each other. Reaction
times to visual and auditory stimuli are measured
separately.
1864 Brain Imaging and Behavior (2017) 11:1862–1872

Table 1 Sample characteristics

HD (n = 20) HC (n = 24) HD vs. HC
P (t-test)

Age (mean ± SD) 47.5 ± 10.3 years 46.8 ± 11.3 years P = 0.8
Sex (male/female) 14/6 14/10
Educational years (median) 14, range 8–18 17, range 12–20 P = 0.006
CAG repeats (median) 43, range 40–50 n.a.
CAG index (median) 351, range 210–663 n.a.
UHDRS motor score (median) 18, range 1–58 n.a.
TFC (median) 11, range 4–13 n.a.
UHDRS total cognitive score (median) 206, range 120–319 317, range 265–406 P < 0.0001
MMSE (median) 28.5, range 22–30 29, range 26–30 P = 0.014

4) Go/NoGo (Selective attention): Subjects are instructed to reversal occurred with a probability of 20 % during one of the
press a button only when the letter Bx^ appears on the next trials after the criterion was fulfilled.
screen, but not after the sign B + ^ appears. Errors are Twenty patients and 24 controls were included in the final
measured. analysis of the fMRI data. One patient had to discontinue the
MRI examination because of a panic attack. To ensure that
only participants who understood the task were entered into
the final analysis, three HD patients and one HC had to be
Probabilistic reversal learning task excluded from the analysis because they achieved less than 10
reversals. All three excluded HD patients had advanced dis-
A probabilistic reversal learning task was administered during ease stages with low TFC scores (7, 5, 3) and high UHDRS
fMRI. This task required the subjects to detect a reversal in motor scores (29, 52, 72).
reward probabilities and to adapt their behavior accordingly.
To maximize reward, participants had to maintain the advan- Reinforcement learning algorithm
tageous response even when probabilistic errors occurred.
Each participant was introduced to the task in a training ses- A reinforcement learning model was applied to dynamically
sion without reversals. Subjects were instructed to try to win quantify learning, and to generate prediction errors as
as much money as possible and to always try to find the
currently best symbol. After the first training run, we
interviewed the participants to ensure they understood the
task. In case they did not, the instruction and the training were
repeated. Afterwards, they performed the task for the duration
of approximately 30 min (400 trials) in the fMRI scanner. Two
symbols were displayed randomly left and right on the screen
(Fig. 1). The subject had to press one of two buttons to choose
the target as quickly as possible (maximum response time:
1.5 s). Otherwise, the message BToo slow!^ was presented
and the experiment continued. A white square surrounded
the chosen symbol and simultaneously positive or negative
feedback occurred: Feedback for the correct answer consisted
of the picture of a 10 Cent coin and the message BWin!
+10Cent^, feedback for the wrong answer consisted of the
same picture with a crossed out coin and the message BLoss!
-10Cent^. Feedback was presented for 1 s. During the expo-
nentially jittered intertrial interval (1–6.5 s), a fixation cross Fig. 1 Probabilistic reversal learning task: One trial consists of stimulus
appeared on the screen. One of the stimuli was associated with presentation with a response time window (1.5 s), feedback (1 s) and an
80 % monetary reward and 20 % loss, and vice versa. In the intertrial interval of 1.5 s to 6.5 s. Choosing the currently ‘good’ stimulus
leads with a higher probability (80 %) to a reward than choosing the other
main task, these probability assignments switched after the option (20 %). After achieving the criterion of 5 correct answers out of the
participant had learned them. The criterion was defined as 5 last 6 trials, the chance of a reversal of the probability distribution
correct choices out of a sliding window of the last 6 trials. The becomes 20 % for the following trials
Brain Imaging and Behavior (2017) 11:1862–1872
regressors for the analysis of fMRI data. We used a modified
Q-learning algorithm, which learns the expected value Q of
the chosen option c based on the outcome of previous trials
(Sutton and Barto 1998). We adapted this model in order to
update both options during one trial simultaneously (Glascher
et al. 2010; Hauser et al. 2014; Schlagenhauf et al. 2014). At
each trial t, expectation values were adjusted according to the
received feedback:
Qc;t ¼ Qc;t−1 þ αδQc;t−1
Here, α determines the individual learning rate and δ, the
prediction error, is defined as the difference between the ex-
pected and the actually received reinforcement R:
δQc;t ¼ Rc;t −c;t Qc;t
Instead of coding reward and punishment simply as 1 and
−1, we let them vary individually as additional free parameters
for reward and punishment sensitivity.
The double-update algorithm learns values of the unchosen
option u, which was additionally modulated by the weighing
parameter κ:
Qu;t ¼ Qu;t−1 −καδQu;t−1
A softmax estimated the probability of choices based on
the model-derived values. The softmax equation calculates the
likelihood of a subject choosing action a over b on trial t,
which is assumed to be proportional to the expected value of
this option.
expðQa ðt ÞÞ
pa ðt Þ ¼
expðQa ðt ÞÞ þ expðQb ðt ÞÞ
The set of 4 free parameters Θ were fitted individually for
each participant by maximizing the sum of the negative log
likelihood of the sequence of choice c made in trial t:
∑ logPðct jΘÞ:
t
Choice behavior of each participant was explained at least
better than chance (based on the likelihood that the observed
data is given by the parameters). Thus, time-series derived
from these parameters reflect important aspects of the exper-
imental data and can therefore be regressed against imaging
data in a meaningful way. For between-group comparison, all
four parameters were entered into a multivariate analysis of
variance (MANOVA).
MRI data analysis
Functional MRI was conducted using a 3.0 Tesla Siemens trio
scanner with a 12-channel head coil to acquire gradient echo
T2*-weighted echo-planar images. 905 EPI volumes (approx.
31 min) containing 38 slices were measured (repetition time
1865
(TR) = 2060 ms, echo time (TE) = 30 ms, slice thickness
2.5 mm, matrix size 64*64 and field of view (FOV)
224*224 mm2, thus yielding an in-plane voxel resolution of
3.5 mm2, flip angle = 80°). Fieldmaps used to correct for inho-
mogeneities in the magnetic field were acquired with the follow-
ing parameters: TR = 434 ms, TE = 5.19 ms (first) and 7.65 ms
(second), slice thickness = 3.5 mm, matrix size = 64*64 and
FOV = 224*224 mm2, voxel size = 3.5*3.5*3.5 mm3, flip an-
gle = 60°. Two 3D anatomical images of the whole brain were
obtained using a T1-weighted 3D spoiled-gradient echo pulse
sequence with TR = 1900 ms, TE = 2.52 ms, matrix size
256*256, FOV 256*256 mm2, voxel size 1*1*1 mm3, flip an-
gle = 9°. The first anatomical image was acquired before, the
second after the reversal task. Anatomical images were acquired
parallel to the anterior commissure-posterior commissure line,
functional images and fieldmaps were tilted by 25°. fMRI data
were analyzed using SPM8 (http://www.fil.ion.ucl.ac.uk/spm)
in Matlab R2010b (The MathWorks, Natick, MA, USA).
For preprocessing, the following steps were performed:
correction for differences in slice time acquisition and
motion correction including unwarping by using the ac-
quired fieldmaps, co-registration of the mean EPI and the
mean of the two anatomical images, spatial normalization
and segmentation into tissue classes of the T1 image
using the unified segmentation approach as implemented
in SPM8 (Ashburner and Friston 2005). Normalization
parameters were applied to all functional images. Data
were spatially smoothed with an isotropic Gaussian kernel
of 8 mm full width at half maximum kernel. A voxel-
based morphometry analysis was used to estimate gray
matter density on the basis of tissue probability maps
implemented in SPM8. Unmodulated maps from the seg-
mentation step were compared between the two groups
using a two-sample t-test.
For statistical analysis of the fMRI data, the preprocessed
images were analyzed using the general linear model ap-
proach as implemented in SPM8. Data analysis was per-
formed in an event-related manner by modeling feedback on-
set events and adding the prediction errors derived from the
learning model as a parametric modulator. These regressors
were convolved with the canonical hemodynamic response
function provided by SPM8.
No significant difference was found between movement
parameters of HC and HD patients in an ANOVA analysis
of translational movement (all F < 3 for p < 0.05). In addition,
in order to account for movement-associated variance, the six
realignment parameters as well as their first temporal deriva-
tive and a regressor censoring scans with more than 1 mm
scan-to-scan movement were included in the model as addi-
tional regressors of no interest. The individual contrast images
were then taken to a random effects group-level analysis (one-
sample t-test for within group and two-sample t-test for be-
tween group analyses).
1866
For correction of multiple comparisons, statistics are re-
ported using family wise error (FWE) small volume correction
(SVC) at the peak level for a priori region of interest (ROI),
the ventral striatum. A mask for the nucleus accumbens de-
rived from the WFUPickAtlas tool using IBASPM 71
(Maldjian et al. 2003) was applied. Whole brain results are
reported FWE-corrected at peak level and cluster level
(assessed at an uncorrected threshold of p < 0.001).
Parameter estimates were extracted from the ventral-striatal
ROI by averaging across all voxels of this ROI. Since we
had a strong a priori hypothesis about the direction of associ-
ations between the data and measures of disease severity, cor-
relational analysis on these data were performed using one-
sided two-sample t-tests. Gray matter density values from the
VBM analysis were extracted using the same ROI, then aver-
aged over both hemispheres to provide a covariate for RPE-
analysis in the patient group.
Results
Behavior
During probabilistic reversal learning, HD patients displayed
impaired performance compared to HC. HD patients made
less correct choices (mean ± sd: HC = 0.72 ± 0.04, HD =
0.69 ± 0.05, t = 2.4, P < 0.05), and achieved less reversal
stages (mean ± sd: HC = 26.13 ± 5.3, HD = 21.65 ± 5.4, t =
2.8, P < 0.001; Fig. 2). A closer inspection of the learning
curve averaged over all reversal stages suggested that HD
patients were able to adapt their behavior after a reversal, but
were impaired in maintaining the advantageous choice behav-
ior (group by trial interaction F = 4.13, P = 0.005; Fig. 2).
In order to control for differences in reaction times and
impulsive behavior, subjects performed several subtests of
the TAP. HD patients showed slower reaction times than
HC in the subtests Alertness without warning tone and
Alertness with warning tone as well as in the Divided
attention task (Table 2). However, the number of errors
in the Go/NoGo task did not differ between both groups
(Table 2), suggesting that the ability of HD patients to
control impulsive behavior was not significantly reduced
in this test.
When taking into account slower reaction times of the HD
group, the behavioral differences did not remain significant
(ANCOVA with reaction times as covariate: % correct
choices: F = 2.3, p = 0.14; number of reversals: F = 2.7, p =
0.1; % switches: F = 2.1, p = 0.15). However, since reaction
time (here: Alertness without warning tone) is highly related
to motor symptoms (correlation with UHDRS motor score:
r = 0.56, p = 0.01), this measure could be understood as an-
other indicator of disease stage.
Brain Imaging and Behavior (2017) 11:1862–1872
Computational modeling of behavior
With respect to the modeling analysis, the parameter x group
MANOVA revealed a significant group difference for reward
sensitivity Rrew (mean Rrew: HC = 13.4 ± 11.9, HD = 5.7 ± 3.8,
F(1,43) = 7.62, P < 0.01) and for learning rate α (mean α:
HC = 0.72 ± 0.1, HD = 0.78 ± 0.1, F(1,43) = 4.16, P < 0.05). In
addition, UHDRS motor score values of HD patients correlated
negatively with κ (R = −0.5, P < 0.05), indicating that more
advanced HD patients tended to use a simpler single-update
strategy instead of a double-update model, where expectations
for both stimuli are updated during each trial.
Structural imaging
Voxel-based morphometry (VBM) analysis revealed a signif-
icant reduction in gray matter density for HD patients com-
pared to HC in the whole striatum (Fig. 3a and Table 3) in-
cluding its ventral part (left [−8 8–12], t = 3.98, right [6 8–10],
t = 3.87, both P FWE SVC nucleus accumbens < 0.01). There was no
difference in gray matter density between the two hemispheres
in the HD group (p = 0.2). The reduction in ventral striatum
gray matter density correlated negatively with patients’ motor
scores, a measure for the severity of motor symptoms with a
higher score indicating more motor symptoms (right nucleus
accumbens r = −0.548, P < 0.05, trend in left nucleus accum-
bens r = −0.425 P = 0.062; Figs. 3b, c).
In line with the group differences on modeling parameters,
gray matter density correlated positively with reward sensitiv-
ity Rrew (nucleus accumbens left: R = 0.047, P < 0.01, right:
R = 0.04, P < 0.01), and negatively with learning rate α (left
and right: R = −0.41, P < 0.01) in the HD group. In addition to
this analysis, which was focused on the ventral striatum as our
region of interest, we also conducted an exploratory correla-
tional analysis on the whole brain level. Here, we found no
significant association between modeling parameters and gray
matter density. Using a less stringent threshold (p < 0.001,
uncorrected for multiple comparisons, cluster size > 40
voxels), we found a negative correlation between learning rate
and bilateral gray matter density in the cerebellum (left: [−36
−60 −42], cluster size = 93, t = 6.52, p (uncorr.) < 0.001; right
[34–62 −42], cluster size = 42, t = 4.9, p (uncorr.) < 0.001).
Functional imaging
The probabilistic reversal learning task elicited the expected
effect of a RPE signal bilaterally in the ventral striatum in HC
(left [−12 5–11], t = 3.82, right [12 11–11], t = 3.61, both P FWE
SVC nucleus accumbens < 0.01; Fig. 4a), and activation in a RPE
network including insular cortex, cingulate cortex and frontal
and parietal regions (cluster level FWE corrected; Table 4). In
HD patients, RPE-associated activation approached signifi-
cance only in the left ventral striatum ([−12 5–8], t = 2.2, P
Brain Imaging and Behavior (2017) 11:1862–1872 1867

Fig. 2 HD patients are impaired in the probabilistic reversal learning the same performance level as HC as depicted by the learning curves
task. a HD patients achieve significantly less reversal stages than HC. b averaged over the first ten trials after each reversal. Error bars indicate
After a reversal, patients are able to adapt their behavior, but do not reach standard deviation

FWE SVC nucleus accumbens = 0.091; Fig. 4b). Here, RPE parameter
estimates correlated negatively with UHDRS motor score
values (R = −0.4, P < 0.05). No significant group difference
for the RPE between HC and HD was obtained.
Based on the observed correlation of symptom severity
with structural and functional changes, we expected an asso-
ciation between gray matter density and the RPE-related func-
tional activation in patients. Indeed, RPE signal in HD patients
in the left ventral striatum correlated with the degree of gray
matter reduction in this area (R = 0.42, P < 0.05). Given the
apparent reduction in gray matter in the striatum, we used the
individual average of gray matter density from nucleus ac-
cumbens bilaterally as a covariate. This recovered the PE-
related activation in the HD group in the left ventral striatum
([−12 8–8], t = 3.38, P FWE SVC nucleus accumbens = 0.002).
Given the heterogeneity in clinical severity in our HD co-
hort, we explored possible subgroup effects by using the
UHDRS motor score as a measure of disease severity for
creating two subgroups of patients by median split. Motor
score values ranged from 1 to 11 in the low symptom group
(mean = 6.7 ± 3.5), and from 18 to 58 in the high symptom
group (mean = 28 ± 12). Seven males and two females were in
the low symptom group, seven males and four females were in
the high symptom group. The mean age was 43.4 ± 9.6 years
in the low symptom group, and 50.8 ± 10.0 years in the high
symptom group. CAG-repeat lengths ranged from 40 to 45 in
the low symptom group (median = 40), and from 41 to 50 in
the high symptom group (median = 46.5). The median TFC
score was 12 (range = 10–13) in the low symptom group, and
10 (range = 4–13) in the high symptom group. MMSE scores
ranged from 26 to 30 in the low symptom group (median =
29), and from 22 to 30 in the high symptom group (median =
27). Total cognitive scores ranged from 213 to 319 in the low
symptom group (median = 278), and from 120 to 255 in the
high symptom group (median = 183). Patients in the low
symptom group had a median of 14 years of education
(range = 13–18), patients in the high symptom group also
had a median of 14 years of education (range = 8–18)
(Table 5).
Comparison of these two subgroups revealed that the sig-
nificant RPE signal in the left ventral striatum of HD patients
was driven by the less symptomatic group with lower motor
scores. In this group, the RPE signal was significant in the left
nucleus accumbens ([−12 8–11], t = 4.64, P FWE SVC nucleus
accumbens < 0.01), but not on the right. In the high motor score
group (moderate to severe motor impairment), no significant
activation was found (P FWE SVC nucleus accumbens > 0.3).
This result was confirmed in a VOI analysis comparing the
parameter estimates in the left ventral striatum, which revealed
a significant effect of group (F = 3.2, p = 0.049). This was

Table 2 TAP results

HD (n = 20) HC (n = 24) HD vs. HC

Mean SD Mean SD P (t-test)

Alertness without warning tone 338.9 ms 84.7 ms 235.6 ms 34.8 ms P < 0.0001
Alertness with warning tone 335.9 ms 92.9 ms 233.4 ms 41.9 ms P < 0.0001
Divided attention (visual) 1031.4 ms 273.6 ms 837.8 ms 136.8 ms P < 0.01
Divided attention (audible) 759.5 ms 210.9 ms 573.0 ms 86.8 ms P < 0.001
Go/NoGo (mistakes) 2.2 1.9 1.6 2.3 P = 0.345
1868 Brain Imaging and Behavior (2017) 11:1862–1872

Fig. 3 a Voxel-based morphometry (VBM) results reveal a significant main effect as depicted in a. c Overlay of the correlation, here depicted in
reduction in gray matter density in HD patients in the striatum compared blue, over the contrast between controls and patients as seen in a, for
to healthy controls. b Individual UHDRS motor score values of HD illustrative purposes thresholded at P = 0.001 uncorr., y = 9, color bar
patients correlate with gray matter density in the striatum (masked with indicates t-values

driven by a difference between the two HD subgroups
(P < 0.05; Fig. 5). The HC group was not different from any
of the two HD groups, but was descriptively found in between
(mean HC = 0.22 ± 0.35, mean low symptom HD = 3.5 ±
0.23, mean high symptom HD = 0.006 ± 0.26).
Discussion
This is the first study to examine reversal learning in symp-
tomatic HD patients without CNS-active medication using
functional and structural MRI. We found that HD patients
were able to perform the reversal learning task, even though
they reached less reversals than healthy controls.
Computational modeling of behavior revealed a reduced
reward sensitivity and an increased learning rate in the patient
group. Interestingly, these parameters were associated with
gray matter reductions in the ventral striatum. On the neural
level, we found RPE encoding in the ventral striatum in the
patient group when taking into account the gray matter al-
terations. An exploratory analysis with two subgroups in-
dicated that there might be an overactivation during early
disease stage.
Notably, we observed that HD patients (except three of
them) were able to adapt their behavior after a reversal had
occurred, but were impaired in maintaining the advantageous
choice behavior, i.e. they switched more often between sym-
bols before the next reversal occurred. This effect is captured
by the higher learning rate in HD patients. Our observation is
in accordance with the results of a neuropsychological study

Table 3 Regions showing a

significant (P < 0.05) difference p cluster k p peak t x y z
between HC and HD in VBM
analysis at cluster level FEW- Caudate 0.000 2392 0.01 6.14 20 12 14
corrected (assessed at P < 0.001) Putamen 0.451 4.55 22 5 5
Thalamus 0.599 4.4 −20 −22 20
Subcallosal Gyrus 0.981 3.9 5 8 −12
Anterior Cingulate 1 3.5 −4 18 −10
Inferior Temporal Gyrus 0.000 1323 0.099 5.32 52 −76 −8
Middle Occipital Gyrus 0.314 4.85 35 −90 −8
Middle Temporal Gyrus 0.528 4.58 54 −55 0
Cuneus 0.748 4.34 20 −100 2
Middle Temporal Gyrus 0.006 387 0.182 5.08 68 −42 −10
Middle Temporal Gyrus 0.001 510 0.192 5.06 −64 −46 −14
Insula 0.002 459 0.309 4.85 −40 2 −4
Claustrum 0.555 4.43 −35 8 4
Inferior Semi-Lunar Lobule 0.068 219 0.464 4.65 40 −72 −52
Middle Temporal Gyrus 0.037 260 0.62 4.48 62 4 −36
Middle Temporal Gyrus 0.079 210 0.778 4.3 −60 −64 4
Transverse Temporal Gyrus 0.001 564 0.805 4.27 46 −26 14
Insula 0.827 4.24 42 −4 4
Precuneus 0.031 272 0.839 4.22 −2 −52 36
Brain Imaging and Behavior (2017) 11:1862–1872 1869

Fig. 4 a Prediction error signal in

24 HC, for illustrative purposes at
t = 3, and b 20 HD patients,
P = 0.05 uncorrected at y = 11,
color bar indicates t-values

Table 4 Regions with significant

(P < 0.05) and trendwise (P < 0.1) p cluster k p peak t x y z
prediction error activation for HC
at cluster level FEW-corrected Middle Frontal Gyrus 0.000 6039 0.000 8.963 42 47 −11
(assessed at P < 0.001) Insula 0.000 8.887 −33 −1 13
Inferior Frontal Gyrus 0.001 8.280 48 41 −11
Superior Frontal Gyrus 0.002 7.844 15 41 46
Medial Frontal Gyrus 0.004 7.623 9 47 7
Claustrum 0.016 7.010 −36 −1 −2
Middle Frontal Gyrus 0.017 6.968 −33 47 −8
Medial Frontal Gyrus 0.025 6.768 −6 47 7
Anterior Cingulate 0.026 6.738 −3 41 7
Putamen 0.031 6.645 −24 −7 1
Middle Temporal Gyrus 0.000 1968 0.001 8.366 57 −37 −11
Postcentral Gyrus 0.001 8.167 60 −25 40
Inferior Parietal Lobule 0.019 6.919 60 −37 34
Superior Parietal Lobule 0.042 6.487 42 −58 58
Supramarginal Gyrus 0.324 5.330 60 −55 34
Angular Gyrus 0.532 4.984 48 −64 31
Middle Temporal Gyrus 0.552 4.954 39 −55 25
Inferior Parietal Lobule 0.000 1687 0.002 7.882 −60 −40 40
Supramarginal Gyrus 0.002 7.855 −63 −43 37
Middle Temporal Gyrus 0.051 6.382 −66 −49 −2
Postcentral Gyrus 0.061 6.289 −51 −28 19
Superior Parietal Lobule 0.535 4.979 −30 −58 67
Inferior Parietal Lobule 0.575 4.921 −51 −43 58
Angular Gyrus 0.631 4.841 −60 −64 40
Pyramis 0.000 1751 0.052 6.368 −15 −82 −32
Inferior Semi-Lunar Lobule 0.191 5.649 −33 −79 −44
Inferior Semi-Lunar Lobule 0.591 4.897 36 −76 −44
Pyramis 0.664 4.792 27 −73 −29
Cerebellar Tonsil 0.936 4.296 −48 −55 −41
Superior Frontal Gyrus 0.053 76 0.265 5.457 −36 26 49
Cingulate Gyrus 0.000 323 0.320 5.338 3 −34 37
Precuneus 0.452 5.106 −6 −49 31
Posterior Cingulate 0.463 5.089 0 −40 22
Precuneus 0.722 4.708 9 −46 34
Cuneus 0.012 110 0.834 4.527 −15 −106 −5
Inferior Occipital Gyrus 0.957 4.223 −24 −97 −11
1870
Table 5 Sample characteristics
of the two subgroups of HD
patients
Age (mean ± SD)
Sex (male/female)
Educational years (median)
CAG repeats (median)
UHDRS Motor score (median)
TFC (median)
UHDRS total cognitive score (median)
MMSE (median)
using a similar probabilistic discrimination and reversal
task in HD patients and HC (Lawrence et al. 1999).
However, we did not observe the perseverative responding
in HD patients described in this study. Reduced reaction
times as a possible confounding factor need to be consid-
ered when interpreting the behavioral data. It remains to be
determined whether reduced reaction times or cognitive
deficits are the cause of decreased performance, or wether
these measures simply covary because they are both related
to disease stage. To this end, a different task design is re-
quired, which allows to disentangle the effects of cognitive
and motor impairments.
The finding of a significant bilateral reduction of gray mat-
ter density in the ventral striatum is in accordance with the
results of numerous other MRI-based studies in HD patients
(Aylward et al. 2012; Kipps et al. 2005; Muhlau et al. 2007;
Weir et al. 2011).
Findings on RPE-related activation in a network includ-
ing ventral striatum, insular cortex, cingulate cortex and
frontal and parietal regions in HC have been reported pre-
viously (D’Ardenne et al. 2008; Haber and Knutson 2010;
O’Doherty et al. 2003). Importantly, we found that HD
patients showed significant RPE signals only in the left
Fig. 5 Parameter estimates from the left ventral striatum in HC and two
subgroups of HD patients differing in the severity of their motor
symptoms. While the group with less severe motor symptoms (HD
low) displays an activation related to reward prediction error, prediction
error encoding of the group of patients with advanced motor stage (HD
high) was significantly lower. Error bars indicate standard deviation
Brain Imaging and Behavior (2017) 11:1862–1872
HD low symptom group HD high symptom group P (t-test)
43.4 ± 9.6 50.8 ± 10.0 P = 0.1
7/2 7/4
14, range 13–18 14, range 8–18 P = 0.8
42, range 40–45 46.5, range 41–50 P < 0.005
7, range 1–11 23, range 18–58 P < 7 × 10−5
12, range 10–13 10, range 4–13 P < 0.1
278, range 213–319 183, range 120–255 P < 5 × 10−6
29, range 26–30 27, range 22–30 P = 0.07
ventral striatum. In a closer analysis, this effect was prob-
ably driven by the low motor score subgroup (UHDRS
motor score ≤11). Interestingly, several morphometry stud-
ies recently detected a leftward-biased striatal loss of gray
matter in HD (Kipps et al. 2005; Muhlau et al. 2007;
Thieben et al. 2002). Given this leftward-biased atrophy
in striatal structures in HD patients, the RPE signal in the
low motor score subgroup could be interpreted as a func-
tional hypercompensation by remaining neurons in the ven-
tral striatum in early disease stages. This has been
discussed before (Andre et al. 2010), and would be in line
with fMRI studies in patients with premanifest and early
Alzheimer’s disease, which revealed similar functional
hypercompensation in hippocampal and frontoparietal
structures (Quiroz et al. 2010; Wishart et al. 2006).
However, our findings need to be interpreted with caution
considering the small group sizes and the exploratory char-
acter of the analysis. In contrast to our results, Saft and
colleagues found a rightward lateralization of hemispheric
activation during auditory processing in response to pre-
sentation of sine tones in manifest HD patients (Saft et al.
2008). The studies are difficult to compare, but it is inter-
esting to note that both observed lateralization effects with
disease progression in HD.
We observed a higher learning rate in HD patients, which
was associated with the reduction in gray matter density. A
higher learning rate is not in general an indicator for better task
performance. In fact, due to the occurrence of probabilistic
events, an immediate reaction to negative feedback does not
improve performance level. Our findings are in line with re-
cent evidence for functional compensation in prodromal HD
using a Monetary Incentive task (Malejko et al. 2014).
In an instrumental learning task, Palminteri and colleagues
observed decreased performance of presymptomatic and
symptomatic HD patients during the loss condition, while
only symptomatic patients showed impairment in the gain
condition (Palminteri et al. 2012). Enzi and colleagues found
an impairment of ventral striatal blood-oxygen-level-
dependent (BOLD) signal during punishment anticipation in
presymptomatic HD patients near disease onset compared to
Brain Imaging and Behavior (2017) 11:1862–1872
HC, but no group difference between the patients groups near
and far from disease onset (Enzi et al. 2012). Our analysis
focused on the RPE-related activation during the feedback
phase and our task was not designed to differentiate between
reward and punishment contributions, which limits direct
comparability. However, we did find differences in reward sen-
sitivity and learning rate in the behavioral model analysis. These
findings differ from the modeling results reported by Palminteri
and colleagues, who only observed a difference in choice ran-
domness (Palminteri et al. 2012). It is important to note here that
our algorithm contained an additional free parameter, which
alters the estimation of all free parameters. This additional pa-
rameter κ describes whether participants tend to use more simple
single-update strategies, or more complex double-update strate-
gies. κ was found to be associated with symptom severity, indi-
cating that HD patients in an advanced stage of disease might
use a less complex strategy to solve the task.
In conclusion, our data suggest that symptomatic HD
patients can be examined by fMRI even in moderate to
advanced disease stages without any potentially interfer-
ing CNS-active medication. Our results portray how be-
havior and RPE-related brain activation during a reversal
learning task are related to disease stages in HD. The
exploratory analysis of two subgroups suggest a possible
hypercompensation in the degenerating left ventral stria-
tum. This effect is apparently lost with disease progres-
sion. Small sample sizes and disease-related differences in
cognitive capacities between the HD and the HC groups
have to be considered as possible confounding factors and
thereby as limitations of our study. Larger longitudinal
trials are needed to assess whether performance in fMRI
reversal learning tasks could be used as a non-invasive
marker of disease progression in HD.
Acknowledgements We wish to thank Dirk Lang for help with pilot
experiments. This work was supported by grants from the Cluster of
Excellence NeuroCure (to J.P.). R.B. received funding by the Deutsche
Forschungsgemeinschaft (GRK-1123). The funding sources were not in-
volved in study design; in the collection, analysis and interpretation of
data; in the writing of the report; or in the decision to submit the article for
publication.
Compliance with ethical standards
Conflict of interest Katharina Nickchen, Rebecca Boehme, Maria del
Mar Amador, Thomas D. Hälbig, Katharina Dehnicke, Patricia Panneck,
Joachim Behr, Konstantin Prass, Andreas Heinz, Lorenz Deserno, Florian
Schlagenhauf and Josef Priller declare that they have no conflict of
interest.
Informed consent All procedures followed were in accordance with
the ethical standards of the responsible committee on human experimen-
tation (institutional and national) and with the Helsinki Declaration of
1975, and the applicable revisions at the time of the investigation.
Informed consent was obtained from all participants for being included
in the study.
1871
References
Andre, V. M., Cepeda, C., & Levine, M. S. (2010). Dopamine and glutamate
in Huntington’s disease: a balancing act. CNS Neuroscience and
Therapeutics, 16(3), 163–178. doi:10.1111/j.1755-5949.2010.00134.x.
Ashburner, J., & Friston, K. J. (2005). Unified segmentation. NeuroImage,
26(3), 839–851. doi:10.1016/j.neuroimage.2005.02.018.
Aylward, E. H., Nopoulos, P. C., Ross, C. A., Langbehn, D. R., Pierson,
R. K., Mills, J. A., … Paulsen, J. S. (2011). Longitudinal change in
regional brain volumes in prodromal Huntington disease. Journal of
Neurology, Neurosurgery, and Psychiatry, 82(4), 405–410.
doi:10.1136/jnnp.2010.208264.
Aylward, E. H., Liu, D., Nopoulos, P. C., Ross, C. A., Pierson, R. K., Mills,
J. A., … Paulsen, J. S. (2012). Striatal volume contributes to the pre-
diction of onset of Huntington disease in incident cases. Biological
Psychiatry, 71(9), 822–828. doi:10.1016/j.biopsych.2011.07.030.
Barker, R., & Priller, J. (2013). Huntington’s disease. In D. Burn (Ed.),
Oxford textbook of movement disorders. Oxford: Oxford University
Press.
D’Ardenne, K., McClure, S. M., Nystrom, L. E., & Cohen, J. D. (2008).
BOLD responses reflecting dopaminergic signals in the human ven-
tral tegmental area. Science, 319(5867), 1264–1267. doi:10.1126
/science.1150605.
Enzi, B., Edel, M. A., Lissek, S., Peters, S., Hoffmann, R., Nicolas, V., …
Saft, C. (2012). Altered ventral striatal activation during reward and
punishment processing in premanifest Huntington’s disease: a func-
tional magnetic resonance study. Experimental Neurology, 235(1),
256–264. doi:10.1016/j.expneurol.2012.02.003.
Glascher, J., Daw, N., Dayan, P., & O’Doherty, J. P. (2010). States versus
rewards: dissociable neural prediction error signals underlying
model-based and model-free reinforcement learning. Neuron,
66(4), 585–595. doi:10.1016/j.neuron.2010.04.016.
Haber, S. N., & Knutson, B. (2010). The reward circuit: linking primate
anatomy and human imaging. Neuropsychopharmacology, 35(1),
4–26. doi:10.1038/npp.2009.129.
Hauser, T. U., Iannaccone, R., Ball, J., Mathys, C., Brandeis, D., Walitza,
S., & Brem, S. (2014). Role of the medial prefrontal cortex in im-
paired decision making in juvenile attention-deficit/hyperactivity
disorder. JAMA Psychiatry, 71(10), 1165–1173. doi:10.1001
/jamapsychiatry.2014.1093.
Kipps, C. M., Duggins, A. J., Mahant, N., Gomes, L., Ashburner, J., &
McCusker, E. A. (2005). Progression of structural neuropathology
in preclinical Huntington’s disease: a tensor based morphometry
study. Journal of Neurology, Neurosurgery, and Psychiatry, 76(5),
650–655. doi:10.1136/jnnp.2004.047993.
Lawrence, A. D., Sahakian, B. J., Rogers, R. D., Hodge, J. R., & Robbins,
T. W. (1999). Discrimination, reversal, and shift learning in
Huntington’s disease: mechanisms of impaired response selection.
Neuropsychologia, 37(12), 1359–1374.
Maldjian, J. A., Laurienti, P. J., Kraft, R. A., & Burdette, J. H. (2003). An
automated method for neuroanatomic and cytoarchitectonic atlas-
based interrogation of fMRI data sets. NeuroImage, 19(3), 1233–
1239.
Malejko, K., Weydt, P., Sussmuth, S. D., Gron, G., Landwehrmeyer, B.
G., & Abler, B. (2014). Prodromal huntington disease as a model for
functional compensation of early neurodegeneration. PLoS One,
9(12), e114569. doi:10.1371/journal.pone.0114569.
Montague, P. R., Dayan, P., & Sejnowski, T. J. (1996). A framework for
mesencephalic dopamine systems based on predictive Hebbian
learning. The Journal of Neuroscience, 16(5), 1936–1947.
Muhlau, M., Gaser, C., Wohlschlager, A. M., Weindl, A., Stadtler, M.,
Valet, M., … Peinemann, A. (2007). Striatal gray matter loss in
Huntington’s disease is leftward biased. Movement Disorders,
22(8), 1169–1173. doi:10.1002/mds.21137.
1872
O’Doherty, J. P., Dayan, P., Friston, K., Critchley, H., & Dolan, R. J.
(2003). Temporal difference models and reward-related learning in
the human brain. Neuron, 38(2), 329–337.
Palminteri, S., Justo, D., Jauffret, C., Pavlicek, B., Dauta, A., Delmaire,
C., … Pessiglione, M. (2012). Critical roles for anterior insula and
dorsal striatum in punishment-based avoidance learning. Neuron,
76(5), 998–1009. doi:10.1016/j.neuron.2012.10.017.
Quiroz, Y. T., Budson, A. E., Celone, K., Ruiz, A., Newmark, R.,
Castrillon, G., … Stern, C. E. (2010). Hippocampal hyperactivation
in presymptomatic familial Alzheimer’s disease. Annals of
Neurology, 68(6), 865–875. doi:10.1002/ana.22105.
Saft, C., Schuttke, A., Beste, C., Andrich, J., Heindel, W., & Pfleiderer, B.
(2008). fMRI reveals altered auditory processing in manifest and
premanifest Huntington’s disease. Neuropsychologia, 46(5), 1279–
1289. doi:10.1016/j.neuropsychologia.2007.12.002.
Schlagenhauf, F., Huys, Q. J., Deserno, L., Rapp, M. A., Beck, A.,
Heinze, H. J., … Heinz, A. (2014). Striatal dysfunction during re-
versal learning in unmedicated schizophrenia patients. NeuroImage,
89, 171–180. doi:10.1016/j.neuroimage.2013.11.034.
Brain Imaging and Behavior (2017) 11:1862–1872
Schultz, W., Dayan, P., & Montague, P. R. (1997). A neural substrate of
prediction and reward. Science, 275(5306), 1593–1599. doi:10.1126
/science.275.5306.1593.
Sutton, R., & Barto, A. (1998). Reinforcement learning: an introduction.
Cambridge: MIT Press.
Thieben, M. J., Duggins, A. J., Good, C. D., Gomes, L., Mahant, N.,
Richards, F., … Frackowiak, R. S. (2002). The distribution of struc-
tural neuropathology in pre-clinical Huntington’s disease. Brain,
125(Pt 8), 1815–1828.
Weir, D. W., Sturrock, A., & Leavitt, B. R. (2011). Development of
biomarkers for Huntington’s disease. Lancet Neurology, 10(6),
573–590. doi:10.1016/s1474-4422(11)70070-9.
Wishart, H. A., Saykin, A. J., Rabin, L. A., Santulli, R. B., Flashman, L.
A., Guerin, S. J., … McAllister, T. W. (2006). Increased brain acti-
vation during working memory in cognitively intact adults with the
APOE epsilon4 allele. American Journal of Psychiatry, 163(9),
1603–1610. doi:10.1176/appi.ajp.163.9.1603.