1.

Age differences in learning emerge from an insufficient representation of

uncertainty in older adults
https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4906358/
Abstract: Healthy aging can lead to impairments in learning that affect many
laboratory and real-life tasks. These tasks often involve the acquisition of
dynamic contingencies, which requires adjusting the rate of learning to
environmental statistics. For example, learning rate should increase when
expectations are uncertain (uncertainty), outcomes are surprising (surprise)
or contingencies are more likely to change (hazard rate). In this study, we
combine computational modelling with an age-comparative behavioural study
to test whether age-related learning deficits emerge from a failure to optimize
learning according to the three factors mentioned above. Our results suggest
that learning deficits observed in healthy older adults are driven by a
diminished capacity to represent and use uncertainty to guide learning.
These findings provide insight into age-related cognitive changes and
demonstrate how learning deficits can emerge from a failure to accurately
assess how much should be learned.

2. Risk, Unexpected Uncertainty, and Estimation Uncertainty: Bayesian Learning in

Unstable Settings
https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3024253/
Abstract: Recently, evidence has emerged that humans approach learning
using Bayesian updating rather than (model-free) reinforcement algorithms in
a six-arm restless bandit problem. Here, we investigate what this implies for
human appreciation of uncertainty. In our task, a Bayesian learner
distinguishes three equally salient levels of uncertainty. First, the Bayesian
perceives irreducible uncertainty or risk: even knowing the payoff
probabilities of a given arm, the outcome remains uncertain. Second, there is
(parameter) estimation uncertainty or ambiguity: payoff probabilities are
unknown and need to be estimated. Third, the outcome probabilities of the
arms change: the sudden jumps are referred to as unexpected uncertainty.
We document how the three levels of uncertainty evolved during the course
of our experiment and how it affected the learning rate. We then zoom in on
estimation uncertainty, which has been suggested to be a driving force in
exploration, in spite of evidence of widespread aversion to ambiguity. Our
data corroborate the latter. We discuss neural evidence that foreshadowed
the ability of humans to distinguish between the three levels of uncertainty.
Finally, we investigate the boundaries of human capacity to implement
Bayesian learning. We repeat the experiment with different instructions,

reflecting varying levels of structural uncertainty. Under this fourth notion of

uncertainty, choices were no better explained by Bayesian updating than by
(model-free) reinforcement learning. Exit questionnaires revealed that
participants remained unaware of the presence of unexpected uncertainty
and failed to acquire the right model with which to implement Bayesian
updating.

3. On Surprise, Change, and the Effect of Recent Outcomes

https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3283116/
Abstract: The leading models of human and animal learning rest on the
assumption that individuals tend to select the alternatives that led to the best
recent outcomes. The current research highlights three boundaries of this
recency assumption. Analysis of the stock market and simple laboratory
experiments suggests that positively surprising obtained payoffs, and
negatively surprising forgone payoffs reduce the rate of repeating the
previous choice. In addition, all previous trails outcomes, except the latest
outcome (most recent), have similar effect on future choices. We show that
these results, and other robust properties of decisions from experience, can
be captured with a simple addition to the leading models: the assumption
that surprise triggers change.
4.