Doctoral Consortium
Towards an Instructional Design of Complex Learning in
Introductory Programming Courses
Rodrigo Duran
Aalto University
Aalto, Finland, 00076
rodrigo.duran@aalto.fi
ABSTRACT
To provide effective learning strategies to complex content, edu-
cational psychology produced frameworks that holistically inte-
grate skills at a controlled pace achieved by sequencing tasks by its
complexity. However, to apply such frameworks to introductory
programming courses, there is a dearth of established methodolo-
gies for assessing how difficult such tasks are from the student’s
cognitive perspective. My goals are twofold: in order to design
an instructional intervention that applies and adapt frameworks
aimed to complex learning, I first establish and empirically validate
a model to evaluate the complexity of programs from a cognitive
perspective, providing metrics to quantify different aspects of pro-
gram’s complexity. I want to explore how to refine and expand this
model analysis, what kind of evidence can be extracted from its
empirical validation and how to impact instructional design with
the results of my work.
CCS CONCEPTS
• Social and professional topics → Computer science educa-
tion; Model curricula; Student assessment;
KEYWORDS
Model of Hierarchical Complexity; Cognitive Load Theory; Program
Cognitive Complexity; Complexity; Plan-Composition Strategies
ACM Reference Format:
Rodrigo Duran. 2018. Towards an Instructional Design of Complex Learning
in Introductory Programming Courses. In ICER ’18: 2018 International Com-
puting Education Research Conference, August 13–15, 2018, Espoo, Finland.
ACM, New York, NY, USA, 2 pages. https://doi.org/10.1145/3230977.3231007
1 PROGRAM CONTEXT
I am in my 3rd year of doctoral studies in Computer Science Educa-
tion (CSE) at Aalto University under the supervision of Prof. Lauri
Malmi. I recently first-authored one paper outlining the theoretical
framework of my thesis [4]. As I intend to defend my dissertation
in 2020, I am working towards publishing papers on my model’s
empirical validation and evaluation of my pedagogical intervention.
Permission to make digital or hard copies of part or all of this work for personal or
classroom use is granted without fee provided that copies are not made or distributed
for profit or commercial advantage and that copies bear this notice and the full citation
on the first page. Copyrights for third-party components of this work must be honored.
For all other uses, contact the owner/author(s).
ICER ’18, August 13–15, 2018, Espoo, Finland
© 2018 Copyright held by the owner/author(s).
ACM ISBN 978-1-4503-5628-2/18/08.
https://doi.org/10.1145/3230977.3231007
262
ICER ’18, August 13–15, 2018, Espoo, Finland
2 CONTEXT AND MOTIVATION
To develop successful learning interventions, instructional design-
ers must consider student’s learning trajectory. Ideally, tasks should
not overwhelm students but at the same time challenge them, as
proposed by Vygotsky’s Zone of Proximal Development. To assist
teachers in designing instruction to complex content, research in ed-
ucational psychology has produced frameworks such as the 4C/ID
[11]. At the core of this framework is the required skill to sequence
tasks based on its complexity, therefore promoting effective schema
construction and automation. However, there is a gap in research
since current methods described by Computing Education Research
(CER) do not formally define or provide enough detail in how each
program’s components impact program’s complexity from a cogni-
tive perspective. How to compare program’s complexity to another?
What makes a program complex?
3 BACKGROUND & RELATED WORK
While Bloom’s and SOLO taxonomies were extensively used by
CER to classify activities [5, 6], this work aims to complement such
approaches by characterizing the content of an activity, not the ac-
tivity itself or the connectedness of its outputs. Whether discussing
how programming structures can have different complexities (“Is
an assignment statement easier to read and understand than a print
statement?” [9]), or building learning trajectories to CSE [8], under-
standing the complexity of programs by evaluating its structures is
a significant part of the analytical work.
The Cognitive Load Theory (CLT) [1] aims to improve the in-
structional design by investigating how learning is constrained by a
limited working memory (WM) where a small number of schemas,
chunks of related elements, are processed. Element interactivity
(EI) determines the degree of interconnectedness between elements
in the WM, which is the main component of Intrinsic Load (IL), the
amount of simultaneously necessary elements required for learn-
ing, characterizing complexity in CLT. As experience grows, we
construct hierarchies of increasingly complex higher-level schemas
that encompass many low-lever schemas, which in turn decreases
IL by reducing the number of active elements in the WM. Soloway
[10] introduced a plans (schema) and goals tree in the programming
context representing abstractions of strategies to solve a problem
that students later realize by composing a program with code.
The Model of Hierarchical Complexity (MHC) [3] is a Neo-
Piagetian framework applied to analyze the complexity of tasks. It
defines stages of development based on a hierarchy of actions orga-
nized by coordinating rules. The axioms of MHC establishes that a
more complex action coordinates in a non-arbitrary way at least
two lower-level actions, thus defining complexity as a recursion
over constituent elements of a task.
Doctoral Consortium
4 STATEMENT OF THE THESIS/PROBLEM
My work aims to inform the instructional design of a complex
subject (programming) to novices (introductory courses) adapting
theoretical frameworks from educational psychology specifically
intended for complex tasks. To be able to achieve my overall goal,
first I aim to develop and validate a model of cognitive complexity
that enables an a priori fine-grained analysis of programs. This
model extends the previous analysis of plans by providing metrics
to evaluate different facets of complexity, including the trajectory of
schema (plan) acquisition, and an evaluation of how plans interact-
ing using different code composition strategies impact complexity.
5 RESEARCH GOALS & METHODS
Conceptualize a model to evaluate computer programs from
a cognitive perspective: The Cognitive Complexity of Computer
Programs (CCCP) framework is a theoretical model for reasoning
about the complexity of computer programs and generating metrics
that summarize aspects of complexity [4]. The CCCP analyzes
the schema construction applying the MHC axioms to create a
hierarchical representation of the plans in a program, defining
the plan depth metric. The plan interactivity metric, an adaptation
of CLT’s element interactivity, estimates which plans need to be
kept in the WM simultaneously while the programmer mentally
manipulates the (higher-level) plans of the program using a given
code composition strategy.
Validate the framework with empirical results: I will col-
lect data from CS1 and K12 students to evaluate their program
comprehension using tracing and code completion exams with
different levels of complexity. The data consists of performance
measurements, subjective ratings of clarity and difficulty[7] and
identifiable concepts to evaluate the program. A pre-test will be
administered to select subjects with no previous knowledge, and a
post-test will summarize a rank of program complexity. The exams
will be administered along with the regular coursework in order
to ensure that they are measuring schema construction for each
content. Performance outcomes will be analyzed by an extended
Rasch model [2] to rank the exam items. I will investigate a correla-
tion between the predicted complexity of programs with student’s
performance and triangulate it with existing methods [7] using
quantitative and qualitative data.
Create a pedagogical intervention for CS1 informed by an
instruction design aimed at complex learning: My goal is to
adapt the 4C/ID framework to create a pedagogical intervention
designing learning tasks and performance assessments. I will se-
quence learning tasks to create tasks classes (same knowledge con-
tent, varying complexity), design supportive information, mental
models representations, part-task practices and cognitive rules. I
will also perform analysis of cognitive strategies and pre-requisite
knowledge aimed at introductory programming courses.
6 DISSERTATION STATUS
I have conceptualized the first iteration of the CCCP model using
a series of case studies of short programs as a proof of concept to
present the plan depth and plan interactivity metrics. To validate
this model I will pilot the first data collection with a smaller cohort
(N=200) of CS1 students, refine the model and the data collection
263
ICER ’18, August 13–15, 2018, Espoo, Finland
instrument to later collect data in a larger cohort (N=600) of CS1
students from the same university. I expect that this data can val-
idate the model and provide me enough insight and evidence to
design the final pedagogical intervention.
Since my model is not yet supported by empirical evidence, at
the doctoral consortium I would like to discuss its limitations and
possible refinements to improve it. I hope to receive feedback on
my empirical validation plan and discuss how to make it more
reliable, what kind of additional data could be collected and how to
present concrete evidence of the model validation. Since the model
and data analysis itself is a necessary step to design a pedagogical
intervention aimed to teach a complex subject (programming), I
hope to receive feedback on future applications of such intervention,
how to evaluate it and future extensions of the model.
7 EXPECTED CONTRIBUTIONS
This research will improve the conceptualization and evaluation of
program’s cognitive complexity providing a more detailed analysis
based on the content of programs and the cognitive effort demanded
to comprehend them. The final result of my work, a pedagogical
intervention aimed for complex learning of programming may
influence practitioners, teachers, and examiners. My model can
influence stakeholders in several CER areas, such as: assessment,
creating learning trajectories to CS1 courses, personalized exams,
design of “more learnable” programming languages based on the
complexity of its structures, plan-composition strategies and its
impact on learning and performance, and theoretical impacts of
interventions that deal with complexity of programs.
REFERENCES
[1] Hwan-Hee Choi, Jeroen JG Van Merriënboer, and Fred Paas. 2014. Effects of the
physical environment on cognitive load and learning: towards a new model of
cognitive load. Educational Psychology Review 26, 2 (2014), 225–244.
[2] Michael Lamport Commons, Eric Andrew Goodheart, Alexander Pekker,
Theo Linda Dawson, Karen Draney, and Kathryn Marie Adams. 2008. Using
Rasch scaled stage scores to validate orders of hierarchical complexity of balance
beam task sequences. Journal of Applied Measurement 9, 2 (2008), 182.
[3] Michael Lamport Commons, Edward James Trudeau, Sharon Anne Stein, Fran-
cis Asbury Richards, and Sharon R Krause. 1998. Hierarchical complexity of tasks
shows the existence of developmental stages. Developmental Review 18, 3 (1998),
237–278.
[4] Rodrigo Duran, Juha Sorva, and Sofia Leite. In review. Towards an analysis of
program complexity from a cognitive perspective. In review (In review).
[5] Cruz Izu, Amali Weerasinghe, and Cheryl Pope. 2016. A study of code design
skills in novice programmers using the SOLO taxonomy. In Proceedings of the 2016
ACM Conference on International Computing Education Research. ACM, 251–259.
[6] Susana Masapanta-Carrión and J. Ángel Velázquez-Iturbide. 2018. A systematic
review of the use of Bloom’s Taxonomy in computer science education. In Pro-
ceedings of the 49h ACM Technical Symposium on Computer Science Education
(SIGCSE ’18). ACM, 441–446.
[7] Briana B Morrison, Brian Dorn, and Mark Guzdial. 2014. Measuring cognitive
load in introductory CS: adaptation of an instrument. In Proceedings of the tenth
annual conference on International computing education research. ACM, 131–138.
[8] Kathryn M. Rich, Carla Strickland, T. Andrew Binkowski, Cheryl Moran, and
Diana Franklin. 2017. K-8 Learning Trajectories Derived from Research Literature:
Sequence, Repetition, Conditionals. In Proceedings of the 2017 ACM Conference
on International Computing Education Research (ICER ’17). ACM, New York, NY,
USA, 182–190. https://doi.org/10.1145/3105726.3106166
[9] Simon, Mike Lopez, Ken Sutton, and Tony Clear. 2009. Surely we must learn
to read before we learn to write!. In Conferences in Research and Practice in
Information Technology Series (ACE ’09), Margaret Hamilton and Tony Clear
(Eds.), Vol. 95. Australian Computer Society, 165–170.
[10] Elliot Soloway. 1986. Learning to program = Learning to construct mechanisms
and explanations. Commun. ACM 29, 9 (1986), 850–858.
[11] Jeroen JG Van Merriënboer and Paul A Kirschner. 2017. Ten steps to complex
learning: A systematic approach to four-component instructional design. Routledge.