University of Nebraska Omaha

DigitalCommons@UNO
Psychology Faculty Publications

Department of Psychology

Spring 2000

Development of leadership skills: Experience and

timing
Michael D. Mumford
University of Oklahoma

Michelle A. Marks
Florida International University

American Institutes for Research

Stephen J. Zaccaro
George Mason University

Roni Reiter-Palmon
University of Nebraska at Omaha, rreiter-palmon@unomaha.edu

Follow this and additional works at: http://digitalcommons.unomaha.edu/psychfacpub

Recommended Citation
Mumford, Michael D.; Marks, Michelle A.; American Institutes for Research; Zaccaro, Stephen J.; and Reiter-Palmon, Roni,
"Development of leadership skills: Experience and timing" (2000). Psychology Faculty Publications. Paper 64.
http://digitalcommons.unomaha.edu/psychfacpub/64

This Article is brought to you for free and open access by the Department
of Psychology at DigitalCommons@UNO. It has been accepted for
inclusion in Psychology Faculty Publications by an authorized
administrator of DigitalCommons@UNO. For more information, please
contact unodigitalcommons@unomaha.edu.

Development of leadership skills: Experience

and timing

Michael D Mumforda, Michelle A Marksb, Mary Shane Connellyc,

Stephen J Zaccarod, Roni Reiter-Palmone

____________________________
a

University of Oklahoma, Norman, OK USA

Florida International University, Miami, FL and Management Research Institute, Bethesda, MD USA

American Institutes for Research, Fairfax, VA and Management Research Institute, Bethesda, MD USA

George Mason University, Fairfax, VA and Management Research Institute, Bethesda, MD USA

University of Nebraska at Omaha, Omaha, NE and Management Research Institute, Bethesda, MD USA

Direct all correspondence to: Michael D. Mumford, Department of Psychology, University of Oklahoma,
455 West Lindsey St., Room 740, Norman, OK 73019

Abstract: To develop organizational leaders we need to understand how requisite skills are acquired over
the course of people's careers. In this article, a cross-sectional design was used to assess differences in
leadership skills across six grade levels of officers in the U.S. Army. Increased levels of knowledge,
problem-solving skills, systems skills, and social skills were found at higher grade levels. Certain skills
and experiences, however, were found to be particularly important at certain phases of leaders' careers.
These findings are used to propose an organization-based model of skill development. Implications of this
model for leader development programs are discussed.

Introduction
Few of us would disagree with the proposition that, under certain conditions, leaders have an enormous
impact on organizations Day & Lord 1988 and Hogan, Curphy, & Hogan 1994. Many organizations have,
as a result, initiated programs intended to enhance leader performance. Assessment and selection
programs reflect one strategy frequently used to improve leader performance Bray et al. 1974 and Russell
& Kuhnert 1992. Another strategy commonly employed is based on a developmental approach. Training,
monitoring and career pathing programs are devised in the hope of producing leaders with the skills
needed on their jobs.
Discussions of leader development have a distinctly practical bent. A question often asked by researchers
and practitioners is How can we develop people to ensure effective leadership? When one considers
skills-based theories, a broader set of theoretical questions comes to mind: Is there reason to suspect that
skills develop as a function of experience? If skills improve as a function of experience, exactly what
kinds of experiences contribute to skill increases at different points in leaders' careers? Our intent in this
article is to examine the leadership skills proposed by Mumford, Zaccaro, Harding, Jacobs, and Fleishman
(2000) and how they differ across multiple organizational levels as a function of experience in
organizational leadership roles.

Skill Acquisition
Psychologists have an interest in the factors that contribute to the acquisition of skilled performance
(Ericsson & Charness, 1994). Broadly speaking, studies of skill acquisition have progressed along two
distinct lines. One line of research has focused on the structure of skill acquisition as people practice
certain tasks Ackerman 1987 and Fleishman 1972. The second line of research has focused on the
processes involved as people acquire knowledge and skills in different domains of experience (Anderson,
1993).
Traditionally, studies of skill acquisition have sought to understand how performance improves over time
as a function of practice. In initial studies along these lines, Fleishman and his colleagues Fleishman &
Hempel 1955 and Fleishman & Mumford 1989 found that although performance improves with practice,
the factors contributing to skill acquisition in the early stages of practice are not identical to those
influencing performance in the later stages of practice. Typically, broad common abilities, such as
intelligence, evidence their strongest effects during the early stages of skill acquisition while other more

narrow abilities influence performance in the later stages of skill acquisition. In a recent extension of this
work, Ackerman 1989 and Ackerman 1991 proposes a three-stage model of skill acquisition. He proposes
that skill acquisition proceeds first by people acquiring an understanding of task performance
requirements, to response assembly, where developing performance capabilities are integrated, to a final
stage where performance becomes automated.
These performance-based models of skill acquisition have a number of noteworthy implications for any
attempt to understand skill development, including the development of leadership skills. They indicate
that with experience, the factors that influence further development may not be identical to those that
influence development early on. Thus, useful experiences at one phase in leader's careers may not be
useful at other phases. Second, the kinds of errors made at one phase in a leader's career may be different
than the kinds of errors occurring later (Mumford, Costanza, Baughman, Threlfall, & Fleishman, 1994).
Third, characteristics associated with knowledge acquisition (e.g., intelligence and mastery motives)
appear particularly important early in the skill acquisition process. Characteristics associated with
performance application (e.g., task allocation and focus) are more strongly linked to later performance
Ackerman & Kanfer 1993 and Mumford, Baughman, Costanza, Uhlman, & Connelly 1993.
In contrast to this performance-based approach, studies of cognition have focused on how people acquire
expertise in different domains of education Chi, Glaser, & Rees 1982, DeGroot 1965, Goldsmith 1991,
Qin & Simon 1990 and Seigler & Richards 1982. Broadly speaking, these studies indicate that expertise
develops slowly over periods of ten years or more. Experts differ from novices in that they have a greater
number of concepts available, organize information on the basis of identifying principles, and are capable
of applying concepts in a flexible fashion contingent on key characteristics of the situation. More recent
research has extended these findings by looking for variables, primarily educational interventions, which
will accommodate the development of expertise. For example, Chi, Bassock, Lewis, Reiman, and Glaser
(1989) found that active self-initiated application of principles can accelerate the development of
expertise. Other studies by Sweller (1989) and Ward, Byrnes, and Overton (1990) have shown that
performance may be enhanced by providing models for organizing and forming concepts, and appropriate
strategies and procedures for applying these concepts.
Although these two lines of research have rather different goals, they paint a coherent picture of the skill
acquisition process. Initially, people must acquire base concepts, learn what is expected of them, and
apply these concepts in well-structured, relatively concrete situations. Next, these concepts must be
elaborated and applied in more complex settings as people begin independent problem-solving and learn
to apply different concepts in different settings. Finally, rapid integration of knowledge drawn from
multiple sources and practice allows people to address complex, rapidly unfolding problems.

Developing Leadership Skills

When one considers the general model of skill development in relation to the model of leadership skills
proposed by Mumford, Zaccaro, Harding, Jacobs, and Fleishman (2000), it has some implications for
understanding the development of leadership skills. Leaders, no matter how gifted, initially enter
organizations as novices. Thus, they lack basic concepts that provide them with an understanding of the

work, organizational contexts, and leadership roles. As a result, the kinds of problems with which they are
presented are typically highly structured, and the activities of these neophyte leaders are likely to be
closely supervised. Performance in these structured, well-defined roles are necessarily situationallycontingent and often rely on technical and social skills that transfer from prior educational or work-related
experiences (Podsakoff, MacKenzie, & Bommer, 1996). During this period, educational and socialization
issues are likely to be of paramount importance as novice leaders attempt to grasp and internalize the
norms, laws and guiding vision of the organization (Schneider & Schneider, 1994).
After novice leaders have acquired some understanding of the organization and their place in it, the key
issues at hand are two-fold. First, leaders must begin to elaborate their initial knowledge structures,
integrating real-world experience into base concepts. Second, they must begin to organize the
knowledge by independently tackling leadership problems, albeit relatively well-structured ones.
Assignments where the leader has primary supervisory responsibilities and some limited discretion are
likely to prove valuable in skill development. Commitment to organizational goals, the evaluation of
others, and commitment to others are all likely to prove particularly important developmentally, along
with an active ongoing involvement in developing others and oneself (Schmeck, 1988).
Reflection on initial learning effectiveness in independent supervisory roles provides a basis for the
elaboration of knowledge and the emergence of principle-based knowledge structures (Lewis & Jacobs,
1992). With the emergence of more principle-based knowledge structures, it becomes possible for leaders
to begin to develop and apply the kind of complex, creative thinking skills needed to solve the type of
novel, ill-defined problems that represent critical determinants of leader performance (Baughman &
Mumford, 1995). At this juncture, however, these skills may prove difficult to apply. Thus, assignments
that present novel challenging problems and require working with others who have different perspectives
may be valuable.
One implication of the Mumford, Zaccaro et al. (2000) model is that it is not enough for leaders to be able
to solve novel, ill-defined organizational problems. They must also be capable of formulating solutions
that will work in complex organizational environment, projecting downstream consequences, assessing
risks and workability, coordinating with multiple constituencies and formulating overarching models and
visions of long-term solutions Conger & Kanungo 1988 and Tusi 1984. Under these conditions, the
development of complex organizational wisdom and perspective taking will be at a premium. Exercises
that promote the acquisition of these skills and mentoring by experienced senior leaders should prove
useful as well as assignments that present novel, challenging organizational problems calling for
autonomy, risk taking, ongoing environmental assessment, and long-term solutions of multiple
subsystems.
The model of leader development described above is depicted in Fig. 1. Although this model may seem
quite straightforward, it has a number of important yet somewhat subtle implications. To begin, skill
development may take a substantial period of time. It may take up to 20 years before leaders acquire all of
the skills needed to solve novel, ill-defined organizational problems. Moreover, development in this sense
is progressive, moving from simple knowledge structures and straightforward technical and social skills,
to complex integrated knowledge structures that support the effective application of creative problemsolving and systems skills. Finally, it should be recognized that the kind of experiences which promote
skill development at one point in a leader's career are different than those which may be beneficial later.

To see this point, consider the predicament of a novice leader who must struggle to understand the
somewhat Machiavellian motivations of an experienced senior executive. It is questionable whether the
issues would be recognized, much less understood. Thus, effective developmental experiences will be
those that lie within the individual's zone of proximal development.

Figure 1. Conceptual Model of Leader Development

Developmental Interventions
When considering the type of model presented above, it is easy to fall into the trap of assuming that leader
development occurs automatically, always following the same pre-defined course. Skill development
depends on learning as people interact with their environments Lerner & Tubman 1989 and Mumford
1992. One crucial influence on development is what the individual is capable of taking from his or her
experience as a leader. This point is nicely articulated in the literature on management derailment factors
where characteristics, such as insensitivity, a lack of openness, and sheer ego, act to inhibit both learning
and the opportunity to learn through interactions with others Kaplan, Drath, & Kofodimos
1991 and Lombardo, Ruderman, & McCauley 1987. On a more positive note, other characteristics, such
as intellectual involvement and a developmental orientation may promote learning through interaction,
contributing to the development of requisite leadership skills Schooler 1990 and Simonton 1990.
Leaders' constructions and interpretations of environmental events are not the only influences on their
development. The environmental opportunities to which they are exposed also exert an influence on the
development of requisite leadership skills. For example, studies by Bray et al. (1974) and McCauley,
Ruderman, Ohlott, and Morrow (1994) indicate that exposure to assignments presenting novel,
challenging problems promotes the development of leadership skills presumably by stimulating the
exercise of creative problem-solving and systems skills. It is important to note that careers in different
organizations often do not provide the same opportunities for exposure to these types of assignments.
Moreover, other kinds of assignments, such as sales and marketing assignments, may promote the
development of other types of skills, including the interactional, communication and systems skills
needed by leaders.
Assignments are not the only kinds of experiences that might promote the development of requisite
leadership skills. Training can be viewed as an attempt to provide a set of systematic experiences to
promote the development of certain knowledge and skills (Anderson, 1993). In fact, many available
leadership training courses have received widespread praise. Unfortunately, evidence bearing on the
ability of these programs to develop requisite leadership skills is often less than compelling (Zaccaro,
1996). Nevertheless, a few studies (e.g., Bass & Avolio 1994 and Streufort, Nogami, Swezey, Pogash, &
Piasecki 1988) indicate that well-timed training interventions can promote the development of certain
problem-solving and systems skills.
These observations about influences on skill development bring us to the principal goals of the present
study. First, we hope to show that leadership knowledge and skills increase as a function of experience.
Second, assuming increases in skill levels across organizational levels are observed, we hoped to show
that the pattern and timing of skill level increases across organizational level are consistent with the
model of leader development presented above. Third, and finally, we hoped to show that certain
assignment characteristics, training courses, and career development orientations are correlated with skill
levels at certain points in leaders' careers in accordance with their current phase of skill development.

Method
Sample
The sample used to test these ideas was obtained as part of a larger study of leadership among Army
officers. A more detailed description of the sample may be obtained by consulting Zaccaro, Mumford,
Connelly, Marks, and Gilbert (2000). This sample is unique in the sense that it contains leaders at
different points in their military careers ranging from second lieutenants to colonels. Officers' ages range
from 21 to 58. Moreover, members of this cross-sectional sample spent their careers in one organization,
the Army, making it possible to assess developmental influences in a common organizational framework.
Based on the significance of company command and battalion command in the Army's career progression
system, the sample was divided into three groups. The first group consisted of 1,160 second lieutenants,
first lieutenants, and junior captains. Officers in this group had not yet been exposed to company
command in the Army (i.e., a major leadership role). The average age of these officers was 27.47 (SD =
3.71), and they had 2.85 (SD = 2.68) years experience in Army leadership roles. The second group
contained senior captains and majors who had company command experience but had not yet commanded
a battalion, the first Army leadership role where officers are responsible for multiple component
subsystems. The 410 officers in this group were, on average, 34.37 (SD = 3.26) years old and had been in
the Army for 10.32 (SD = 2.60) years. The third group was composed of lieutenant colonels and colonels,
all of whom had commanded battalions. The 220 lieutenant colonels and colonels in this third group were,
on average, 44.52 (SD = 2.86) years old, having been in the Army for 20.98 (SD = 2.71) years.
Skill and Criterion Measures
The officers included in this sample were asked to complete a number of measures ranging from
standardized tests intended to assess basic abilities and dispositional characteristics, to measures assessing
some of the key leadership skills identified by Mumford et al. (2000). For the purpose of the present
investigation, we will focus only on a limited subset of measures (i.e., those intended to capture
leadership skills as opposed to measures of abilities, personality and motivation).
Table 1 provides a brief description of each skill measure. Detailed descriptions of these measures may be
found in Zaccaro, Mumford, Connelly, Marks, and Gilbert (2000). Each measure that was presented to
respondents is comprised of one or more tasks expressly selected to elicit certain skills. Responses to
these written paper-and-pencil scenarios were rated by four judges, all doctoral candidates, with respect to
the extent to which certain attributes of a skill or expertise were manifest in performance. The resulting
interrater agreement coefficients ranged from .56 to .91, averaging .75. Evidence for the criterion-related
validity of these scales was provided by Connelly, Gilbert, Zaccaro, Threlfall, Marks, and Mumford
(2000).
In addition to the skill measures, two leadership criterion measures described by Zaccaro et al. (2000) are
used in the present effort. The first measure was a self-report, career achievement record where officers
were asked to report objective, verifiable accomplishments (e.g., medals won, awards received, etc.).
Evidence of the reliability and the validity of this kind of objective career achievement record has been

provided by Kilcullen (1993). Quality of problem solutions was used as the second criterion measure,
where judges rated the overall quality and originality of solutions to novel, ill-defined military problems.
A third leadership criterion not described in Zaccaro et al. (this issue) used a variation of Hough's (1984)
behavioral consistency technique. Officers were asked to think about their performance during the last
year and provide written best performance examples for four general dimensions of leadership behavior
identified by Fleishman, Mumford, Zaccaro, Levin, Korotkin, and Hein (1991). Four doctoral candidate
judges evaluated the quality of leadership manifest in these incidents on a five-point scale. The mean
interrater agreement across these dimensions was .94. Average ratings across all four dimensions
provided the overall score.
Career Development Measures
In addition to these skill and leadership criterion measures, officers were asked to complete two
inventories examining different kinds of developmental influences. The first inventory was intended to
assess career development experiences that might influence skill acquisition. This inventory contained 52
background data items generated based on (1) interviews with senior Army officers concerning
significant developmental experiences and (2) a review of the literature examining the kind of experiences
contributing to the development of high-level talent Barron & Harrington 1981, Simonton
1990 and Walberg & Stariha 1992.
A subsequent principal components analysis of these items indicated that a nine-factor solution,
accounting for 30.7% of the total item variance, provided the most appropriate summarization of these
items. The factors identified in this analysis were labeled (1) Development (e.g., training at home,
counseling subordinates, learning new things); (2) Basic Technical Training (e.g., completed basic
advance course, specialist course, branch qualification); (3) Advanced Professional Training (e.g.,
completed staff college, college, senior service college); (4) Networking (e.g., discusses work problems
with peers, has close friends at work); (5) Enculturation (e.g., positions or views similar to those of
peers); (6) Traditional Army Background (e.g., attended officer candidates school); (7) Career Intellectual
Involvement (e.g., read military history issues, public policy issues); (8) Special Tasking (e.g., had special
assignments in the last five years); and (9) Senior Officer Mentoring (e.g., had exposure to senior officers,
participated in discussions with senior officers).
The second career development measure was intended to capture the aspects of job assignments which
might influence the acquisition of leadership skills. Here, officers were asked to provide a written threeparagraph description of the assignments that proved most beneficial over the last five years of their
Army careers. In describing these assignments, they were asked to describe (1) their major duties on this
assignment, (2) the most significant aspects of these duties, (3) the kind of problems they worked on, and
(4) how they went about solving these problems. A sample of 50 descriptions was then reviewed by three
psychologists familiar with data from the military and the career development literature to formulate a
content coding scheme based on key assignment features. Table 2 lists the 29 content coding categories
that emerged from this analysis.
Four judges reviewed the assignments descriptions provided by officers. After reading through the
material, judges were asked to decide whether a particular type of assignment characteristic was

mentioned or not. Dichotomous ratings were to be made only if the judges felt that the descriptive
material provided sufficient information to assess the presence or absence of certain assignment
characteristics. The average interrater agreement obtained for these ratings was .71.
Control Analyses
Prior to conducting the central analyses intended to examine address the goals of this study, a series of
control analyses were conducted. The first of these analyses was intended to examine whether cohort
effects, or changes in the types of officers entering the Army, might account for any differences observed
in skills across junior, mid-level, and senior officer groups Baltes & Schaie 1976 and Cook & Campbell
1979. Here, the ability, personality, and motivational measures were used to construct a set of subgroups
describing the typical profiles of officer characteristics observed at the time of entry into the organization.
Subsequently, an attempt was made to slot senior officers into these junior officers subgroups using a Kmeans analysis. It was found that 88% of the senior officers could be unanimously assigned to a junior
officer subgroup (p > 80see Mumford, Zaccaro, Johnson, Diana, Gilbert, & Threlfall, 2000). Although
this finding does not rule out cohort-by-time interactions, it does suggest that simple cohort effects
attributable to changes in recruitment policies could not easily account for any developmental efforts
obtained in the present study.
The second set of analyses was intended to assess whether spurious effects might arise from changes in
the structure of the skill measures (Hertzog, 1989). Here, the variance covariance matrices obtained in the
junior, mid-level, and senior officer groups were compared using a LISREL VI analysis of correlation
structures. In this analysis, it was found that the structure of relationships among the component scales
obtained for each skill measure was constant across groups. As a result, the correlation patterns and
meaning of the skill measures appear to be quite consistent across groups. Thus, changes in the structure
of the skill measures across groups does not appear to be a plausible alternative explanation for observed
developmental effects.
A third alternative explanation for observed developmental effects involves the Army `up or out' policy.
In other words, development might be attributed to the selective survival of certain individuals. Of
particular concern in this program is whether this policy results in a situation where the selective retention
of more intelligent individuals produces higher scores on the various skill measures. To provide some
preliminary evidence for this, junior, mid-level, and senior officers were compared on a standard measure
of general intelligence (verbal reasoning) using analysis of variance (ANOVA) contrasts. Junior officers
(M = 23.75, SD = 6.25) differed significantly from mid-level officers (M = 24.78, SD = 5.00; t (2,1462) =
2.76, p < .01). However, mid-level officers did not differ significantly from senior officers (M = 25.83,
SD = 5.08). Given that the difference between junior and mid-level officers reflects roughly one-fifth of a
standard deviation, it is questionable whether there is any practical difference in intelligence. From this
analysis, it appears that the selective survival of more intelligent officers does not provide an
overwhelming threat to the validity of our conclusions. This finding is consistent with the results obtained
in the subgrouping analysis described above.

Developmental Analyses
Once these control analyses had been completed, analyses to assess developmental changes in the various
leadership skills were conducted. Initially, means and standard deviations of each component scale in the
five measures of knowledge and skill were obtained for the three officer groups. Mean differences were
tested using a series of ANOVAs, contrasting less experienced with more experienced groups. Variance
differences were tested using Levene's procedure.
In the next set of analyses, an attempt was made to assess differences in skill levels at different points in
leaders' careers. To assess differences in skills across organizational levels, a series of discriminant
analyses were completed where the scales included in each skill measure were used to account for
differences between less experienced and more experienced officers. These analyses contrasted highperforming junior officers, as defined by a within-group median split on the career achievement scale,
with all mid-level officers. Likewise, high-performing mid-level officers were contrasted with all senior
officers. Use of high performers in each analysis controlled for the promotional criteria reflected in the
career achievement measure. This was intended to minimize the effects of the up-or-out policy on skill
and knowledge scores.
The discriminant functions obtained in this analysis indicate the skills associated with development at
certain points in leaders' careers. In the next set of analyses, the discriminant function scores were
correlated with (1) scores on the critical incidents and solution quality criteria, (2) scores on the career
development factors, and (3) scores on the job assignments measure. These analyses were intended to
provide some initial evidence indicating whether differences in skill development across organizational
levels were related to career development events, job assignments and training.

Results
Descriptive Statistics
Table 3 presents the means, standard deviations, and ANOVA contrasts for junior, mid-level, and senior
officers on each scale included in the criterion, knowledge and skill measures. As might be expected,
given the fact that manifest leader achievement depends on time and opportunity, scores on the leader
achievement measure increase significantly with grade level across groups of junior (M = 6.37, SD =
3.80), mid-level (M = 11.75, SD = 2.44), and senior (M = 13.95, SD = 2.66) officers. Critical incidents
scores for senior officers (M = 3.20; SD = .44) were significantly higher than mid-level (M = 3.00, SD =
.47) and junior officers; however, junior and mid-level officers did not differ on this measure. Increases in
solution quality were obtained on the unstructured military problems with more senior officers producing
higher quality solutions (M = 3.38, SD = .53) than mid-level officers (M = 2.91, SD = .54) who, in turn,
produce higher quality solutions than junior officers (M = 2.50, SD = .56).
One potential explanation for these increases in the leadership criteria is the acquisition of expertise. In
examining the findings obtained for the leader knowledge measure it is clear that complexity of
knowledge increases from junior officers to mid-level officers. This finding is consistent with the earlier
observations of Streufort and Streufort (1978). Experienced officers evidence the coherent, organized,

principle-based knowledge structures characteristic of experts, and formulated structures consistent with a
theoretically-based model of leader behavior. For example, mid-level officers are significantly more
likely to use principle-based organizations of leadership tasks than junior officers, while senior officers
are significantly more likely to use principle-based organizations than mid-level officers. It is not just
complexity, but rather frameworks for organizing complexity that characterize more senior leaders. These
changes in means are not necessarily associated with changes in variance. However, in contrasting junior
and mid-level officers, a significant difference in variance (p < .05) is observed on our measures of
organization, coherence, and theoretical similarity.
Not only are there increases in expertise, there are increases in the skills needed to work with this
knowledge in solving novel, ill-defined leadership problems. On the measure examining the application
of complex problem-solving skills (cued), mid-level officers perform significantly better than junior
officers on scales examining aspects of solution production, including problem construction, encoding,
category search, category fit, category combination, and idea evaluation. Senior officers perform
significantly better than mid-level officers, not only on the production scales, but also on the
implementation, planning, and monitoring measures. This finding suggests that officers with more
experience are better at translating new ideas into action, an observation consistent with the increased (p <
.01) variability observed among senior officers on these scales.
The social judgment scenarios, solution construction scenarios, and creative thinking measures all
examined various skills needed to implement solutions to leadership problems in organizational settings.
More personally-oriented scales in the military scenarios and consequences measures, such as a focus on
personal goals, positive consequences, and negative consequences, did not distinguish among junior, midlevel, and senior officers. However, more objective types of scales related to implementing solutions in
complex, organizational systems showed increases from junior to mid-level and senior positions. The
solution construction scenarios measure expressions of these systems skills in the domain of Army
leadership. Here, attention to restrictions, solution time frames, and convergence of solutions with
organizational goals display significant increases when contrasting senior officers with mid-level and
junior officers. The creative thinking scales examine similar attributes outside of a military context. Here
again, senior leaders evidence a significantly longer time span than do mid-level officers when appraising
the implications of novel situations. Mid-level officers evidence significantly longer time frame than do
junior officers. More experienced officers also identify consequences of novel events with significantly
greater realism, complexity, and abstraction. Thus, the increases in systems skills apparently evidence
some generality.
To implement solutions in a complex system, leaders must also be able to go outside organizational
structures to generate social judgment and perspective taking. The social judgment scenarios attempt to
assess skills relevant to social judgment and perspective taking by using a set of general business
problems intended to elicit these attributes. Across the self-reflection, objectivity, judgment, systems
perception, systems commitment, and solution fit scales, senior officers score significantly higher than
mid-level officers, who, in turn, score significantly higher than junior officers. The objectivity, systems
perception, systems commitment, and solution fit scales also evidenced a slight, but significant decrease
in variance when comparing junior and mid-level officers. Although the reason for these effects is
somewhat obscure, they may mirror the normalization effects observed on the knowledge measure, given
Smith and Baltes (1990) observations of the positive relationships between related types of social

judgment variables and expertise. These findings, taken as a whole, provide some support for our
argument that knowledge, problem solving skills, and social judgment skills increase with experience.
Discriminant Analyses
The preceding findings do not provide a great deal of information about how knowledge and skills change
from lower- to upper-level grades. Initial evidence along these lines is provided in the discriminant
analyses presented in Table 4.
Table 4 presents the chi-square values and canonical correlations obtained in these analyses, along with
the loadings of the skill scales on the relevant functions. As may be seen, scales examining problemsolving processes failed to yield significant effects when used to discriminate high-performing junior
officers from mid-level officers. Apparently, the significant means obtained in contrasting officer groups
reflect sample characteristics rather than skill development. However, in contrasting high-performing
mid-level officers to senior officers, a significant (2 = 34.2, p < .001) function was obtained yielding a
canonical correlation of .62. Inspection of the standardized group function means indicated some increase
in problem-solving skills in comparing mid-level (M = .69) and senior (M = .91) leaders. The scale
loadings indicate that these increases are most strongly related to having better complex problem-solving
skills (Mumford, Baughman, Supinski, Costanza, & Threlfall, 1993).
This finding is hardly surprising when it is recognized that effective application of complex problemsolving skills depends on prior acquisition of requisite expertise. Elements of expertise develop early in
leaders' careers as indicated by the significant discriminant function (2(5) = 52.3, p < .001) obtained in
contrasting high-performing junior officers (M = .37) with mid-level officers (M = .40). The principles
(r = .65) and complexity (r = .57) scales have the highest loadings of the leader expertise which yield a
canonical correlation of .36. Although the acquisition of principles is crucial at lower and mid-levels, true
expertise also requires organized principle-based structures, which are seen at the senior levels.
Accordingly, the significant function (2(5) = 44.1, p < .001) in contrasting high-performing mid-level
officers (M = .44) with senior officers (M = .52), coherence (r = .77), organization (r = .71), and
principles (r = .60) shows increases across groups. This function, yielding a canonical correlation of .45,
suggests that increases in the application of creative thinking skills to leadership problems occurs along
with the acquisition of relevant expertise.
Expertise and complex problem-solving skills alone are not sufficient for effective organizational
leadership (Mumford et al., this issue). Leaders must also be able to revise and implement solutions,
while taking into account the demands of the organization. The solution construction scenarios provide a
domain-specific measure of these skills while the creative thinking exercise provides a more general
measure of these skills. In contrasting high-performing junior officers with mid-level officers, both the
solution construction (2 (4) = 45, p < .001) and creative thinking (2 (6) = 214.1, p < .001) scales
significantly discriminate these groups, yielding canonical correlations of .31 and .45, respectively. The
resulting function means indicate increases in solution construction skills (M = .28 versus M = .37) and
creative thinking skills (M = .44 versus M = .58). All solution construction scales discriminated the
officer groups, except self-oriented goals. Creative thinking scales that discriminated officer groups
include attention to organizational goals, attention to restrictions, time span, realism, complexity, and
abstraction increase by organizational level.

In contrasting high-performing mid-level officers and senior officers on these scales, a similar pattern of
findings emerges. Again, canonical correlations of .35 and .47 are significant for both the solution
construction skills (2 (4) = 26.4, p < .001) and the creative thinking skills (2 (6) = 104.5, p < .001). Here
the standardized function means again indicate increases in skills on the solution construction measure (M
= .33 versus M = .42) and creative thinking measure (M = .47 versus M = .59) with attention to
organizational goals, attention to restrictions, time span, realism, complexity, and abstraction all being
associated with increases across organizational level.
The scenarios examining social judgment also indicate changes across organizational grade for all the
levels under consideration. The canonical correlation of .45 is significant (2 (6) = 106.8, p < .001) in
contrasting high-performing junior officers (M = .44) with mid-level officers (M = .57), as well as the
canonical correlation of .49 (2 (6) = 57.1 p < .001) in contrasting high-performing mid-level officers (M
= .50) with senior officers (M = .42). The most striking findings emerging in this analysis, however, are
the loadings of the scales that demonstrate increases in organizational level. In the junior to mid-level
contrast, systems perception (r = .75), and systems commitment (r = .65) display the largest loadings.
However, in the mid-level to senior contrast, solution fit (r = .73), judgment (r = .80), reflection (r = .72),
objectivity (r = .63), systems perception (r = .64), and systems commitment (r = .63) also have sizable
loadings. This pattern of findings suggests, in accordance with our model of leader development, that
awareness of and commitment to social systems is present in lower level officers, while additional
characteristics such as a more balanced, mature approach to organizational problems are present in more
mature and experienced officers.
Career Development
The discriminant functions indicate increases in leadership skills even when factors contributing to
promotion are taken into account. The functions reflect skill increases across early, middle, and later
portions of leaders' careers. As a result, the discriminant function scores derived from these analyses
provide a particularly appropriate index for examining the influences on, and consequences of, skill
increases. Table 5 presents the correlations of these discriminant function scores with the critical incident
and solution quality criteria. This table also presents the correlations of the assignment characteristics
ratings with the discriminant function scores.
One would expect increases in leadership skills across organizational levels to be related to indices of
leader effectiveness such as those reflected in the critical incidents. As may be seen, critical incident
performance is significantly and positively related to scores on the discriminant functions as obtained by
contrasting high-performing junior and mid-level officers (avg. r = .19). A similar pattern of relationships
emerges for the discriminant functions contrasting high-performing mid-level offices with senior officers.
Here, all of the discriminant functions yield significant (p < .001) correlations with incident performance
(avg. r = .19), roughly equivalent to those obtained in the earlier period.
The design used in our initial data collection prohibits examining the relationship between complex
problem-solving skill increases and solution quality. Solution quality correlations could, however, be
obtained with the function scores for the other constructed response measures. Solution quality is
significantly related (p < .001) to increases in social judgment (avg. r = .35); systems skills, as measured
by the creative thinking measure (avg. r = .48); and leader expertise (avg. r = .18). Thus, is appears that

skill increases across organizational level are indeed related to both improved performance and the
capacity to solve the kind of novel, ill-defined organizational problems that are critical determinants of
effective leader performance.
If skill increases are related to problem solving and leader performance, a new question arises: What kind
of experiences are likely to contribute to the development of leadership skills at different points in leaders'
careers? In our earlier discussion of skill acquisition, we argued that certain kinds of job assignments
might influence skill increases. Thus, some initial evidence along these lines is provided by the
correlations between ratings of assignment characteristics and the discriminant functions. These
correlations are also presented in Table 5.
Correlations between assignment characteristics and discriminant functions indicating skill increases
across grade were modest. This finding, however, is not especially surprising given that dichotomous
assignment characteristics ratings represent single-item measures and that no single-item measure
provides a fully adequate description of a beneficial assignment. Even considering this caveat, a number
of assignment characteristics showed significant correlations with one or more discriminant functions. In
keeping with the notion that teaching others is a good way to learn, assignments requiring leaders to
develop others correlate with increases in problem-solving (avg. r = .23) and solution characteristics (avg.
r = .20) across both the junior to mid-level and mid-level to senior periods. Along similar lines, it was not
surprising that assignments calling for communication correlate with increases in social judgment (avg. r
= .15) and solution characteristics skills (avg. r = .10).
Although these findings are of some interest, the most consistent effects across both organizational level
shifts show positive correlations between exposure to complex, ill-defined organizational problems and
increases in problem-solving and solution characteristics, and creative thinking skills across
organizational levels. Higher function scores are associated with exposure to valued assignments such as
(1) problems with multiple components, (2) long-term planning, (3) novel, ill-defined problems, (4)
diverse experiences, (5) autonomy, and (6) boundary spanning.
Assignments are not the only type of developmental experience that might influence the acquisition of
requisite leadership skills. Other potential developmental influences are examined in the career
development inventory analyses. Table 6 presents correlations among the nine career development factors
derived from the inventory and scores on the discriminant function scores obtained in contrasting highperforming junior officers with mid-level officers, and high-performing mid-level officers with more
senior officers. Correlations for the first three factors tended to be significant, while correlations with the
remaining factors tended to be non-significant. The first factor, developmental orientation, was also
significantly related (p < .05) to increases across both organizational levels in complex problem-solving
skills and solution characteristics skills. In moving from junior to mid-level positions, developmental
orientation was also significantly related to increases in social judgment.
These relationships, however, are marginal when compared to the relationships obtained for the two
training factors. The basic technical training factor shows significant correlations with skill increases from
junior to mid-level positions for all of our measures of skills and expertise (avg. r = .22). Although basic
technical training yields somewhat weaker relationships in accounting from mid-level to senior positions
(avg. r = .11), this finding might be attributed to the diminished importance of technical skills in more

complex leadership skills. Advanced professional training was significant (p < .001), with increases in
knowledge and skills as leaders moved from mid-level to more senior positions (r = .28) showing
somewhat weaker (r = .16) but significant effects (p < .05) on skill increases in moving from junior to
mid-level positions. One implication that can be drawn from this is that well-designed training may
contribute to the development of leadership skills.
Discussion
Before proceeding to the broader implications of our findings, certain limitations should be noted. To
begin, our findings were obtained in a sample of leaders drawn from one organization, the Army. Thus,
some caution is called for in generalizing these findings to other settings. Clearly, training was related to
increases in requisite leadership skills across officer groups at different phases of their careers. It should
be recognized, however, that Army training programs are usually well-designed, involving intense course
work, sometimes over a period as long as nine months. Thus, it is open to question whether these findings
can be used as an omnibus justification for the value of leadership development programs, although they
may well say something about the value of the systematic progressive approach employed by the Army.
To generalize these findings, the particular sampling design in use should also be considered. Ideally,
attempts to examine development should be based on a cohort sequential design. Any study employing a
purely longitudinal approach or a purely cross-sectional approach (such as the one employed here) suffers
from cohort problems (Baltes & Schaie, 1976). For example, the generality of findings emerging from
longitudinal design are open to question unless multiple cohorts have been studied. The generality and
validity of findings obtained in cross-sectional studies can be questioned based on cohort effects, selective
survival, or changes in available developmental opportunities. Although these effects cannot be entirely
ruled out here, the results of the control analyses presented earlier suggest that cohort effects and selective
survival are not unduly influencing the results across the groups compared in this study.
Even bearing these caveats in mind, we believe that the present study has some important implications for
studies of leadership development. The theory of effective organizational leadership proposed by
Mumford et al. (2000) holds that leader performance ultimately depends on the person's ability to solve
novel, ill-defined organizational problems. Quality of responses or solutions to these problems, moreover,
may depend on whether the individual possesses the requisite knowledge and a more complex set of
skills. One key proposition of this kind of skills-based model of leadership is that skills should provide
strong support for this general proposition. In our initial analyses, higher scores on measures of creative
problem-solving skills, social judgment, systems skills, and leadership expertise were observed as leaders
moved into more responsible senior positions. Of greater importance, however, were the effects observed
in the discriminant analyses. Results obtained in these analyses (with one exception: problem-solving
skills from junior to mid-level positions) indicated increases in levels of requisite leadership skills from
junior to mid-level to senior positions. Moreover, the solution construction and creative thinking
measures indicated that these effects with regard to systems skills appeared to evidence some generality.
Taken as a whole, these findings provide some initial support for the theory proposed by Mumford,
Zaccaro et al. (2000) in three areas. First, the measures in use evidenced increases in expertise and skills
across grade levels. Second, different aspects of expertise and different skills appear to be relevant for
lower and upper level leaders. Third, more senior leadership positions apparently require higher levels of

skills in general. In addition, one would expect that as leaders acquire capability, they will exhibit better
performance. In fact, increases in leader expertise, problem-solving skills, social judgment, and systems
skills, appear to be accompanied by improved critical incident performance and higher quality solutions
to ill-defined military leadership problems.
Not only does the skills-based model of leader development imply development in skills, it implies that
this development occurs in a progressive, systematic fashion. More specifically, leaders must initially
acquire basic concepts and principles. These concepts and principles make it possible for leaders to apply
and begin to develop creative problem-solving skills which are subsequently integrated with the practical
demands of implementing problem solutions within a complex organizational system. In fact, the results
obtained in the present investigation provide some support for this model of leader development.
Mid-level leaders appeared to have more complex principle-based knowledge structures than junior level
leaders. Because creative problem solving is contingent on the availability of such structures (Baughman
& Mumford, 1995), only small increases were seen in the application of these skills. As the more
coherent, organized principle-based structures characteristic of true expertise emerged Anderson
1993 and Ericsson & Charness 1994, creative problem-solving skills emerged, accompanied by a more
pragmatic evaluation of solutions taking into account a longer time span. Thus, in broad terms, the
patterns of skill increases observed in this study seem to conform to a skill or capacity-based model of
leader development.
Although the general patterns of our findings imply support for a systematic, progressive model of leader
development, one proviso should be noted. Certain systems skills, for example, realism and attention to
restrictions, increased from junior to mid-level and from mid-level to senior positions. One potential
explanation for this finding is that the increased levels of skills is contingent on experience and
knowledge of the organization as opposed to more formal, expertise-based knowledge structures
Mumford & Gustafson in press and Sternberg, Wagner, & Williams 1995. Alternatively, organizational
knowledge may interact with more formal knowledge structured in the acquisition and application of
systems skills. In fact, the results obtained for our measure of systems skills provide some indirect
support for this latter explanation.
This structured model of the development of leadership skills is of interest for practical as well as
theoretical reasons. By understanding the structure of leader skill acquisition, it becomes possible to draw
some general conclusions about the kinds of interventions likely to contribute to leader development. One
noteworthy direct application of this model of leader development is that assignments providing
experience in solving complex organizational problems should contribute to skill development and
performance, particularly in terms of requisite problem-solving and systems skills. The findings obtained
in our analysis of assignment characteristics support this proposition, confirming the earlier findings of
Bray et al. (1974) and McCauley et al. (1994).
The effect of these experiences, however, appears to depend on where the individual is within the
developmental model outlined above. This point was illustrated in our finding that exposure to complex,
organizational problems was not related to increases in social judgment in moving from junior to midlevel positions, although it was related to increases in social judgment in moving from mid-level to more
senior positions, where leaders had the experience and maturity to interpret the implications of those

experiences. Similarly, in accordance with our model of leader development, it was found that many
experiences, such as those involved in administration and systems management, were related to skill
increases across mid-level and senior positions. These findings, suggest that the outcomes of assignments
and the features of assignments contributing to skill increases depend on the nature of the skills that are
relevant at that point in leaders' careers. These assignments must be carefully designed to maximize their
impact on leader development.
This point also is illustrated in our findings with respect to the impact of training on the increases in
requisite leadership skills. Basic technical training was more strongly related to skill increases in moving
from junior to mid-level positions while advanced professional training, which in the Army focuses on
complex problem solving and the acquisition of more advanced principles (Zaccaro, 1996), was more
strongly related to increases in requisite skills as leaders moved from mid-level to more senior positions.
Thus, it appears that training interventions, like assignments, must be carefully tailored to current
developmental needs if they are to have optimal effects on the acquisition of requisite skills and expertise.

Conclusion
The above observations about influences on skill acquisition bring us to a final comment. Some studies of
leadership assume that interventions which promote skill increases at the point in leaders' careers can be
readily generalized to all leaders, regardless of role. For example, it is not fashionable to assume that
complex organizational structures will benefit all young leaders. The model of leadership simulating
proposed action, and the findings obtained in the present study, tend to argue against this proposition,
suggesting that simpler, structured exercises illustrating key principles will prove most beneficial.
Hopefully, future studies will extend this model, showing how particular types of interventions interact
with available skills. We believe that studies of this sort will serve not only to enhance our understanding
of leader development, but will ultimately result in far more powerful systems for developing those skills
leaders need to solve organizational problems in an ever more dynamic world.Mumford Zaccaro Johnson
Diana Threlfall 2000

Acknowledgements
The research described in this article was supported by a contract to Management Research Institute from
the Army Research Institute for the Behavioral and Social Sciences under the Small Business Innovation
Research program and was also carried out at George Mason University. During this study the first author
was at George Mason University. Parts of this research stem from earlier work completed at the Georgia
Institute of Technology and Advanced Research Resources Organization. The material presented in this
article represents the opinions of the authors and does not necessarily represent the views of the Army or
the Department of Defense.

References
Ackerman, P. L. (1987). Individual differences in skill learning: An integration of psychometric and
information processing perspectives. Psychological Bulletin, 102, 5-27.
Ackerman, P. L. (1989). Within-task intercorrelations of skilled performance: Implications for predicting
individual differences? Journal of Applied Psychology, 74, 360-364.
Ackerman, P. L. (1991). A correlational analysis of skill specificity: Learning, abilities, and individual
differences. Journal of Experimental Psychology: Learning, Memory, and Cognition, 16, 883901.
Ackerman, P. L., & Kanfer, R. (1993). Integrating laboratory and field study for improving selection:
Development of a battery for predicting air traffic controller success. Journal of Applied
Psychology, 78, 413-432.
Anderson, J. R. {1993). Problem solving and learning. American Psychologist, 48(1), 35-44.
Baltes, P. B., & Schaie, K. W. (1976). On sequential strategies in developmental research: Description or
explanation. Human Development, 18, 384-390.
Barron, F., & Harrington, D. M. (1981). Creativity, intelligence, and personality. In M. R. Rosenzweig &
L. W. Porter (Eds.), Annual review of psychology (pp. 439-476). Palo Alto, CA: Annual Review.
Bass, B. M., & Avolio, B. J. (1994). Transformational leadership and organizational culture. International
Journal of Public Administration, 17, 541-554.
Baughman, W. A., & Mumford, M.D. (1995). Process analytic models of creative capacities: Operations
influencing the combination and reorganization process. Creativity Research Journal, 8(1), 3762.
Bray, D. W., Campbell, R. S .. & Grant, D. L. (1974). Formative years in business. New York: Wiley.
Chi, M. T. H., Bassock, M., Lewis, M. W., Reiman, P., & Glaser, R. (1989). Self explanations: How
students study and use examples in learning to solve problems. Cognitive Science, 13, 145-182.
Chi, M. T., Glaser, R., & Rees, E. (1982). Expertise in problem solving. In R. J. Sternberg (Ed.),
Advances in the psychology of human intelligence (pp. 7-75). Hillsdale, NJ: Lawrence Erlbaum.
Conger, J. A., & Kanungo, R.N. (1988). Charismatic leadership: The elusive factor in organizational
effectiveness. San Francisco: Jossey-Bass.
Connelly, M. S., Gilbert, J. A., Zaccaro, S. J., Threlfall, K.V., Marks, M. A., & Mumford, M. D. (2000).
Exploring the relationship of leader skills and knowledge to leader performance. Leadership
Quarterly, 11(1), 65-86.
Cook, T. D., & Campbell, D. T. (1979). Quasi-experimentation: Design and analysis issues for field
settings. Boston: Houghton Mifflin.
Day. D. V., & Lord. R. G. (1988). Executive leadership and organizational performance: Suggestions for
a new theory and methodology. Journal of Management, 14, 453-464.
DeGroot, A D. (1965). Perception and memory versus thought. In B. Kleinmuntz (Ed.), Problem-solving.
New York: Wiley.
Ericsson, K. A, & Charness, N. (1994). Expert performance: Its structure and acquisition. American
Psychologist, 49, 725-747.
Fleishman, E. A (1972). On the relation between abilities, learning and human performance. American
Psychologist, 27, 1017-1032.
Fleishman, E. A., & Hempel, W. E .. Jr. (1955). The relation between abilities and improvement with
practice in a visual discrimination reaction task. Journal of Experimental Psychology, 49. 301312.
Fleishman, E. A., & Mumford, M. D. (1989). Abilities as causes of individual differences in skill
acquisition. Human Performance, 2, 201-222.
Fleishman, E. A, Mumford, M. D .. Zaccaro, S. J., Levin, K. Y., Korotkin, A L., & Hein, M. B. (1991 ).
Taxonomic efforts in the description of leader behavior: A synthesis and functional interpretation.
Leadership Quarterly, 2, 245-287.
Goldsmith, G. (1991). The dialectics of sketching. Creativity Research Journal, 4, 123-144.

Hertzog, C. (1989). Influences of cognitive slowing on age differences in intelligence. Developmental

Psychology, 25, 636--651.
Hogan, R .. Curphy, G. J., & Hogan, J. (1994). What we know about leadership effectiveness and
personality. American Psychologist, 49, 493-504.
Hough, L. M. (1984). Development and evaluation of the "Accomplishment Record" method of selecting
and promoting professionals. Journal of Applied Psychology, 69(1), 135-146.
Kaplan, R. E., Drath, W. H., & Kofodimos, J. R. (1991). Beyond ambition: How driven managers can
lead better and live better. San Francisco: Jossey-Bass.
Kilcullen. R. N. (1993). The development and use of rationally-keyed background data scales to predict
leader effectiveness. Unpublished doctoral dissertation, George Mason University, Fairfax, VA
Lerner, R. M., & Tubman, J. G. (1989). Conceptual issues in studying continuity and discontinuity in
personality development across life. Journal of Personality, 57, 343-374.
Lewis, P .. & Jacobs, J. 0. (1992). Individual differences in strategic leadership capacity: A constructive
developmental view. In R. L. Phillips & J. G. Hunt (Eds.), Strategic leadership: A
multiorganizationallevel perspective (pp. 121-137). Westport, CT: Quorum Books.
Lombardo, M. M., Ruderman, M. N., & McCauley, C. D. (1987). Explanations of success and derailment
in upper-level management positions. Paper, Academy of Management, New York.
McCauley. C. C., Ruderman, M. N., Ohlott, P. J., & Morrow, J. F. (1994). Assessing the developmental
components of managerial jobs. Journal of Applied Psychology, 37, 46-67.
Mumford, M.D. (1992). Individuality in adult development. [n R. Kastenbaum (Ed.), The encyclopedia of
adult development (pp. 229-236). Phoenix, AZ: Onyx Press.
Mumford, M. D .. Baughman, W. A., Costanza, D. P., Uhlman, C. E., & Connelly, M. S. (1993).
Developing creative capacities: Implications of cognitive processing models.
Mumford, M. D., Costanza, D. P., Baughman, W. A., Threlfall, K. V., & Fleishman, E. A. (1994). The
influence of abilities on performance during practice: Effects of massed and distributed practice.
Journal of Educational Psychology, 86, 1-11.
Mumford, M. D., & Gustafson, S. B. (in press). Creative thought: Cognition and problem solving in
dynamic systems. In M. Runco (Ed.), Creativity research handbook. Cresskill, NY: Hampton.
Mumford. M. D., Zaccaro, S. J., Harding, F. D., Jacobs, T. 0., & Fleishman, E. A. (2000). Leadership
skills for a changing world: Solving complex social problems. Leadership Quarterly, 11(1), 1135.
Mumford, M.D., Zaccaro, S. J., Johnson, J. F., Diana, M., & Threlfall, K. V. (2000). Patterns of leader
characteristics: Implications for performance and development. Leadership Quarterly, 11(1), 115133.
Podsakoff, P. M., MacKenzie, S. B., & Bommer, W. H. (1996). Transformational leader behaviors and
substitutes for leadership as determinants of employee satisfaction, commitment, trust, and
organizational citizenship behaviors. Journal of Management, 22, 259-298.
Qin, Y., & Simon, H. A. (1990). Laboratory replication of scientific discovery processes. Cognitive
Science, 14, 281-312.
Russell, C. J., & Kuhnert, K. W. (1992). Integrating skill acquisition and perspective taking capacity in
the development of leaders. Leadership Quarterly, 3, 109-135.
Schmeck, R. R. (1988). Individual differences and learning styles. In C. E. Weinstein, E. T. Goetz, & P.
A. Alexander (Eds.), Learning and study strategies: Issues in assessment, instruction, and
evaluation (pp. 171-191). San Diego, CA: Academic Press.
Schneider, B., & Schneider, J. L. (1994). Biodata: An organizational focus. In G. S. Stokes, M.D.
Mumford, & W. A. Owens (Eds.), Biodata handbook: Theory research and use of biographical
information in selection and performance prediction (pp. 423-450). Palo Alto, CA: Consulting
Psychologists Press.
Schooler, C. (1990). Psychological effects of complex environments during the life-span: A review and
theory. Intelligence, 8, 254-281.

Seigler, R. S., & Richards, J.D. {1982). The development of intelligence. In R. J. Sternberg (Ed.),
Handbook of human intelligence (pp. 897-975). New York: Cambridge University Press.
Simonton, D. K. (1990). Personality and politics. Handbook of personality: Theory and research. New
York: Guilford Press.
Smith, J ., & Baltes, P. B. (1990). Wisdom related knowledge: Age/cohort differences in response to life
planning problems. Developmental Psychology, 26, 494-505.
Sternberg, R. J., Wagner, R. K., & Williams, W. M. {1995). Testing common sense. American
Psychologist, 50, 912-927.
Streufort, S., Nogami, G., Swezey, R. W., Pogash, R. M., & Piasecki, T. (1988). Computer assisted
training on complex managerial performance. Special issue: Dialogue on the relationship of
learning theory to instructional theory. Computers in Human Behavior, 4, 77-88.
Streufort, S., & Streufort, S. C. (1978). Behavior in the complex evironment. New York: Wiley.
Sweller, J. (1989). Cognitive technology: Some procedures for facilitating learning and problem solving
in mathematics and science. Journal of Educational Psychology, 81, 457-466.
Tusi, A. S. (1984). A role set analysis of managerial reputation. Organizational Behavior and Human
Performance, 34, 64-96.
Walberg, H. J., & Stariha, W. E. (1992). Productive human capital: Learning, creativity, and eminence.
Creativity Research Journal, 5, 323-340.
Ward, S. L., Byrnes, J. P .. & Overton, W. F. (1990). Organization of knowledge and conditional
reasoning. Journal of Educational Psychology, 82, 849-855.
Zaccaro, S. J. (1996). Models and theories of executive leadership: A conceptual/empirical review and
integration. Alexandria, VA: U.S. Army Research Institute for the Behavioral and Social
Sciences.
Zaccaro, S. J., Mumford, M. D., Connelly, M. S., Marks, M. A., & Gilbert, J. A. (2000). Assessment of
leader problem-solving capabilities. Leadership Quarterly 11(1), 37-64.