TMP D4 C1

This article was downloaded by: [Spain, Seth M.
][University of Nebraska]
On: 19 August 2010
Access details: Access Details: [subscription number 917350222]
Publisher Psychology Press
Informa Ltd Registered in England and Wales Registered Number: 1072954 Registered office: Mortimer House, 37-
41 Mortimer Street, London W1T 3JH, UK
Multivariate Behavioral Research

Publication details, including instructions for authors and subscription information:
http://www.informaworld.com/smpp/title~content=t775653673
Job Performance as Multivariate Dynamic Criteria: Experience Sampling

and Multiway Component Analysis
Seth M. Spaina; Andrew G. Minerb; Pieter M. Kroonenbergc; Fritz Drasgowa
a
University of Illinois, Urbana-Champaign b Target Corporation, c Leiden University,
Online publication date: 19 August 2010
To cite this Article Spain, Seth M. , Miner, Andrew G. , Kroonenberg, Pieter M. and Drasgow, Fritz(2010) 'Job Performance
as Multivariate Dynamic Criteria: Experience Sampling and Multiway Component Analysis', Multivariate Behavioral
Research, 45: 4, 599 — 626
To link to this Article: DOI: 10.1080/00273171.2010.498286
URL: http://dx.doi.org/10.1080/00273171.2010.498286
PLEASE SCROLL DOWN FOR ARTICLE
Full terms and conditions of use: http://www.informaworld.com/terms-and-conditions-of-access.pdf
This article may be used for research, teaching and private study purposes. Any substantial or
systematic reproduction, re-distribution, re-selling, loan or sub-licensing, systematic supply or
distribution in any form to anyone is expressly forbidden.
The publisher does not give any warranty express or implied or make any representation that the contents
will be complete or accurate or up to date. The accuracy of any instructions, formulae and drug doses
should be independently verified with primary sources. The publisher shall not be liable for any loss,
actions, claims, proceedings, demand or costs or damages whatsoever or howsoever caused arising directly
or indirectly in connection with or arising out of the use of this material.
Multivariate Behavioral Research, 45:599–626, 2010
Copyright © Taylor & Francis Group, LLC
ISSN: 0027-3171 print/1532-7906 online
DOI: 10.1080/00273171.2010.498286
Job Performance as Multivariate

Dynamic Criteria: Experience Sampling
Downloaded By: [Spain, Seth M.][University of Nebraska] At: 21:30 19 August 2010
and Multiway Component Analysis

Seth M. Spain
University of Illinois, Urbana-Champaign
Andrew G. Miner
Target Corporation
Pieter M. Kroonenberg
Leiden University
Fritz Drasgow
University of Illinois, Urbana-Champaign
Questions about the dynamic processes that drive behavior at work have been
the focus of increasing attention in recent years. Models describing behavior
at work and research on momentary behavior indicate that substantial variation
exists within individuals. This article examines the rationale behind this body
of work and explores a method of analyzing momentary work behavior using
experience sampling methods. The article also examines a previously unused set of
methods for analyzing data produced by experience sampling. These methods are
known collectively as multiway component analysis. Two archetypal techniques of
multimode factor analysis, the Parallel factor analysis and the Tucker3 models, are
used to analyze data from Miner, Glomb, and Hulin’s (2010) experience sampling
study of work behavior. The efficacy of these techniques for analyzing experience
sampling data is discussed as are the substantive multimode component models
obtained.
Correspondence concerning this article should be addressed to Seth M. Spain, Department of

Management, University of Nebraska at Lincoln, P.O. Box 880497, Lincoln, NE 68588-0497. E-mail:
smspain@gmail.com
599
600 SPAIN, MINER, KROONENBERG, DRASGOW
Historically, measures of job performance have been of great interest for orga-
nizational scientists attempting to establish the validity of selection systems
(Austin & Villanova, 1992). For instance, if a city uses a cognitive ability
measure to select its police officers, applied psychologists want to validate
those cognitive ability scores against a criterion such as the new officers’ arrest
records where arrest records serve as a measure of job performance. Validity
for predicting a criterion is commonly evaluated via the correlation between
the predictor measure and criterion. For the police example, this would be the
correlation between the cognitive ability scores and the arrest record.
Of central interest to applied psychologists is how to define job performance,

both conceptually and operationally (e.g., Austin & Villanova, 1992; Borman,
1991). Most validation research treats job performance as a monolithic and
static construct. There is considerable empirical evidence that job performance
is multidimensional (e.g., Campbell, 1991, 1994; Campbell, McHenry, & Wise,
1990; cf. Borman & Motowidlo, 1997). Furthermore, it is possible that job per-
formance is not stable over time (Hulin, Henry, & Noon, 1990; Keil & Cortina,
2001). In fact, job performance data can usually be classified by three modes:
the individuals assessed, the variables measured, and the times of measurement
(e.g., Cattell, 1952; cf. Smith, 1976). Ghiselli (1956) postulated three systematic
sources of variance in job performance data, that is, all three modes show
multidimensionality. Dalal and Hulin (2008) have called for job performance
studies that include changes over time, referring to this approach as multivariate
dynamic. Even this perspective ignores qualitative differences in performance
and thus potential dimensionality in the individuals mode. This article deals with
multidimensionality of job performance in all its modes and takes an individual
differences perspective to multivariate dynamic aspects of job performance.
In order to validly measure the frequency and the patterning of mental pro-
cesses in everyday-life situations procedures are needed that capture variations
in self-reports of those processes. To this end, experience sampling methodology
has been developed in which a participant at random or specific times has
to report on his or her mental state or those activities in which he or she is
involved at that moment. To capture those reporting instances participants are
supplied with beepers or more recently with palmtop computers. With the help
of these devices, several brief surveys each day are administered to partici-
pants (Csikszentmihalyi & Larson, 1987; Larson & Csikszentmihalyi, 1983).
As experience sampling allows information to be gathered from individuals
about several variables over time, the procedure provides data to study Ghiselli’s
(1956) three sources of job performance variance. Several methods for analyzing
such experience sampling data have been used in the literature, in particular
spectral analysis and multilevel modeling. This article demonstrates techniques
that have not been frequently used in the organizational literature and explores
their usefulness for understanding experience sampling data.
MULTIVARIATE DYNAMIC CRITERIA 601
The techniques presented in this article are methods of multiway component

analysis (see Kroonenberg, 2008, p. 16, for the distinction between mode and
way). We focus on the Tucker3 model (Tucker, 1966) and the Parallel factor
analysis (Parafac) model (Harshman & Lundy, 1984a, 1984b). These three-mode
models can be used to explore the individual, static, and dynamic structures
of work experience data. The results show multidimensionality in all three
modes. The meaning of these dimensions and implications for understanding
job performance are discussed.
THE THREE SOURCES OF JOB PERFORMANCE

VARIANCE
There are three sources of meaningful variance in job performance scores (Ghis-
elli, 1956): (a) static dimensionality, the ordinary factorial representation of
performance; (b) dynamic dimensionality, temporal factors influencing the per-
formance domain; and (c) individual dimensionality, variability in the type of
performance across persons in the same job. We consider each of these sources
in turn, beginning with individual dimensionality. For similar distinctions in a
more general sense, see Cattell (1952; refer to Figure 1).
Ghiselli (1956) described individual dimensionality as the way that individu-
als assigned to the same job within an organization perform that specific job in
qualitatively different ways, not merely with different proficiency. Dimensions
in the individuals mode differentiate types of employees. For example, two
salespersons may provide the same economic benefit to the organization, but
one contributes by directly making sales, whereas the other contributes by cre-
ating goodwill, encouraging customers to make purchases throughout the store
(Ghiselli, 1956, p. 4). Such individual difference dimensions of performance
could be important in a variety of situations. If the organization derives the same
economic benefit from different employees in different ways, these differences
should be reflected in selection and reward systems.
Static dimensionality refers to the latent structure of the variables measuring
job performance. Historically, the study of job performance was characterized
by a search for the “ultimate criterion,” a comprehensive index of performance.
It has been pointed out that this is an inappropriate way to conceptualize
performance (e.g., Dunnette, 1963; Ghiselli, 1956). There is evidence across
many jobs that overall job performance can consist of as many as eight dimen-
sions (Campbell, 1994; Campbell et al., 1990). At minimum, job performance
researchers should consider both task performance, the technical core of the
job, and contextual performance, the social and nontechnical contributions an
individual makes at work (Borman & Motowidlo, 1997). These dimensions
have been found to independently contribute to supervisors’ perceptions of their
FIGURE 1 The Basic Data Relations Box (Cattell, 1952).
subordinates’ overall job performance (Motowidlo & Van Scotter, 1994). For
instance, consider two salespeople who have equal sales. One is known to be
a loner, whereas the other gives advice and assistance to coworkers. The latter
will be viewed as the superior performer due to the social contributions this
salesperson makes.
It may be necessary to have a single measure of job performance, for example,
when establishing the predictive validity of a selection battery (Hulin, 1982).
One might then construct some composite of scores on various performance
dimensions. The weighting scheme chosen to combine different dimensions of
job performance has been found to greatly impact those validity estimates in
Monte Carlo simulations (Murphy & Shiarella, 1997). These authors found that
34% of the variance in validity estimates over simulated samples was accounted
for by the performance dimension weighting scheme.
Finally, the dynamic nature of performance criteria is important to consider
for employee selection. There are well-documented changes in the predictive
relationships between selection instruments and performance measures (Alvarez
& Hulin, 1972). In an academic example, SAT scores are most predictive of
first-semester college grade point average (GPA), and the correlations between
SAT and GPA decrease after that (e.g., Humphreys, 1976). These changes in
predictive validity have been found to be virtually ubiquitous (Hulin et al., 1990).
The fact that predictive validity changes over time is usually called validity
degradation and is the primary approach to the study of dynamic criteria, though
there are other approaches (e.g., Austin, Humphreys, & Hulin, 1989; Barrett,
Caldwell, & Alexander, 1985). This work is summarized in meta-analyses by
Hulin et al. and Keil and Cortina (2001), which show that validity degradation
is pervasive and occurs with all manner of ability predictors.
If performance changes over time, it would useful to find predictors of the
change itself. Personality measures have been used in addition to cognitive
ability measures to predict individual growth curves for performance criteria

(e.g., Zyphur, Bradley, Landis, & Thoresen, 2008). The results of this study
indicate that both cognitive ability and conscientiousness predict initial academic
performance, but only conscientiousness predicts performance trajectories. This
may happen because early performance is a transition phase of skill acquisition
and later performance is a maintenance phase (Murphy, 1989). In a study testing
that proposition, both conscientiousness and extraversion predicted maintenance
performance differences between participants, but only conscientiousness pre-
dicted maintenance performance growth; moreover, agreeableness and openness
to experience predicted both performance differences and trends in the transition
phase (Thoresen, Bradley, Bliese, & Thoresen 2004). These attempts remain
univariate in their approach to criterion measurement, however. Theoretical
models should guide the sampling of variables as research progresses into a
multivariate dynamic framework.
One such model is provided by affective events theory (Weiss & Cropanzano,
1996). These authors suggest that workplace behavior comes in two basic kinds:
affect driven and judgment driven. Workplace events cause affective reactions,
and these affective reactions directly influence affect-driven behavior. But these
affective reactions also influence job attitudes that in turn directly influence
judgment-driven behaviors. It is hypothesized that momentary affect should thus
show stronger relationships with momentary behaviors such as work withdrawal
(e.g., taking long coffee breaks or surfing the Web) and that job attitudes
should have stronger relationships with more considered behaviors such as job
withdrawal (e.g., job search, turnover intentions, quitting). Furthermore, it is
expected that individual differences in personality will moderate both the link
between events and affect and predict the affective reactions themselves.
In complement to affective events theory, the episodic process model (Beal,
Weiss, Barros, & MacDermid, 2005) suggests that there will be important
momentary fluctuations in the affective and regulatory resources available for
employees to apply to performance behaviors. This model articulates reasons
performance behaviors should meaningfully vary within persons over short time
periods. For example, if my supervisor yells at me, and I then need to interact
with a client, I may have to regulate my emotional display to appear positive
to the client. This act of emotion regulation uses up some of my regulatory

resources and may therefore make it more difficult for me to focus my attention
on a report I need to write later in the day. Obtaining evidence to test such a
model requires research designs that are capable of untangling both within- and
between-individual variability.
Experience sampling methods are ideally suited to explore dynamic models of
work behavior because measurements may be taken throughout the work day on
several variables. For example, Weiss, Nicholas, and Daus (1999) tested propo-
sitions drawn from affect events theory about the influence of job beliefs and
moods on overall job satisfaction. Dalal, Lam, Weiss, Welch, and Hulin (2009)
investigated the dimensionality of organizational citizenship and counterpro-
ductive behaviors and their relationships to state affect and performance. Miner,
Glomb, and Hulin (2005, Miner & Glomb, 2010) found general support that indi-
viduals modify their work behaviors in response to mood changes. These studies
demonstrate that a substantial portion of the variance in organizationally relevant
variables is within-person, underscoring the need for longitudinal studies.
Experience sampling data are three-mode (Persons Variables Occasions)
and are frequently analyzed with multilevel models with the occasions mode
nested within persons. This article addresses whether such data may be examined
using multiway analysis, as illustrated in the next section. Situations where
these analyses would be useful include examination of whether the structures
of counterproductive and citizenship behaviors change over time, or, even more
fitting for multiway methods, research on whether different groups of employees
display different patterns of change in these structures. Multiway models are a
natural way of addressing such questions but may prove difficult to implement
with experience sampling data. For instance, in the data presented here, the
measurement occasions occurred randomly within 2-hr windows. Therefore,
Participant 1’s Time 1 is not strictly simultaneous with Participant 2’s but should
be basically equivalent. One goal of this article is to serve as a methodological
exercise in an attempt to judge whether this assumption of near simultaneity is
tenable. This concern is further addressed in the Discussion section.
MULTIWAY ANALYSES
When viewed in the way specified by Ghiselli (1956), the performance domain
is defined by a three-way data relation box (Cattell, 1952). This box is defined
by its three modes: individuals, variables, and occasions. This box is referred to
as an array, a generalization of the matrix concept (see Figure 1). In Figure 1,
participants, variables, and occasions are all both modes and ways. The front
side of the array in Figure 1, the participants by variables “slice” is the matrix
we usually handle in psychological data. Each measurement occasion in a
longitudinal/experience sampling study would have such a matrix. Stacking

these slices as in Figure 1 leads to a three-mode array. We next consider
two broad classes of multiway component analysis that are considered in this
article, the Parafac model and the Tucker3 model. These are both components
analysis models; however, both were initially referred to as factor analysis (e.g.,
Tucker, 1966). In this article, we refer to components, except when discussing
the historical development of the models.
Cattell (1944) presented a solution to the problem of rotational freedom in
standard factor analysis, which he called the principle of parallel proportional
profiles. Cattell’s (1944) idea was to measure the same individuals on the same
variables across two systematically different occasions, say before and after an
experimental manipulation. If one recovered the same factors at both occasions
then this common orientation of the factors or these parallel proportion profiles
could, according to Cattell (1944), only occur if the factors represented real
psychological constructs.
The Parafac model (Harshman, 1970) is essentially both a model formulation
and computational solution of this principle applied to an arbitrary number of
conditions or occasions. Moreover, Harshman and Carroll and Chang (1970)
showed that the parallel proportional profile principle could be extended to the
n-way case. The Parafac model is mathematically identical to the canonical
decomposition model in multidimensional scaling (Carroll & Chang, 1970). In
this article, we refer only to the former name. The Parafac model is presented
in matrix notation as
Xk D ADk B0 C Ek ; (1)
where Xk is the kth frontal slice of the data array X (matrices are boldfaced and
multiway arrays boldfaced and underlined). A is the coefficient matrix for the
first mode; B0 is the transpose of the coefficient matrix of the second mode; Dk
is a diagonal matrix containing the kth row of the third-mode coefficient matrix
along its diagonal, that is, dssk D cks. Ek is the kth frontal slice of residual array
E. In sum notation this becomes
xijk D †s dssk Œais bjs C eijk D †s csk Œais bjs C eijk ; (2)
which shows that for the kth slice of the data array the dssk -coefficients provide
weights for the coefficients of the other two modes. They are the “proportion-
ality” constants that capture the systematic variation between slices of the data
array (cf. Harshman & Lundy, 1984a). A dssk (csk ) coefficient defines the relative
importance of component s to occasion k. It should be noted that the model is
in fact proportional with respect to all its modes.
It is important to distinguish two types of variation: system variation, pertain-
ing to the system as a whole, and object variation, which pertains to variation
of specific (groups of) objects, that is, part of the system but not the system as a
whole. For the Parafac model to be applicable sufficient system variation should
be present in the multiway data. System variation involves a conceptualization
of components that lie “in the system” under examination such that the same
instances of the components affect all of the objects; there should be parallel
proportional profiles. The object variation model involves a conceptualization
of components as having an instance of the component “in” each object under
study, such that the components are properties of (groups of) individuals under
study. The system model implies synchronous variation in component influences
across levels of the third mode (e.g., occasions), whereas the object model does
not imply such synchronous variation (cf. Harshman & Lundy, 1984a, pp. 130–
133). The basic Parafac model has no provision for modeling object variation.
Therefore, if substantial object variation is present, a more complicated model
such as the Tucker3 should be used.
Tucker (1964, 1966) formally introduced the three-mode factor analytic model
for three-way data as an extension to the well-known two-mode model for two-
way data. This model was later named after him by Kroonenberg and de Leeuw
(1980). The Tucker family of models can account for both system and object
variability. Of these models the Tucker3 is the most complex and it can also be
shown to be more complex than the Parafac model. The sum notation for the
model is
xijk D †p †q †r Œaip bjq ckr gpqr C eijk : (3)
Here, aip represents the coefficient of the i th individual on the pth A-mode
factor, whereas bjq represents the coefficient of the j th variable on the qth
variable factor, and ckr represents the coefficient of the kth timepoint on the r th
time factor. Finally gpqr represents the element of the core array that links the
pth, qth, and r th factors in the A-, B-, and C-modes, respectively, with each
other. It can also be interpreted as a representation of the interaction between the
factors of the three modes (for further details see Kroonenberg, 2008, Sec. 4.8).
Thus the columns of the coefficient matrices A, B, and C are the components
that may be represented as vectors in graphs displaying the component spaces.
For convenience, we often speak of component scores for participants, compo-
nent loadings for variables, and component scores or weights for timepoints.
Only the variable loadings can, however, be understood as in the two-way
case, that is, as variable-component correlations, if the variables have been
centered and normalized in a specific way (see the following). In most cases, the
interpretation of the component scores proceeds in the same way as component
scores in two-way component analyses where the components are in normalized
coordinates, that is, have unit sums of squares. Another way of thinking about
the component matrices is in terms of “idealized” instances of the real objects
in that mode, that is, ideal individuals, ideal (or latent) variables, and ideal
measurement occasions or trends (Kroonenberg, 2008, Sec. 9.4; Tucker, 1966).
The core array is what allows Tucker models to account for object variability
where the Parafac model cannot. So whereas Component 1 in the individuals
mode must correspond to Component 1 in the variables and time modes in
a Parafac analysis, the same does not apply to a Tucker3 model. The core
allows three-way interactions between the components in different modes and
for different numbers of components in each mode. A number of ways of
interpreting the core are possible. One may view the core array as “idealized
data” such that each element represents the score on the ideal variable by
the ideal person at the ideal measurement occasion corresponding to that core
element (cf. Tucker, 1966). As the scalar notation makes clear (Equation 3),
one can also think of the core elements as regression weights for predicting the
original data using the three-way interaction of the components (Kroonenberg,
2008, pp. 225–228). Thus, core elements give the importance of a combination
of mode components for reproducing the original data.
RESEARCH QUESTIONS
Inn, Hulin, and Tucker (1972) offered perhaps the only multivariate dynamic
criterion study using multiway analysis. They fit a Tucker3 model to data from a
sample of 184 airline reservation agents measured five times on 11 performance
variables. These authors found a 4 3 3 solution, demonstrating multidi-
mensionality in each mode, consistent with Ghiselli’s (1956) three sources
conjecture. Given the dearth of previous research, the current study is largely
exploratory and we propose few a priori hypotheses. The major expectation of
this study is that multiple components will be extracted in each mode of the data,
consistent with Ghiselli and with Inn et al.. Given previous work on affect cycles
in experience sampling designs, it is likely that at least one of the time compo-
nents will show cyclic trends in loadings (Weiss et al., 1999). These loadings are
analogically similar to variable loadings on growth parameters in latent growth
curve analysis (e.g., Chan, 1998), though they are both exploratory in character
and expected to display cyclical rather than curvilinear patterns. Further, we
expect that we should be able to recover three B-mode components for the ob-
jective, behavioral, and self-rated performance variables discussed in the Method
section. We have no hypotheses regarding individuals-mode components.
This article is primarily a methodological exploration. We want to know if
it is possible to fit well-behaved multiway models to experience sampling data.
We expect to be able to do so, and the only remaining question is whether the
Parafac model will be sufficient. The Parafac model is a constrained Tucker3
model, which itself can be seen a constrained two-way principal components
analysis (PCA) of the wide combination-mode matricization (e.g., I JK) of

the data array X. Therefore, the sums of squares for each model (SSmod ) must be
SSmod(PCA) SSmod (Tucker3) SSmod (Parafac) (4)
provided all models have equivalent numbers of components (Kiers, 1991).

Therefore, Tucker3 and PCA models will provide similar though better fits
to the data than an equivalent Parafac model. However, if the Parafac model
is appropriate, the additional parameters of these models will either model
systematic variation redundantly or simply model noise (Smilde, Bro, & Geladi,
2004, pp. 154–155). If a degenerate Parafac solution is obtained, it usually
indicates the presence of Tucker structure or object variability.
METHOD
Data Source and Participants

The experience sampling data analyzed here were originally collected by Miner
and Glomb (2010), who sampled 100 individuals from a pool of about 300
incumbents at a Fortune 500 technological sector call center. These individuals
served in either customer service or technical support capacities. Sixty-seven
of the invitees actually participated. Of these, 55 participants had sufficient
criterion data for analyses. Those 55 participants were mostly White (94%) and
nonstudents (85%). Fifty-one percent were male, 78% had at least some college
education, and 66% worked 9-hr shifts. The mean age of participants was 34
years (SD D 11 years) and their mean tenure with the organization was 2.4
years (SD D 2:3 years).
Measures and Procedures

Participants’ palmtop computers signaled them by beeping either four or five
times a day, depending on whether they work a 5-day, 9-hr shift or a 4-day,
11-hr shift, respectively. Participants completed surveys each workday for 3
weeks, resulting in a total of 60 possible measurement opportunities for each
respondent. Participants were excluded from analyses if they answered one half
(30) or fewer of the surveys. Participants tended to miss most surveys late in
the study, so we used the first 40 measurement occasions.
Experience sampling measures. Participants answered questions about

whether they solved the customer’s problem; made three ratings of their own
performance; and responded to eight items sampling behaviors related to task
performance, organizational citizenship behavior, and work withdrawal. Par-

ticipants were also asked if they were engaged in several work behaviors,
which were scored dichotomously. The behavioral items were aggregated into
unit-weighted composites for focal performance (solving the problem, doing
work tasks), being on the job (at workstation, away but on the job), negative
work behaviors (withdrawal, doing personal tasks), positive citizenship behaviors
(helping, doing organizational citizenship), and neutral behaviors (at lunch, on
break). In addition to the self-reported measures, performance was also indicated
by objective measures of average call handle time, average call wait time, and
average call hold time in the 30-min window in which the signal occurred.
Individual difference measures. Additionally, participants completed sev-

eral individual differences measures in a poststudy survey. These measures can
be used to explore the meaning of the individuals-mode components. These
measures were the Trait Meta-Mood Scale (Salovey, Mayer, Goldman, Turvey,
& Palfai, 1995) and the International Personality Item Pool Extraversion and
Neuroticism (IPIP; Goldberg, 2001). These measures are linked to dispositional
affectivity, which is predicted by affective events theory to moderate an indi-
vidual’s reactions to affective events as well as to directly influence affective
reactions to events. Trait meta-mood is a form of emotional intelligence and
related to an individual’s ability to recognize and regulate his or her feelings.
IPIP extraversion is primarily an index of sociability and gregariousness, whereas
neuroticism tends to focus on anxiety/stress reactance. Participants also com-
pleted the Job Descriptive Index, a measure of job satisfaction (Smith, Kendall,
& Hulin, 1969). Affective events theory predicts that job satisfaction should
be more related to judgment-driven behaviors than to impulsive behaviors.
Additionally, job satisfaction has an empirical association with job performance,
so it is reasonable that some individuals-mode components may be associated
with job satisfaction. Finally, self-reports of average work and job withdrawal
and citizenship performance were assessed as well.
Description of the Experience Sampling Data Array

Participants tended to miss many surveys in the last week, resulting in sub-
stantial missing data after the 40th measurement occasion. Therefore, the data
we analyzed consisted of 55 participants by 11 variables by 40 measurement
occasions (24,200 data points). About 17,295 data points contained valid entries,
and about 6,905 did not (28.5% missing). Missing entries were handled by
imputing model-implied values. Missing values were initialized with two-way
MANOVA estimates. The information in a three-way array is quite rich, and
with an array of sufficient size, a substantial portion of missing values can be
handled relatively reliably but often requires special care in defining starting
values for the missing-data locations (Smilde et al., 2004). Though this data set
is not large enough to allow for cross-validation, the stability of the results were
examined using several random starts.
RESULTS
Both Parafac and Tucker3 models were fit to the data. Analyses were performed
using the 3WayPack suite of programs (Kroonenberg, 1994). Multiway models
were compared based on variance accounted for (VAF) measures. The VAF of
the model was divided by number of factors estimated in order to compare the
model’s explanatory power with its parsimony (Timmerman & Kiers, 2000).
Similar indices were discussed in Kroonenberg and Oort (2003; Kroonenberg,
2008, pp. 179ff.). These are not formal fit indices but aid the researcher’s decision
making regarding which models are good candidates for interpretation.
Component models assume that variables are ratio-scaled ones (Harshman
& Lundy, 1984a). However, psychological measurements mostly have interval
properties. Centering will change the data into ratio-scale deviations from the
mean so that they can be handled in a component analysis. Harshman and
Lundy (1984b) and Kroonenberg (2008, chap. 6) discuss appropriate types of
centering for three-way data, and centering across participants per variable-
timepoint combination is their recommended procedure for the present data.
Our data were accordingly centered across the participant mode, such that
the mean for each variable at each measurement occasion was subtracted out
of each measurement. This is the most appropriate centering strategy because
the starting point of the study is arbitrary. An accompanying advantage of this
centering is that the matrix of variable-occasion means can be seen as the score
of the average person so that the deviation scores can be interpreted with respect
to this average person. Additionally, the three objective measures (call handle
time, call wait time, and call hold time) were corrected for nonnormality by
taking their natural logarithms. Finally, the variables were size-normalized over
all participants-timepoint combinations so that the values of the variables were
comparable as standard scores across variables and remained comparable per
variable across all occasions. To be precise, the scores analyzed with the three-
mode models were
zijk D .xijk x jk /=sj ; (5)
where a dot replaces the index over which the sum was taken. A final advantage
of this form of standardization is that given proper scaling of the output (see
the following) the component loadings for the variables can be interpreted as
variable-component correlations (Harshman & Lundy, 1984a).
Descriptive Statistics
Presenting even descriptive statistics for the entire data array would be cumber-
some. Thus descriptive statistics for the A-mode matricization are presented in
Table 1. Matricization is the unfolding of the three-mode data array into a two-
way data matrix extended along one of the modes of data classification. Given
an i j k data array, the A-mode matricization is of order ik j (cf. Kiers,
2000), where the participant mode (k) is the slower running mode such that the
rows of the matrix consist of the 1st participant’s 40 measurement occasions
followed by the 2nd participant’s 40 measurement occasions, and so on. Table 1
presents the means, standard deviations, and correlations for the variables in
the A-mode matricization. This correlation matrix can be interpreted as usual,
though it conflates within- and between-person covariances.
Means Analysis
Prior to three-way analysis, we explored the mean time trends for the variables,
that is, trends for means that were removed before the three-way analyses.
Figures 2 through 4 present the time trends for seemingly similar variables.
Figure 2 displays the self-rated variables: average handle time, average quality
of service, and overall service. These variables display very similar time trends.
Figure 3 shows the trends for the behavioral variables: the focal performance
composite, withdrawal behaviors, citizenship behaviors, neutral behaviors, and
location. Again, these display highly similar trends. The objective measures,
average handle time, average wait time, and average call duration, do not show
such a clear pattern.
Table 2 displays the component loadings for a varimax-rotated PCA of the
time series for these variables averaged over participants. Two components were
recovered: the self-rated variables loaded strongly on one, with location, focal
performance, withdrawal, and neutral behaviors loading strongly on the other.
Log-average handle time and citizenship performance also showed reasonable
loadings on this component. The call duration and call wait time variables did
not load strongly on either component.
Three-Way Analysis
Based on several initial solutions, we decided to eliminate 1 participant from
the three-way analyses. This participant had much too high an influence on the
solutions. After this, the preferred models are 2 2 1, 3 2 2, 4 2 2,
and 4 2 4 Tucker models and a two-component Parafac model. The two-
component Parafac model was a rotation of the 2 2 2 Tucker model (see
Table 3).
612
TABLE 1
Descriptive Statistics for the A-Mode Matricization of ESM Data Array
M SD 1 2 3 4 5 7 8 9 11 10 6
1 Rated average handle time 1.91 0.74 1.00

2 Rated quality of service 2.05 0.54 0.39 1.00
3 Rated overall service 1.99 0.57 0.50 0.62 1.00
4 Focal performance 0.52 0.39 0.05 0.04 0.07 1.00
5 Focal location 0.72 0.45 0.01 0.01 0.00 0.69 1.00
7 Withdrawal 0.51 0.30 0.02 0.01 0.00 0.53 0.72 1.00
8 Neutral behavior 0.13 0.34 0.01 0.01 0.00 0.35 0.64 0.23 1.00
9 Call duration 6.26 0.89 0.02 0.06 0.05 0.09 0.02 0.04 0.02 1.00
11 Call handle time 4.85 1.30 0.10 0.06 0.04 0.06 0.03 0.01 0.07 0.40 1.00
10 Call wait time 2.36 2.02 0.01 0.11 0.07 0.07 0.09 0.08 0.10 0.26 0.05 1.00
6 Citizenship 0.18 0.31 0.05 0.01 0.01 0.07 0.12 0.04 0.12 0.01 0.11 0.11 1.00
Note. Call time variables were log-transformed prior to analysis. Correlations in boldface are statistically significant at the .05 level.
FIGURE 2 Centered average time series for self-rated customer service variables. Corre-
lations among time series: 0.3–0.6.
Parafac analyses. We fit a two-component Parafac model using multi-

ple random starts. Three of these starts produced absolute values of Tucker’s
congruence coefficients greater than .85. The components in the two-component
solutions are highly negatively congruent with an average congruence coefficient
of 0.84 and an average core consistency of 25.9. These results indicate that
the higher component Parafac solutions are degenerate. There are several options
for dealing with degenerate solutions. One is to impose orthogonality on the
factors in one mode. Another option is to attempt to fit a more general three-
way model such as a Tucker3.
Tucker3 analyses. Figure 5 displays the time components from the

3 2 2 Tucker model solution with loess curves fitted to them with 70%
FIGURE 3 Centered average time series for behavioral variables. Correlations between .6
and .9 except with Citizenship, which range from .2 to .4.
of the data used for the local regressions (Cleveland & Devlin, 1988). The
first component has been rotated to optimal constancy. The second component
is orthogonal to the first one. There is no clear periodicity (i.e., performance
cycles) in these time components. However, to uncover such periodicity would
require that the measurement occasions were spaced evenly and truly taken at
the same time for all participants. We constructed joint biplots to investigate
the relationships between individuals and variables for the two time components
(e.g., Kroonenberg, 2008, Appendix B). Variables are displayed as vectors in the
two-dimensional space, and individuals projecting highly on a variable vector
scored highly on it; individuals plotted near each other have similar profiles.
Figure 6 is the joint biplot of individuals and variables for Time Component
1 (variable coordinates presented in Table 4). Because Time Component 1
is largely constant, this biplot represents individual differences in how the
participants carried out their job duties over time. For instance, we can see how
Participants 19, 20, 23, 46, and 50 had high scores on call wait time but low
FIGURE 4 Uncentered average time series for logged call time variables (objective
performance). Correlations between .0 and .3. Note. The trends for these variables are
rather dissimilar. Therefore they are displayed uncentered so as to show their unique trends
more clearly.
scores on call duration and handle time. On the other hand, Participants 25, 40,
and 48 scored highly on call duration and handle time but low on call wait time.
Similarly, Participants 20, 23, and 27 had high scores on citizenship whereas Par-
ticipants 3 and 9 had very low scores on citizenship. The objective call time mea-
sures and citizenship are the most important variables for this time component.
Figure 7 displays the biplot for the second time component (variable co-
ordinates are also presented in Table 4). This component showed much more
pronounced fluctuations than Component 1 and an increasing trend toward the
end of the study. This suggests that the importance of the relationships displayed
in Figure 7 became more important over the course of the study. This increase
of importance is especially due to handling time and citizenship but also with-
drawal, focal performance, focal location, call duration, and quality of service.
TABLE 2
Varimax-Rotated Component Matrix for Time Trends
of Variable Averages
Component
Variable Name 2 1
Rated overall service 0.865

Rated average quality 0.846
Rated average handle time 0.638
Focal location 0.962
Focal performance 0.212 0.882
Withdrawal 0.879
Neutral behavior 0.837

Call handle time 0.540
Citizenship 0.458
Call duration 0.267
Call wait
Note. Values smaller than .20 are not shown. Variables

have been ordered to illustrate component structure. Loadings
in boldface are considered substantial enough to define the
component.
TABLE 3
Fit Information on Parafac and Tucker Models
Overall
Proportional Fit Proportional Fit Components
Completed Data
Valid Completed
Model Data Data 1 2 3
PF2 .098 .272

PF3 .138 .298
T3-1 1 1 = PF1 .059 .208 .208
.208
.208
T3-2 2 2 .098 .272 .219 .053
.221 .052
.264 .008
T3-3 2 2 .114 .287 .196 .060 .031
.209 .078
.242 .045
T3-3 3 2 .122 .326 .237 .055 .033
.239 .052 .035
.291 .035
T3-3 3 3 .137 .315 .224 .059 .032
.219 .054 .041
.265 .034 .015
Note. Because the presence of a large amount of missing data, the fit of the completed data
of an analysis with fewer components may have a better completed fit than a model with more
components. PF D Parafac; T3 D Tucker3.
616
FIGURE 5 Time component loading of the 3 2 2 Tucker3 solution after rotation of

the first component to optimal constantness.
Participants with positive scores, such as 30, show increasing trends over time.
Those with negative scores, like 3 and 9, show decreases.
To aid interpretation of the individual components, we examined the cor-
relations of these components with individual difference measures (e.g., Van
Mechelen & Kiers, 1999). The first component correlated positively with meta-
mood clarity and negatively with overall job withdrawal (r D :30, p D :04 and
r D :32, p D :03, respectively). The second component correlated with meta-
FIGURE 6 Joint biplot for participants and variables for the first time component.
TABLE 4
Variable Coordinates for Joint Biplots
Time Time
Component 1 Component 2
Component 1 2 1 2
Call wait time 0.458 0.109 0.069 0.073

Call duration 0.455 0.027 0.140 0.063
Call handle time 0.400 0.174 0.296 0.033
Citizenship 0.164 0.378 0.271 0.064
Withdrawal 0.030 0.201 0.187 0.018
Focal location 0.035 0.189 0.178 0.016
Focal performance 0.053 0.186 0.182 0.014
Rated average handle time 0.112 0.079 0.109 0.006
Rated overall service 0.125 0.077 0.113 0.008
Rated quality of service 0.140 0.099 0.137 0.007
Neutral behavior 0.013 0.039 0.029 0.006
Note. Variable ordering has been changed to illustrate influence. Loadings in boldface are
considered substantial enough to define the component.
FIGURE 7 Joint biplot for participants and variables for the second time component.
mood repair positively (r D :29, p D :05) and marginally positively with overall
citizenship performance (r D :27, p D :07) and marginally negatively with over-
all work withdrawal (r D :25, p D :10). The third component correlated with
overall citizenship performance (r D :38, p D :01) and marginally negatively
with satisfaction with supervisor ( :23, p D :10). The first component has
perhaps the clearest interpretation as these individuals report great awareness of
their moods and high overall job withdrawal: these employees are likely very
in tune with their emotions and unlikely to quit. They are probably having a
relatively positive work experience relative to the average employee. The second
component also has a reasonably clear interpretation. Given that individuals are
able to intentionally repair their moods and report high levels of citizenship
performance and low work withdrawal, these are likely individuals who can
deal with negative emotions effectively and remain engaged with their work.
The third component may again reflect some form of engagement. Employees
scoring high on this component report high citizenship in spite of lower ratings
on supervisory satisfaction.
The variables-mode component loadings are displayed in Table 5. The two
components cleanly separate the objective and behavioral measures, and the
self-reported measures do not load substantially on either component. With the
objective measures, call duration and handle time have the opposite sign as call
wait time. Citizenship performance has the highest loading for the behavioral
measures, with focal performance, location, and withdrawal all loading between
.31 and .34. Interestingly, all of these loadings, even withdrawal, are positive.
It is also interesting that the more direct self-reported measures of performance
do not systematically load on these components.
The Tucker core is shown in Table 6. The combination of the first components
in all modes has the largest core element and accounts for the most variance.
The most substantial elements following this are the combination of the second
participant and the behavioral variables component with each of time compo-
nents (g221 and g222 , respectively) accounting for 3% and 2% of the original
variance, respectively.
TABLE 5
Component Loadings for B Mode (Variables)
Variable Objective Behavioral
Rated average handle time 0.159 0.115

Rated quality of service 0.199 0.145
Rated overall service 0.175 0.110
Focal performance 0.118 0.314
Focal location 0.098 0.323

Withdrawal 0.095 0.344
Citizenship performance 0.091 0.692
Neutral behaviors 0.005 0.071
Call duration 0.546 0.138
Call handle time 0.537 0.223
Call wait Time 0.526 0.283
Note. Variables are ordered to best show component

structure. Loadings in boldface are considered substantial
enough to define the component.
TABLE 6
Tucker Core and Variance Explained for Each Three-Way
Combination of Components
Time Component 1
Variance
Objective Behavioral Explained
Participant 1 1.437 0.216 0.188 0.004

Participant 2 0.002 0.589 0 0.031
Participant 3 0.122 0.391 0.001 0.014
Time Component 2
Variance
Objective Behavioral Explained
Participant 1 0.137 0.173 0.002 0.003

Participant 2 0.347 0.442 0.011 0.018
Participant 3 0.286 0.297 0.007 0.008
DISCUSSION
The Tucker3 model provides a parsimonious explanation of the observed data,

even though the overall variance explained for all of the models is rather low.
Experience sampling data prove difficult to analyze for a number of reasons. The
procedures are intrusive, which can lead to large amounts of missing data when
participants are too busy or simply choose not to answer questions. Furthermore,
individuals were studied over a relatively short time and were measured at points
very close together in time. Perhaps if measurement occasions were sampled at
fully equivalent timepoints across individuals or if the study took place over
organizationally meaningful epochs, such as a new product rollout or layoff
announcement, the Parafac model may have proven more useful in describing
the data.
This theme bears some greater consideration. Often with longitudinal and
experience sampling studies, in particular, researchers wade into the stream of
events with no real care as to when they do so. Put simply, Time 1 often is
not really Time 1 but an arbitrary starting point for the study. Likewise, the
studies often end at an equally arbitrary point in time. Multiway thinking can
help encourage researchers to treat time as an important facet in the design of
studies and sample measurement opportunities appropriately.
Implications for the Analysis of Experience

Sampling Studies
This study attempts to fit a dynamic structural model to data derived from
an experience sampling design. ESM studies often come in one of two types:
event-triggered and signal-triggered. In event-triggered studies, participants opt
to answer surveys when a particular event or type of event occurs. In signal-
triggered studies, individuals respond to surveys when signaled by their beeper
or palmtop computer. When the ESM study is event-triggered, measurement
occasions can only be viewed as nested within individuals. In signal-triggered
designs, signals will be random within time windows. Our results indicate that
these measurement occasions may be considered, although perhaps not strictly
equivalent, comparable across individuals.
This article demonstrates that dynamic and structural investigations may be
conducted with signal-triggered experience sampling data by using methods such
as multiway analysis. Using tools such as multilevel modeling is also useful
but might sacrifice information that could be examined using techniques like
multiway analysis. Furthermore, had the data been collected with the intent of
examining structure and dynamics, the performance variables would likely have
been more systematically sampled from the repertoire of performance behaviors.
This would probably allow for stronger substantive conclusions regarding the
multivariate dynamic criterion space. As it stands, it is very interesting that

the self-reported performance variables did not load substantially on either
of the variables-mode components. Given that the components reflect behav-
ioral activities and objective measures of performance, this suggests that the
self-reported measures may not accurately represent what a worker is actually
doing.
Limitations and Future Directions

There are three major limitations to this study. The first is that the data were not
collected for the purpose of a structural analysis. Therefore, the amount of shared
variance among the indicator variables may not adequately cover the latent job-
performance variables. Although this does not preclude the analyses described
in this article, data collected with such analyses in mind would undoubtedly
provide cleaner results. Second, there was a substantial amount of missing data.
However, only Participant 16 appeared to show undue influence on the model
solutions, and this participant was removed from the analyses reported here.
Third, the sample involved only two groups of employees from one organization,
limiting the external validity of the results. Further study of the job performance
domain and its three sources of variance should be examined across a sample
from the population of jobs. This requires collecting experience sampling and
other longitudinal data in many organizations with many different types of
employees. One possibility is applying these techniques to large samples of
jobs that are appraised on multiple performance indicators at fixed intervals,
such as in the military. Such formal performance evaluations at multiple points
in time would address questions about the normative development of job per-
formance.
However, without examining other studies of performance and its determi-
nants, the underlying dynamics in studies with longer measurement epochs
will never be fully understood (cf. Beal et al., 2005). Data to answer these
sorts of questions should be collected using multiple methods and examined
using multiple analytic procedures. Such a strategy will allow for a scientific
understanding of the dynamic interaction of individual and workplace attributes
in the study of organizational behavior.
CONCLUSION
The major performance theories in applied psychology indicate that the criterion
space is a multidimensional one. However, these theories are static; they are
largely agnostic as to the dynamic nature of performance and its determinants.
It is well established that the relationship between measures of predictors of
job performance and performance appraisal measures degrades over time. This
situation suggests the possibility of dynamism on both sides of the performance
prediction model. In order to understand the dynamic relationship between
two variables, we must first understand the processes underlying each of the
variables.
The results of this study provide further evidence that organizational scientists
must attend to dynamic processes. Further, these results echo the exhortations of
Ghiselli (1956) and Dalal and Hulin (2008) to more broadly consider the criterion
space when designing personnel interventions. It is important that these between-
and within-person sources of lawful variance be considered in such endeavors.

Finally, continuing to think of the performance prediction problem in terms of
bivariate correlations prevents the development of a truly scientific understanding
of the processes by which attributes of employees interact dynamically with the
work environment to produce job performance.
ACKNOWLEDGMENTS
Portions of this research were presented at the 2007 Annual Meeting of the
Academy of Management, Philadelphia, PA. This article is based on the first
author’s master’s thesis. We thank Chuck Hulin, Sungjin Hong, and Theresa
Glomb for their assistance and for comments on earlier drafts of this article.
REFERENCES
Alvarez, K. M., & Hulin, C. L. (1972). Two explanations of temporal changes in ability-skill
relationships: A literature review and theoretical analysis. Human Factors, 14, 295–308.
Austin, J. T., Humphreys, L. G., & Hulin, C. L. (1989). Another view of dynamic criteria: A critical
reanalysis of Barrett, Caldwell, and Alexander. Personnel Psychology, 42, 583–596.
Austin, J. T., & Villanova, P. (1992). The criterion problem: 1917–1992. Journal of Applied Psy-
chology, 77, 70–88.
Barrett, G. V., Caldwell, M. S., & Alexander, R. A. (1985). The concept of dynamic criteria: A
critical reanalysis. Personnel Psychology, 38, 41–56.
Beal, D. J., Weiss, H. M., Barros, E., & MacDermid, S. M. (2005). An episodic process model of
affective influences on performance. Journal of Applied Psychology, 90, 1054–1068.
Borman, W. C. (1991). Job behavior, performance, and effectiveness. In M. Dunnette & L. Hough
(Eds.) Handbook of industrial/organizational psychology (Vol. 2, pp. 271–326). Palo Alto, CA:
Consulting Psychologists Press.
Borman, W. C., & Motowidlo, S. J. (1997). Task performance and contextual performance: The
meaning for personnel selection research. Human Performance, 10, 99–109.
Campbell, J. P. (1991). Modeling the performance prediction problem. In M. Dunnette & L. Hough
(Eds.), Handbook of industrial/organizational psychology (Vol. 2, pp. 687–732). Palo Alto, CA:
Consulting Psychologists Press.
Campbell, J. P. (1994). Alternative models of job performance and their implications for selection
and classification. In M. G. Rumsey & C. B. Walker (Eds.), Personnel selection and classification
(pp. 33–51). Hillsdale, NJ: Erlbaum.
Campbell, J. P., McHenry, J. J., & Wise, L. L. (1990). Modeling job performance in a population
of jobs. Personnel Psychology, 43, 313–333.
Carroll, J. D., & Chang, J.-J. (1970). Analysis of individual differences in multidimensional scaling
via an n-way generalization of “Eckart-Young” decomposition. Psychometrika, 35, 283–319.
Cattell, R. B. (1944). “Parallel proportional profiles” and other principles for determining the choice
of factors by rotation. Psychometrika, 9, 267–283.
Cattell, R. B (1952). The three basic factor analytic research designs: Their interrelations and
derivatives. Psychological Bulletin, 49, 499–520.
Chan, D. (1998). The conceptualization and analysis of change over time: An integrative approach
incorporating longitudinal mean and covariance structures analysis (LMACs) and multiple indi-
cator latent growth model (MLGM). Organizational Research Methods, 14, 421–483.
Cleveland, W. S., & Devlin, S. J. (1988). Locally weighted regression: An approach to regression
analysis by local fitting. Journal of the American Statistical Association, 83, 596–610.
Csikszentmihalyi, M., & Larson, R. W. (1987). Validity and reliability of the experience sampling
method. Journal of Nervous and Mental Disease, 175, 526–536.
Dalal, R. S., & Hulin, C. L. (2008). Motivation in organizations: Criteria and dynamics. In R. Kanfer,
G. Chen, & R. Pritchard (Eds.), Work motivation: Past, present, and future. (pp. 63–100). New
York: Routledge.
Dalal, R. S., Lam, H., Weiss, H. M., Welch, E. R., & Hulin, C. L. (2009). A within-person ap-
proach to work behavior and performance: Concurrent and lagged citizenship-counterproductivity
associations, and dynamic relationships with affect and overall job performance. Academy of
Management Journal, 52, 1051–1066.
Dunnette, M. D. (1963). A note on the criterion. Journal of Applied Psychology, 47, 251–254.
Ghiselli, E. E. (1956). Dimensional problems of criteria. Journal of Applied Psychology, 40,
1–4.
Goldberg, L. R. (2001). International Personality Item Pool: A scientific collaboratory for the
development of advanced measures of personality and other individual differences. Available
from http://ipip.ori.org/ipip/ipip.html
Harshman, R. A. (1970). Foundations of the Parafac procedure: Models and conditions for an “ex-
planatory” multimodal factor analysis. UCLA Working Papers in Linguistics, 16, 1–84. (University
Microfilms, Ann Arbor, MI, No. 10,085)
Harshman, R. A., & Lundy, M. E. (1984a). The PARAFAC model for three-way factor analysis and
multidimensional scaling. In H. G. Law, C. W. Snyder, J. A. Hattie, & R. P. McDonald (Eds.),
Research methods for multimode analysis (pp. 122–215). New York: Praeger.
Harshman, R. A., & Lundy, M. E. (1984b). Data preprocessing and the extended PARAFAC model.
In H. G. Law, C. W. Snyder, J. A. Hattie, & R. P. McDonald (Eds.), Research methods for
multimode analysis (pp. 216–284). New York: Praeger.
Hulin, C. L. (1982). Some reflections on general performance dimensions and halo rating error.
Journal of Applied Psychology, 67, 165–170.
Hulin, C. L., Henry, R. A., & Noon, S. L. (1990). Adding a dimension: Time as a factor in the
generalizability of predictive relationships. Psychological Bulletin, 107, 328–340.
Humphreys, L. G. (1976). The phenomena are ubiquitous-but the investigator must look. Journal of
Educational Psychology, 68, 521.
Inn, A., Hulin, C. L., & Tucker, L. R. (1972). Three sources of criterion variance: Static dimen-
sionality, dynamic dimensionality, and individual dimensionality. Organizational Behavior and
Human Performance, 8, 58–83.
Keil, C. T., & Cortina, J. M. (2001). Degradation of validity over time: A test and extension of
Ackerman’s model. Psychological Bulletin, 127, 673–697.
Kiers, H. A. L. (1991). Hierarchical relations among three-way methods. Psychometrika, 56, 449–
470.
Kiers, H. A. L. (2000). Toward a standardized notation and terminology in multiway analysis.
Journal of Chemometrics, 14, 105–122.
Kroonenberg, P. M. (1994). The TUCKALS line: A suite of programs for the analysis of three-way
data. Computational Statistics and Data Analysis, 18, 73–96.
Kroonenberg, P. M. (2008). Applied multiway data analysis. Hoboken, NJ: Wiley.
Kroonenberg, P. M., & de Leeuw, J. (1980). Principal component analysis of three-mode data by
means of alternating least squares algorithms. Psychometrika, 45, 69–97.
Kroonenberg, P. M., & Oort, F. J. (2003). Three-mode analysis of multimode covariance matrices.
British Journal of Mathematical and Statistical Psychology, 56, 305–335.
Larson, R., & Csikszentmihalyi, M. (1983). The experience sampling method. In H. T. Reis (Ed.),
Naturalistic approaches to studying social interaction (pp. 41–56). San Francisco: Jossey-Bass.
Miner, A. G., Glomb, T., & Hulin, C. L. (2005). Experience sampling mood and its correlates at
work. Journal of Occupational and Organizational Psychology, 78, 171–193.
Miner, A. G., Glomb, T., & Hulin, C. L. (2010). State mood, task performance, and behavior at
work: A within–persons approach. Organizational Behavior and Human Decision Processes, 112,
43–57.
Motowidlo, S. J., & Van Scotter, J. R. (1994). Evidence that task performance should be distinguished
from contextual performance. Journal of Applied Psychology, 79, 475–480.
Murphy, K. R. (1989). Is the relationship between cognitive ability and job performance stable over
time? Human Performance, 2, 183–200.
Murphy, K. R., & Shiarella, A. H. (1997). Implications of the multidimensional nature of job
performance for the validity of selection tests: Multivariate frameworks for studying test validity.
Personnel Psychology, 50, 823–854.
Salovey, P., Mayer, J. D., Goldman, S. L., Turvey, C., & Palfai, T. P. (1995). Emotional attention,
clarity, and repair: Exploring emotional intelligence using the Trait Meta-Mood Scale. In J. W.
Pennebaker (Ed.), Emotion, disclosure, and health (pp. 125–154). Washington, DC: American
Psychological Association.
Smilde, A., Bro, R., & Geladi, P. (2004). Multiway analysis: Applications in the chemical sciences.
Hoboken, NJ: Wiley.
Smith, P. D. (1976). Behaviors, results, and organizational effectiveness: The problem of criteria. In
M. Dunnette (Ed.), Handbook of industrial and organization psychology (pp. 745–775). Chicago:
Rand McNally.
Smith, P. C., Kendall, L., & Hulin, C. L. (1969). The measurement of satisfaction in work and
retirement. Chicago: Rand McNally.
Thoresen, C. J., Bradley, J. C., Bliese, P. D., & Thoresen, J. D. (2004). The big five personality
traits and individual job performance growth trajectories in maintenance and transitional job
stages. Journal of Applied Psychology, 89, 835–853.
Timmerman, M. E., & Kiers, H. A. L. (2000). Three-mode principal components analysis: Choosing
the number of components and sensitivity to local optima. British Journal of Mathematical and
Statistical Psychology, 53, 1–16.
Tucker, L. R. (1964). The extension of factor analysis to three-dimensional matrices. In H. Gullikson
& N. Frederikson (Eds.), Contributions to mathematical psychology (pp. 110–127). New York:
Holt, Rinehart, & Winston.
Tucker, L. R. (1966). Some mathematical notes on three-mode factor analysis. Psychometrika, 31,
279–311.
Weiss, H. M., & Cropanzano, R. (1996). Affective events theory: A theoretical discussion of the
structure, causes and consequences of affective experiences at work. In B. M. Staw & L. L.
Cummings (Eds.), Research in organizational behavior (Vol. 18, pp. 1–74). Greenwich, CT: JAI
Press.
Weiss, H. M., Nicholas, J. P., & Daus, C. S. (1999). An examination of the joint effects of affective
experiences and job satisfaction and variations in affective experiences over time. Organizational
Behavior and Human Decision Processes, 78, 1–24.
Van Mechelen, I., & Kiers, H. A. L. (1999). Individual differences in anxiety response to stressful
situations: A three-way component analysis model. European Journal of Personality, 13, 409–
428.
Zyphur, M. J., Bradley, J. C., Landis, R. S., & Thoresen, C. J. (2008). The effects of cognitive
ability and conscientiousness on performance over time: A censored latent growth model. Human
Performance, 21, 1–27.

TMP D4 C1

Enviado por

Dados do documento

Descrição original:

Título original

Direitos autorais

Formatos disponíveis

Compartilhar este documento

Compartilhar ou incorporar documento

Opções de compartilhamento

Você considera este documento útil?

Este conteúdo é inapropriado?

Direitos autorais:

Formatos disponíveis

TMP D4 C1

Enviado por

Direitos autorais:

Formatos disponíveis

This article was downloaded by: [Spain, Seth M.

Multivariate Behavioral Research

Job Performance as Multivariate Dynamic Criteria: Experience Sampling

Online publication date: 19 August 2010

PLEASE SCROLL DOWN FOR ARTICLE

Full terms and conditions of use: http://www.informaworld.com/terms-and-conditions-of-access.pdf

Job Performance as Multivariate

and Multiway Component Analysis

Correspondence concerning this article should be addressed to Seth M. Spain, Department of

Of central interest to applied psychologists is how to define job performance,

The techniques presented in this article are methods of multiway component

THE THREE SOURCES OF JOB PERFORMANCE

FIGURE 1 The Basic Data Relations Box (Cattell, 1952).

ability measures to predict individual growth curves for performance criteria

to the client. This act of emotion regulation uses up some of my regulatory

longitudinal/experience sampling study would have such a matrix. Stacking

xijk D †p †q †r Œaip bjq ckr gpqr C eijk : (3)

analysis (PCA) of the wide combination-mode matricization (e.g., I JK) of

SSmod(PCA)  SSmod (Tucker3)  SSmod (Parafac) (4)

provided all models have equivalent numbers of components (Kiers, 1991).

Data Source and Participants

Measures and Procedures

Experience sampling measures. Participants answered questions about

performance, organizational citizenship behavior, and work withdrawal. Par-

Individual difference measures. Additionally, participants completed sev-

Description of the Experience Sampling Data Array

zijk D .xijk x jk /=sj ; (5)

1 Rated average handle time 1.91 0.74 1.00

Parafac analyses. We fit a two-component Parafac model using multi-

Tucker3 analyses. Figure 5 displays the time components from the

Rated overall service 0.865

Neutral behavior 0.837

Note. Values smaller than .20 are not shown. Variables

PF2 .098 .272

FIGURE 5 Time component loading of the 3 2 2 Tucker3 solution after rotation of

Call wait time 0.458 0.109 0.069 0.073

Variable Objective Behavioral

Rated average handle time 0.159 0.115

Focal location 0.098 0.323

Note. Variables are ordered to best show component

Participant 1 1.437 0.216 0.188 0.004

Participant 1 0.137 0.173 0.002 0.003

The Tucker3 model provides a parsimonious explanation of the observed data,

Implications for the Analysis of Experience

multivariate dynamic criterion space. As it stands, it is very interesting that

Limitations and Future Directions

and within-person sources of lawful variance be considered in such endeavors.

Você também pode gostar

xijk D †p †q †r Œaip bjq ckr gpqr C eijk : (3)

SSmod(PCA) SSmod (Tucker3) SSmod (Parafac) (4)