Chapter

10
Factor Analysis and Structural Equa9ons
Modeling
 Outline

Exploratory Factor Analysis

Conrmatory Factor Analysis (CFA)

Structural Equa9ons Modeling

 Basics
Types of Factor Analysis (FA): (1) Exploratory factor analysis and (2)
Conrmatory Factor Analysis.
The Exploratory Factor Analysis (EFA), is a data-driven process that
explores the structure in the data. It does not (necessarily) start with
the theory and does not have specied hypothesis-tes9ng mechanism.
The Conrmatory Factor Analysis (CFA) is a theory-driven process that
aJempts to test whether the given data ts to a theore9cal model.
Now the CFA is considered as a part of the broad framework of
structural equa9on modeling (SEM).
The Structural Equa9on Modeling (SEM) is an important alterna9ve to
carry out predic9ve analysis with variables that are not observable
(latent variables).
 Exploratory Factor Analysis
Data reduc9on technique.
Cross-sec9onal data.
Iden9fy variables in the data that form theore9cally
meaningful subgroups.
Independence of the subgroups.
EFA is a group method that explains varia9on
among the variables in terms of fundamental
en99es called factors.
Factor is dieren9ally weighted linear combina9on
of variables (it is unobserved or latent).
 Exploratory Factor Analysis
Correla9on matrix of six personality variables (Belhekar, 2008)
Variabl
es E1 E2 E6 N3 N4 N6
E1 1 0.493 0.523 0.091 0.08 0.011
E2 0.493 1 0.358 0.05 0.048 0.051
E6 0.523 0.358 1 0.059 0.006 0.023
N3 0.091 0.05 0.059 1 0.614 0.595
N4 0.08 0.048 0.006 0.614 1 0.523
N6 0.011 0.051 0.023 0.595 0.523 1
 Exploratory Factor Analysis
Correla9on matrix of six personality variables (Belhekar, 2008)
Variable
s E1 E2 E6 N3 N4 N6
E1 1 0.493 0.523 0.091 0.08 0.011
E2 0.493 1 0.358 0.05 0.048 0.051
E6 0.523 0.358 1 0.059 0.006 0.023
N3 0.091 0.05 0.059 1 0.614 0.595
N4 0.08 0.048 0.006 0.614 1 0.523
N6 0.011 0.051 0.023 0.595 0.523 1
 Steps in
Exploratory Factor Analysis

Model
Data and Rota9on Interpret
Input to Number Chose
Collect Data Extrac9on of Factors Interpret
on Large FA Method Rota9on rotated
Determine Method Factor
Number of Correla9on Select Number of (Orthogonal
Variables Matrix or Method of or Oblique)
PaJern/
(k) and on Factors for
Covariance Extrac9on: and Rotate Factor
Large Rota9on (r
Matrix PCA or Selected Structure
Sample < k)
Common Factors matrix
Factor
 Factor Analysis Jargon
Correla9on matrix (or covariance matrix)
Extrac9on: Process of transforming variance from a
correla9on matrix into factors or components.
Communali9es: Es9mates of shared variance.
Rota9on: Transforming selected factors to make them
interpretable.
Factor PaUern matrix: Variables by factors matrix;
elements are loadings (weights).
Factor Structure Matrix: Variables by factors matrix;
elements are correla9ons.
Phi matrix: Inter-factor correla9ons matrix.
 Purpose of Exploratory Factor
Analysis
Understanding the number of factors that can
describe the exis9ng correla9on matrix.
Understanding the theore9cal meaning of those
factors.
The EFA can be carried out to evaluate the theory
(CFA is a beJer op9on than EFA).
EFA factors can be saved and used as variables in
analyses.
 Specic Issues in EFA
Sample size
Normality, linearity, outliers, and mul9collinearity
Factorability of correla9on matrix
BartleJs test
2k + 5
2 = n 1 ln R
6

KaiserMeyerOlkin (KMO) n
rij2
i j i
KMO = n n

r
i j i
2
ij + aij2
i j i
 Basic Equa9ons
PCA Model
z j = a j1F1 + a j 2 F2 + + a jk Fk for, j = 1, 2,..., k

where,
z j is standardized form of variable j
F1,F2 ,...,Fk are k unrelated components such that each component one after
another that makes maximum contribution to variance of k variables;
a j1 is a weight for the variable j for the factor F1, and so on
 Basic Equa9ons
Common Factor MZodel
z j = a j1F1 + a j 2 F2 + + a jm Fm + d jU j for, j = 1, 2,..., k

where,
m < k that is common factors,
U j is unique j factors,
d j is weight associated with U j
 Variance Component
If factors are uncorrelated, then
S j = 1 = a j1 + a j 2 + + a jm + d j
2 2 2 2 2

Communality of a variable
h 2j = a 2j1 + a 2j 2 + + a 2jm for, j = 1, 2,..., k
Unique factors can be decomposed into
z = a F + a F + + a F + b S + e E for, j = 1, 2,..., k
j j1 1 j2 2 jm m j j j j

Sj and Ej are specic and error factors respec9vely

d 2j = b 2j + e 2j
 Variance Component

S = 1 = a + a + + a + b + e
2
j
2
j1
2
j2
2
jm
2
j
2
j

S =1= h +b + e
2
j
2
j
2
j
2
j

variance of z j = 1 = communality of j + Specificity of j + unrealibility of j

rj = h 2j + b 2j
Matrix Equa9ons for EFA
R kk correlation matrix among the variables
obtained from Z nk data matrix

L = V RV
Vkk eigenvectors
L principal diagonal of L are the eigenvalues

factor pattern is
A=V L
 Extrac9on Models
Component Model
o Keeps uni9es in the diagonal of correla9on matrix
o Maximum variance
Common Factor Model
o Uses some es9mate of common factors in the diagonal of
correla9on matrix.
o Only common variance is condensed.
o Common factors are those which contribute to two or more
variables.
o Es9mates of communality are:
(i) highest correla9on, (ii) reliability coecients, (iii)
squared mul9ple correla9on (SMC), (iv) itera9ve methods.
 Methods of Extrac9on
Principal Component Analysis (PCA)
Extract maximum variance
HoJeling (1933)
Full model is seldom used.
Truncated component solu9on is usually used.
Accounts for maximum available variance.
Orthogonal (uncorrelated, independent) with all prior
factors.
Full solu9on (as many factors as variables), that is, accounts
for all the variance.
 Methods of Extrac9on
Principal Axis Factoring (PAF)
Common factor model
Uses an es9mate of communali9es.
PAF extracts smaller variance than PCA.
SMC is used as star9ng values and communali9es
are es9mated using an itera9ve process.
Factors are latent variables and have causal
meaning associated with them.
 Methods of Extrac9on
Maximum Likelihood
Extrac9on method based on sta9s9cal considera9ons.
Lawley and Maxwell, and further rened by Jreskog (1967).
Logic: Es9mate popula9on values of factor loading that
maximizes the likelihood of obtaining the sample correla9on
matrix from popula9on.
Chi-square sta9s9cs used to assess goodness-of-t of a model
to the data
ML extrac9ons are over-factoriza9on.
 Methods of Extrac9on
Minimum Residual Analysis (MinRes)
MinRes minimizes the residuals
sum-of-squares o-diagonal loadings are smaller
Unweighted Least Squares
Minimizes the squared dierence between the observed
correla9on matrix and the reproduced correla9on matrix
Generalized (weighted) Least Squares
GLS uses weights for variables while compu9ng dierence
Image Factoring
Distributes the variance of the variable reected by other variables
into factors
Alpha Factoring
Alpha reliability of the factors is maximized
 Number of Factors
Full solu9on not useful
Truncated solu9on required
First few components/factors are retained
Eigenvalues
Percentage of variance
Eigenvalue
Percentage of variance = 100
k
How to decide number of factors?
Methods to Deciding Number
of Factors
GuJmanns eigenvalue above one criteria
Scree plot
Parallel analysis and eigenvalue larger than Monte Carlo
eigenvalues
Percentage of variance
The sta9s9cal test
Use of guiding theory
Interpretability of dierent solu9on
Why is Rota9on Required?

Extrac9on of ar9fact and redistribute the

variance

To obtain a simple structure

Theore9cal interpretability
 What is Rota9on?
Figure'10.3'A:'The'Unrotated'Solution' Figure'10.3'B:'Graphically'rotated'
Solution'

' '
'
 Types of Rota9on
Two Types: Orthogonal and Oblique
Orthogonal: Uncorrelated Factors
Oblique: Correlated Factors
Orthogonal Methods
Quar9max
Varimax
Transvarimax
Equamax
Parsimax
Oblique Rota9ons
Oblimin
Oblimax,
Promax
 Factor Structure and Factor
PaUern
The factor paJern matrix (A) is a k ! r matrix.
=
C = A
Factor Scores
B = R -1A
F = ZB
Conrmatory Factor Analysis
CFA is a special case of structural equa9ons model
(SEM).
SEM has two types of models: one, measurement
model and two, full SEM model.
CFA is a measurement model of SEM
Falsica9on of a hypothesis
Tests of hypothesis
CFA in that sense is a scien9c method
Conrmatory Factor Analysis
Basic Idea
Measured variable (also called as observed variables,
manifest variables or indicators) are imperfect indicators
of latent variables.
Latent variable is an unobservable, underlying construct
that causes the observable and measured variables (also
called as manifest variables).
Observed variable is known as the imperfect indicator of
the latent variables.
For example, the intelligence test score (observed) is the
func9on of ability intelligence (latent).
Observed Variable = Latent Variable + Random Error

 Steps in CFA
(i) Have theory
(ii) Get data
(iii) Specify model
(iv) Test for iden9ca9on
(v) Es9mate model parameters
(vi) Sta9s9cal test and Fit indices
(vii) Compare dierent models
(viii) Interpret and conclude
 The Graphical Model
Latent Varibale Observed Variable Error

Meaning of the CFA Symbols
Sign% Description%% Represents%
! Circle! Latent!variable!(unobservable,!not!
! measured)!
!
! Square!! Observed!variable!(manifest!variable!or!
! Indicator).!Directly!measured.!
!
! Straight!Arrow! Causal!Direction!or!Influence!path!
! Curved!double! Correlation/Covariance!between!
! headed!arrow! exogenous!(latent)!variables!or!errors.!!
!
!
 CFA Model

X = +
X is observed variables
matrix of structural coefficients associated with each
of the manifest variables for the .
latent variables
error associated with each of the manifest variables
Matrix representa9on
X1 11 0 1

X2 21 0 2
X3 31 0 1 3
= +
X4 0 42 2 4

X5 0 52 5
X6 0 62 6

X 61 = 62 21 + 61
 Assump9ons
(A)$ E ( X ) = E ( ) = 0 .$The$mean$of$the$observed$and$latent$variables$is$zero.$$
(B)$The$relationships$between$the$observed$ ( X ) and$latent$variables$ ( ) is$$linear.$$
(C)$There$are$assumptions$about$measurement$error:$$
$ (i)$ E ( ) = 0 .$The$errors$have$mean$zero.$
$ (ii)$The$errors$have$constant$variance$across$observations$
$ (iii)$The$errors$are$independent$across$observations$
(iv)$ E ( ) = E ( ) = 0 $.$The$errors$are$uncorrelated$with$latent$variables$
$
 Unrestricted and Model-implied
Covariance Matrix
Sample variance covariance matrix (S)
Popula9on variancecovariance matrix of the observed
variables (). Also called as unrestricted variancecovariance
matrix
= (XX )
( ) = XX

( ) = ( + )( + )

( ) = ( ) + E ( ) + E ( ) + E ( )
( ) = ( ) + E ( )
 Iden9ca9on
Iden9ca9on problem for the CFA model is whether
unique solu9on exists for each parameter.
Nonredundant elements of correla9on matrix
c = k ( k + 1) 2
Necessary but not a sucient condi9on for
iden9ca9on is that the number of parameters
es9mated, p, are smaller than or equal to c.
Bollen (1989) discussed three indicators rule and
two indicators rule for the iden9ca9on as a
sucient condi9on.
 Es9ma9on
( ) model implied variance covariance matrix
The maximum likelihoodestimation ( ML )
FML = log ( ) + tr S -1 ( ) log S p
Unweighted Least Squares (ULS )

FULS
1
{
= tr S ( )
2
2
}
Generalized Least Squares ( GLS )

FGLS
1
{
= tr I ( ) S
2
-1 2
}
 Model Evalua9on
Evalua9on of Null hypothesis
H 0 : = ( )
H A : ( )

2 = ( n 1) F S ,
( )
Chi-square test is a very sensi9ve test.
Signicance of chi-square is not used.
So t measures are used.
 Fit Measures
Goodness-of-Fit (GFI) and Adjusted Goodness-of-
Fit Index (AGFI)
(
tr S I

) ()
2
1
F S ,

GFI = 1 = 1
F S , ( 0 )
tr S

( )
2
1

c
AGFI = 1 (1 GFI )
df h
 Normed Fit Index (NFI) and
Non-normed Fit Index (NNFI)

i2 h2 Fh
NFI = = 1
i2
Fi

NNFI =
( i
2
dfi ) ( df h )
2
h

dfi 1
i
2
Compara9ve Fit Index (CFI)

lh
FI = 1
li
l1
CFI = 1
l2
Parsimonious Fit Indices

df h
PGFI = GFI
df n
df h
PNFI = NFI
dfi
 Comparing Models

AIC ( )
LR = 2 log L R log L U
( )
Likelihood Ra9o Test (LR)
LR = R U
2 2

Wald Test (W) 12

W=
( )
avar 1

Lagrange Mul9plier test (LM Test)

( ) ( ) ( )
LM = S R I R S R

Structural Equa9ons Modeling
(SEM)
Model
= + + !
is#a# m 1 vector#of#endogenous#variable,# is#a# n 1 vector#latent#exogenous#
variable,# is#a# m m coefficients#indicating#influence#of#latent#exogenous#variables#
on#eachother,### is#a# m n coefficients#matrix#of#effects#of# on# ,# is#a#vector#
containg#disturbance#term.#The# E ( ) = 0 and# is#uncorrelated#with# .##
The#measuremnt#model#specified#by#SEM#are#as#follows:#
#
Y = + ! Y

X = X + !
 Steps
The Model Specica9on

Iden9ca9on

Obtaining Data

Assessing Modeldata Fit (Chi-square)

Using Fit indices