European

V. E.Journal
Caballo
of et
Psychological
al.: Measuring
Assessment
2010
Social
Hogrefe
Anxiety
2010;&Vol.
Huber
in 11
26(2):95107
Co
Publishers
untries

Original Article

Measuring Social Anxiety

in 11 Countries
Development and Validation of the
Social Anxiety Questionnaire for Adults
Vicente E. Caballo1, Isabel C. Salazar2, Mara Jess Irurtia3, Benito Arias3,
Stefan G. Hofmann4, and the CISO-A Research Team
1

University of Granada, Spain, 2Pontificia Javeriana University at Cali, Colombia,

3
University of Valladolid, Spain, 4Boston University, Boston, MA, USA

Abstract. This paper reports on two studies conducted to develop and validate a new self-report measure of social phobia/anxiety the
Social Anxiety Questionnaire for Adults (SAQ-A) (Cuestionario de ansiedad social para adultos, CASO-A). A diary-item recording
procedure was used to generate the initial pool of items. In Study 1, data from 12,144 participants provided 6 factors with moderate
intercorrelations. Estimates of internal consistency reliability were adequate (range = .86 to .92) for the 6 factors included in the final
confirmatory factor analysis. In Study 2, data provided by 10,118 nonclinical participants were used to explore preliminary reliability
and validity estimates for a revised version of the SAQ-A the Social Anxiety Questionnaire for Adults Revised (SAQ-AR). Approximately 106 researchers from 10 Latin American countries and Spain contributed to this data collection process. Specific comments are
made on the structure of the new questionnaire as regards some commonly-used self-report measures of social phobia/anxiety.
Keywords: social anxiety, social phobia, SAQ-AR, self-report measures, cross-cultural research

Introduction
Once described as a neglected disorder (Liebowitz, 1987),
social anxiety has attracted a great deal of research interest
among psychiatrists and psychologists alike over the past
two decades. Several measures (interviews and inventories) have been developed to tap the social anxiety construct, including the Liebowitz Social Anxiety Scale
(LSAS; Liebowitz, 1987), the Social Phobia and Anxiety
Inventory (SPAI; Turner, Beidel, Dancu, & Stanley, 1989),
the Brief Social Phobia Scale (BSPS; Davidson et al.,
1991), the Social Phobia Scale (SPS; Mattick & Clarke,
1998), the Social Interaction Anxiety Scale (SIAS; Mattick
& Clarke, 1998), the Self-Statements During Public Speaking Scale (Hofmann & DiBartolo, 2000), and the Social
Phobia Inventory (SPIN; Connor et al., 2000). In addition,
a number of older, but still popular, scales exist, such as the
Fear of Negative Evaluation (FNE) and Social Avoidance
and Distress (SAD) Scales (Watson & Friend, 1969).
Although frequently used to assess social anxiety in clinical and research settings, the existing instruments have a
number of limitations. First, items from most of these instruments were not empirically derived. For instance, the
2010 Hogrefe & Huber Publishers

items on the Social Phobia Scale (SPS) and the Social Interaction Anxiety Scale (SIAS) were subjectively derived
mainly from an initial pool of statements comprising 164
items, which themselves were derivatives of other existing
fear survey schedules and social anxiety inventories (Mattick & Clarke, 1998). For example, the Social Phobia and
Anxiety Inventorys (SPAI) initial item pool was generated
by the authors after reviewing available inventories and
DSM-III criteria for social phobia (APA, 1980), and by
compiling a list of complaints from a patient population
(Turner, Beidel et al., 1989). The Social Phobia Inventory
(SPIN; Connor et al., 2000) was based and modeled on a
former inventory, the Brief Social Phobia Scale (BSPS; Davidson et al., 1991), and the Liebowitz Social Anxiety
Scale (LSAS) was generated by the author of the instrument (Liebowitz, 1987). Most of the items in these questionnaires corresponded to those in already existing surveys, and issues regarding content validity were not addressed in greater detail (see Haynes, Richard, & Kubany,
1995, about the importance of this issue).
Not surprisingly, the most popular instruments vary considerably in the number and type of factors underlying the
social anxiety construct; in other words, their factor strucEuropean Journal of Psychological Assessment 2010; Vol. 26(2):95107
DOI: 10.1027/1015-5759/a000014

96

V. E. Caballo et al.: Measuring Social Anxiety in 11 Countries

tures appear far from robust. In the case of the LSAS, some
authors have identified four factors (Safren et al., 1999;
Slavkin, Holt, Heimberg, Jaccard, & Liebowitz, 1990),
whereas others have found a 5-factor solution (Baker,
Heinrichs, Kim, & Hofmann, 2002). An additional problem
is that not only the number, but the general content of the
factors differ across studies. Similar inconsistent findings
in factor solutions have been reported for other social anxiety/phobia measures, such as the SPIN (Antony, Coons,
McCabe, Ashbaugh, & Swinson, 2006; Connor et al., 2000;
Johnson, Inderbitzen-Nolan, & Anderson, 2006; Radomsky et al., 2006), the SPAI (Olivares, Garcia-Lopez, Hidalgo, Turner, & Beidel, 1999; Osman, Barrios, Aukes, & Osman, 1995; Turner, Stanley, Beidel, & Bond, 1989), and the
SAD and FNE (Hofmann, DiBartolo, Holaway, & Heimberg, 2004; Olivares, Garca-Lpez, Hidalgo, 2004;
Turner, McCanna, & Beidel, 1987).
In addition to the aforementioned methodological problems with the nonobjective method of social anxiety scale
development is the fact that all of the above measures were
created exclusively for English speakers, primarily from
North America and Australia. The use of these instruments
with Spanish-speaking samples usually involves a somewhat simplistic direct translation of the questionnaires from
English to Spanish (e.g., Olivares et al., 1999, 2004). Unfortunately, this procedure ignores cultural differences in
the expression of social anxiety and social norms (Heinrichs et al., 2006). This is rather ironic when one considers
that social interaction styles and norms are probably among
the most important defining features of a culture and are
often precisely the locus of differences across cultures.
Thus, it remains to be seen whether a questionnaire that
describes a variety of social situations is applicable across
cultures. To address the cultural and methodological limitations of the existing literature, we conducted an extensive
series of studies in order to develop a new social anxiety
questionnaire, without directly relying on items from existing self-report instruments. In contrast to existing measures, we developed the instrument based on items generated by large and very diverse Spanish and Portuguese
speaking samples.

Study 1: Development of the

Initial Scale
Method
Initial Item Selection
For 3 months per year over a period of 6 years, volunteer
students from the Department of Psychology at the University of Granada (Spain), along with their volunteer family
members, partners, and friends, were asked to keep a diary
of social situations that elicited some degree of anxiety,
nervousness, uneasiness, fear, or stress. Several examples
European Journal of Psychological Assessment 2010; Vol. 26(2):95107

were given to students, who in turn had to explain the task

to their significant others, who also kept such a diary. Different students took part each year and the situations only
had to be recorded if they directly affected the participants.
It should be noted that the University of Granada teaches
students from all over Spain. Furthermore, the 3 months of
data collection included periods during the regular academic year as well as holidays (Christmas). Accordingly, a variety of different situations from people varying greatly in
age, schooling, and geographical origin were generated by
these diaries.
More than 1,000 participants recorded situations over 6
years, generating a pool of more than 10,000 social situations. From these, two pairs of social anxiety experts selected scenarios for initial analysis, excluding those situations that were redundant or were not social in nature (i.e.,
another person[s] played a role in the situation). This left
2,171 scenarios, which were then grouped together based
on substantive similarity, leaving a total of 512 social situations.

Scale Construction
The experts then paraphrased the 512 social situations into
items. Four additional situations that typically produce
great distress were also selected (stressful life events, such
as suffering an armed attack) and added to control response biases. These 516 items formed the Social Anxiety
Questionnaire for Adults (SAQ-A) (Cuestionario de Ansiedad Social para Adultos; CASO-A), the initial version
of a new self-report instrument intended to assess social
anxiety. The items were randomly ranked and each item
could be answered on a seven-point Likert scale to indicate
the level of uneasiness, stress or nervousness in response
to each situation: 0 = not at all, 1 = very slight, 2 = slight,
3 = moderate, 4 = high, 5 = very high, and 6 = extremely
high. Instructions given to those completing the scale were
as follows:
There follows a series of social situations that may cause you
unease, stress or nervousness to a lesser or greater extent.
Please place an X on the number that best reflects your reaction. If you have never experienced the situation described,
please imagine what your level of unease, stress, or nervousness might be if you were in that situation, placing an X on
the corresponding number.

Several blank lines were included at the end of the answer

sheet for participants filling out the questionnaire to add
more social situations if they wanted to do so.

Participating Countries and Researchers

A large number of potential collaborators were contacted
via e-mail and asked to assist in conducting the study. A
total of 106 research collaborators from 10 Latin American
2010 Hogrefe & Huber Publishers

V. E. Caballo et al.: Measuring Social Anxiety in 11 Countries

97

Table 1. Distribution of subjects by country in Study 1 (SAQ-A) and Study 2 (SAQ-AR)

Participant subjects by country in the first study with the SAQ-A
Women

Men

Country

Mean age
(SD)

Mean age
(SD)

Argentina

,499

30.25
(10.89)

,378

29.82
(11.42)

Brazil

,702

26.07
(9.48)

,547

Chile

,376

26.90
(10.86)

Colombia

,852

Costa Rica
Spain
Mexico
Paraguay

Participant subjects in the second study with the SAQ-AR

All subjects
N

Mean age
(SD)

Women

Men

All subjects

Mean age
(SD)

Mean age
(SD)

Mean
age (SD)

,877 30.05
(11.11)

,329

23.38
(5.42)

,348

24.77
(8.53)

,677 24.09
(1.56)

27.55
(10.79)

1,249 26.76
(10.12)

,405

31.04
(13.06)

,358

30.12
(11.39)

,763 30.61
(12.30)

,308

27.91
(11.52)

, 684 27.36
(11.16)

,310

26.76
(11.65)

,297

26.53
(10.83)

,607 26.64
(11.25)

24,70
(9.60)

,774

25,47
(9.81)

1,626 25.21
(9.78)

,870

26,11
(11.98)

,857

27,80
(13.00)

1,727 26.96
(12.53)

,205

23.23
(9.42)

,122

18.87
(5.82)

,327 21.58
(8.51)

,363

25.87
(9.10)

,186

25.35
(9.68)

,549 25.69
(9.29)

,905

22.80
(8.80)

,668

27.01
(12.00)

1,573 24.58
(10.48)

1,335

23.24
(8.66)

,907

26.21
(11.41)

2,242 24.44
(9.97)

2,377

25.14
(10.34)

1,954

25.29
(9.68)

4,331 25.22
(10.05)

1,258

25.25
(12.16)

1,128

25.55
(9.93)

2,386 25.39
(11.16)

,91

24.62
(8.03)

,77

21.91
(6.82)

,168 23.27
(7.57)

,100

22.48
(5.83)

,100

21.85
(5.85)

,200 22.16
(5.83)

1,002

23.08
(8.37)

,978

23.25
(8.00)

1,980 23.16
(8.18)

,529

21.27
(6.33)

,497

21.71
(6.84)

1,026 21.49
(6.58)

Uruguay

,101

32.39
(12.27)

,100

33.43
(10.93)

,201 32.92
(11.60)

,135

31.30
(12.78)

,114

34.29
(13.11)

Venezuela

,195

27.53
(11.91)

,186

25.56
(9.73)

,381 26.52
(10.88)

,301

19.77
(4.12)

,299

20.53
(4.57)

7,271

25.15
(10.05)

6,126

25.75
(10.22)

13,397 25.43
(10.13)

5,935

24.79
(10.51)

5,091

25.81
(10.74)

Peru

All countries

countries and Spain agreed to participate in data collection.

Most worked at academic institutions, and some worked in
private clinical service centers. The distribution by country
of researchers (and research groups) was as follows: Argentina = 16 collaborators (6 groups of researchers); Brazil
= 7 collaborators (5 groups of researchers); Chile = 7 (3
groups of researchers); Colombia = 16 (8 groups of researchers); Costa Rica = 1 (1 group of researchers); Spain
= 10 (8 groups of researchers); Mexico = 35 (22 groups of
researchers); Paraguay = 3 (1 group of researchers); Peru
= 8 (8 groups of researchers); Uruguay = 2 (1 group of
researchers); and Venezuela = 1 (1 group of researchers).

,249 32.67
(12.99)
600

20.15
(4.36)

11,026 25.65
(10.63)

year and five months. Collaborators used a prepared database in Excel to enter the data.

Participating Subjects

The SAQ-A was sent to each collaborator with a request to

suggest changes in the wording of the items to be more
consistent with the specific language style of their culture.
The questionnaires were also completed by several students in each country to evaluate whether the wording of
the items was correct. In order to derive the Portuguese
version, the SAQ-A was translated and backtranslated from
Portuguese to Spanish until agreement was reached between translators. Data was collected over the course of one

An initial pool of 13,397 participants completed the SAQA (mean age = 25.43; SD = 10.13) (see Table 1 for the
distribution of participating subjects by country). Approximately half (7,271) were women (mean age = 25.15; SD
= 10.05), and 6,126 were men (mean age = 25.76; SD =
10.22). The minimum age for subjects was 16 years. With
regard to age distribution, 5,420 (40.4%) subjects were
younger than 20 years old, 3,029 (22.7%) were between the
ages of 20 and 24, 1674 (12.49) were between 25 and 30,
2225 (16.61) were between 31 and 50, and 1,049 (7.83%)
were 51 years or older. The participants had different levels
of education (students, workers, etc.). Specifically, 17.6%
were university psychology students, 40.6% were university students from other majors, 14% were workers with a
university degree, 13.1% were workers with no university
degree, 9.3% were high school students, and 3.7% could
not be included in any of the former categories (e.g., retired
or unemployed). No data were obtained for the remaining
1.7% of participants.
Missing data were expected, given the size of the partic-

2010 Hogrefe & Huber Publishers

European Journal of Psychological Assessment 2010; Vol. 26(2):95107

Procedure

98

V. E. Caballo et al.: Measuring Social Anxiety in 11 Countries

ipant pool, but did not appear to affect validity of statistical

analyses. To confirm that there was no systematic data loss
pattern we tested data with SPSS MVA (missing value analysis). None of the variables exceeded 5% of missing data,
so it was not necessary to use t-test to verify if there was a
systematic relationship for missingness between the different pairs of variables, nor was there a need to implement
multiple imputation to substitute missing data. We opted
for a listwise deletion of cases with missing data. Of the
13,397 subjects in the original sample, a total of 12,144
participants were retained for the different factor analyses.

Results
Factor Analysis of the Initial Version of the
Questionnaire (SAQ-A = 516 Items)
In order to reduce the number of items, we performed an
exploratory principal components factor analysis with varimax rotation, which optimizes complex structures by capturing a small number of large loadings and a large number
of small loadings for each factor. Examination of the scree
plot suggested a 6-factor solution. The hierarchical analysis
of oblique factors gave the same 6-factor solution (Statsoft,
2006). We then performed an oblique principal component

cluster analysis in order to group the items into nonoverlapping clusters, so each cluster could be interpreted as unidimensional. This procedure allowed us to substitute a
group of variables with a smaller one (n-m) with the minimum loss of information in order to maximize the explained variance by the components of the cluster. This procedure is iterative, at each step suppressing those variables
that have the highest ratio values. The smaller these values
are, the greater the evidence that the variable has a strong
relationship with the rest of the components of the cluster
and a weak relationship with the components of the other
clusters. The 512 variables were considered in the analysis
(forcing a solution of 6 clusters). The four control items
were not included in the analysis, but they did allow us to
estimate how many subjects might be filling the questionnaire at random because they were answerable in only one
direction of increasing distress. Given the large sample size
relative to the extremely small number of participants
flagged by the control items, no action was taken. After
successive analyses suppressing variables with the highest
(1 R2own)/(1 R2next)1 ratio values, a solution of 12 items
per cluster was reached. The final distribution of the items
by cluster that were used in the subsequent analyses (exploratory and confirmatory factor analyses) is the same as
that found in Table 2.

Table 2. Item loadings for every factor and correlations item-total score for the SAQ-A
Factor loadings
Items and name of each factor

F1

F2

F3

F4

F5

F6

Item
total

F1. Awkward Behavior in Embarrassing Situations

304. Making a mistake in front of other people

.54

.02

.06

.05

.06

.23

.648

306. Wanting to start a conversation and not knowing how

.54

307. Realizing that I am boring the person that I am talking to

.68

.05

.13

.11

.05

.02

.17

.03

.01

.659

.02

.08

.629

386. Not knowing how to continue a conversation after a topic has been exhausted

.52

.00

.25

.07

.02

.03

.634

387. Speaking and it appearing like nobody is listening to me

.79

.05

388. Proposing an idea to a group of friends and not being taken seriously

.71

.05

.04

.16

.02

.09

.592

.05

.14

.03

.08

.600

389. Being alone at a party where I do not know anyone

.58

417. Wanting to end a conversation, but not knowing how

.52

.11

.08

.12

.11

.07

.654

.08

.14

.04

.10

.01

.665

420. Being at a friends house and not having anyone talking to me

456. Being told off or scolded by a superior or a person in authority

.69

.06

.08

.10

.01

.01

.609

.60

.08

.20

.01

.18

.15

.621

470. Talking to a stranger who keeps prying into my personal life

.66

.12

.15

.04

.06

.02

.557

487. Being in the home of strangers and not knowing what to say or do

.47

.09

.07

.09

.05

.04

.617

.29

.65

.4

.20

.12

.07

.570

F2. Interactions with the Opposite Sex

230. Being phoned by a person I am very attracted to
247. Feeling watched by people of the opposite sex

.10

.48

.13

.09

.02

.08

.658

289. Expressing to a person of the opposite sex that I love them

.04

.74

.07

.04

.00

.03

.549

In the formula, R2own represents the determination coefficient of each variable with its own cluster, and R2next the determination coefficient
of each variable with the nearest cluster. Naturally, we would want each component of the cluster to be strongly related with its own cluster
(R2own 1) and less related with the nearest cluster (R2next 0).

European Journal of Psychological Assessment 2010; Vol. 26(2):95107

2010 Hogrefe & Huber Publishers

V. E. Caballo et al.: Measuring Social Anxiety in 11 Countries

99

Factor loadings
Items and name of each factor

F1

316. Approaching someone I am attracted to but have never met

342. Maintaining a conversation with a person of the opposite sex whom I find attractive

F2

F3

F4

F5

F6

Item
total

.26

.45

.05

.02

.03

.03

.656

.01

.73

.09

.02

.01

.02

.640

343. Being openly stared at by someone

.25

.50

.02

.03

.05

.05

.601

362. Asking someone attractive of the opposite sex for a date

.20

.67

.06

.03

0.05

.02

.642

397. Being told by someone of the opposite sex that they like me

.11

.72

.01

.02

.01

.02

.636

421. Asking someone I find attractive to dance

.19

.52

.11

.03

.03

.03

.616

447. Being alone with someone I like very much

.13

.74

.05

.01

.06

.03

.643

452. Being asked out by a person I am attracted to

.02

.71

.16

.06

.06

.02

.642

.05

.61

.17

.06

.05

.04

.611

270. My friends bringing along people I do not know

.06

.03

.56

.07

.09

.05

.569

275. Greeting each person at a social meeting when I dont know most of them

.26

.05

.41

.07

.07

.11

.641

283. Attending a social event where I know only one person

.18

.01

.43

.10

.03

.11

.630

453. Talking about my personal feelings with someone of the opposite sex
F3. Interactions with Strangers

332. Talking on the phone with someone I do not know very well

.03

.03

.68

.07

.05

.04

.554

333. Greeting someone I do not know very well

.03

.02

.76

.07

.00

.00

.563

418. Making new friends

.04

.13

.58

.04

.00

.09

.542

441. Talking to a stranger

.07

.09

.70

.02

.05

.02

.594

443. Being introduced to new people

.07

.09

.78

.00

.02

.00

.567

449. Being asked to dance at a party

.02

.33

.37

.06

.10

.00

.545

467. Maintaining a conversation with someone Ive just met

.11

.20

.54

.03

0.04

.09

.667

501. Looking into the eyes of someone I have just met while we are talking

.07

.22

.44

.01

.07

.03

.523

504. Asking a stranger a question

.12

.04

.67

.00

.15

.01

.470

F4. Criticism and Embarrassment

14. Going to a party on my own when I dont know anyone

.05

.14

.08

.56

.19

.08

.479

18. Asking for a favor from a stranger

.04

.02

.09

.55

.00

.04

.456

20. Being told that I am doing something wrong

.12

.08

.21

.50

.04

.15

.458

39. Sitting at a table with strangers at a wedding

.00

.05

.20

.57

.10

.07

.521

44. Being criticized

.05

.08

.12

.48

.11

.08

.455

52. Greeting someone and being ignored

.12

.05

.17

.61

.02

0.10

.470

54. Expressing my opinion and not being understood

.09

.05

.03

.51

.18

.07

.446

70. Being teased in public

.07

.11

.04

.47

.05

.15

.488

73. Talking to someone who does not look at me

.21

.08

.06

.55

.02

.17

.369

128. Asking for a favor that is denied

.19

.02

.05

.48

.20

.00

.545

147. Entering or leaving in the middle of a social event

.08

.02

.13

.40

.09

.12

.551

197. Asking a question in public and not getting an answer

.31

.08

.08

.40

.13

.15

.582

160. While on a bus, asking someone not to step on me or push me

.05

.01

.04

.20

.56

.11

.542

201. Asking someone to stop kicking the back of my chair

.13

.02

.01

.19

.63

.5

.511

F5. Assertive Expression of Annoyance, Disgust or Displeasure

217. Expressing my annoyance to someone that is picking on me

.10

.04

.10

.14

.64

.13

.524

222. Asking someone who is speaking loudly at the movies to lower their voice

.08

.03

.03

.14

.63

.09

.549

260. Asking someone for an explanation

.07

.14

.19

.02

.46

.05

.578

263. Contradicting my parents opinion

.15

.01

.15

.06

.54

.15

.464

264. Arguing with my parents because I do not want to do a chore

.26

.06

.09

.02

.52

.16

.472

285. Having to ask a neighbor to stop making noise

.27

.03

.06

.01

.53

.01

.597

2010 Hogrefe & Huber Publishers

European Journal of Psychological Assessment 2010; Vol. 26(2):95107

100

V. E. Caballo et al.: Measuring Social Anxiety in 11 Countries

Factor loadings
Items and name of each factor

F1

F2

F3

F4

F5

F6

Item
total

299. Telling a taxi driver that he/she has taken an abnormally long route

.17

.02

.06

.06

.55

.00

.548

411. Telling a family member that they are bothering me

.32

.06

.05

.09

.46

.02

.596

482. Telling someone that their behavior bothers me and asking them to stop

.13

.04

.06

.07

.56

.01

.549

513. Telling a colleague they have done something that bothers me

.14

.05

.07

.08

.55

.01

.554

F6. Speaking/Performing in Public/ Talking with People in Authority

23. Being asked a question in class by the teacher or by a superior in a meeting

.11

.02

.00

.26

.10

.65

.503

167. Talking to a famous person or celebrity

.12

.16

.03

.13

.16

.45

.578

194. Having to speak in class, at work, or in a meeting

.11

.07

.08

.12

.01

.77

.578

195. Being interviewed

.06

.02

.04

.15

.09

.62

.576

208. Being summoned to speak to my superiors or a person in authority

.09

.16

.03

.16

.21

.42

.603

249. Participating in a meeting with people in authority

.11

.14

.10

.06

.11

.44

.647

269. Performing in public

.29

.04

.03

.06

.08

.60

.577

327. Speaking in public

.27

.02

.08

.10

.10

.68

.624

376. Asking questions in class, at a public event or in a crowded meeting

.25

.03

.17

.10

.02

.57

.651

401. Starting and maintaining a conversation with people in authority

.19

.14

.17

.15

.13

.39

.680

465. Taking the initiative in front of a group of strangers

.46

.03

.12

.07

.01

.32

.681

476. Making a presentation to people who know more than I do

Note. Factor loadings of items grouped under each specific factor are marked in bold.

.45

.13

.09

.13

.03

.39

.636

Exploratory Factor Analysis

In order to test whether the 72 items of the abbreviated instrument map onto the 6-factor structure of the original scale, we
conducted an exploratory factor analysis. Given the ordinal
nature of the data, we first computed a polychoric correlation
matrix from the direct scores of the 72 items. We then verified
that the items complied with the following conditions: (1)
there were no items with extreme distributions (skewness
from .36 to .41 with standard error of .023, kurtosis from
1.07 to .33 with SE = .05); (2) all the items within each
cluster separately had high corrected item-total correlations
(homogeneity index) (from .459 to .726); (3) all the proposed
factors had more than four items; (4) the sample was big
enough to thwart possible fluctuations of correlations; (5)
most of the elements of the anti-image correlation matrix
tended to zero; and (6) the KMO (Kaiser-Meyer-Olkin) index
exceeded the recommended cut-off of .50 (.98 in the current
sample). Given that the data met these conditions, we proceeded to apply the ordinal analysis through the unweighted
least squares (ULS) method and promax rotation.
Results by Bartletts test with 2556 df = 352275.768
(p < .000) showed that the variables were positively correlated, and that the data were adequate for an exploratory
factor analysis. Furthermore, the KMO index of .984
showed a high proportion of common variance explained
by factors. Both indices support the adequacy of factorial
analysis of data.
Matrix sampling adequacy (MSA) indices (ranging from
.951 to .994) confirm that the measure of sampling adequacy of the variables in all cases fits the structure of the rest

of the variables (in fact, they are above the value of .500
which is usually used as a threshold to discard a variable
from analysis). Finally, 60% of communalities were above
.50 (ranging from .35 to .70).
In order to decide the optimal number of factors, a parallel
analysis (Velicer, Eaton, & Fava, 2000; Watkins, 2000) was
implemented using the Monte Carlo procedure with 200 replications to determine the number of eigenvalues with values
above those that could be obtained from the same number of
subjects and variables (i.e., generating a group of random
values with normal distribution, calculating the matrix of correlations and subjecting it to principal components analysis
to calculate the mean eigenvalues). Results show that the 6factor solution is the best fit to our data, given that the size of
randomly generated eigenvalues after factor 6 is higher than
the observed eigenvalues.
This exploratory factor analysis identified 6 factors with
eigenvalues higher than 1.00 explaining 50.24% of the cumulative variance. Item loadings are presented in Table 2.
The first factor (eigenvalue = 25.49) explained 35.42% of
the variance. The 12 items loading highly on this factor
describe Awkward Behaviors in Embarrassing Situations.
The second factor showed an eigenvalue of 3.22 and explained 4.47% of the total variance. The 12 high loading
items describe situations of Interaction with the Opposite
Sex. Factor 3 showed an eigenvalue of 2.32 and explained
3.23% of the variance. The items of this factor refer to situations of Interaction with Strangers. Factor 4, with an eigenvalue of 1.98, explained 2.76% of the variance. The
items refer to situations of Criticism and Embarrassment.
Factor 5, with an eigenvalue of 1.67, explained 2.33% of

European Journal of Psychological Assessment 2010; Vol. 26(2):95107

2010 Hogrefe & Huber Publishers

V. E. Caballo et al.: Measuring Social Anxiety in 11 Countries

the variance and is related to Assertive Expression of Annoyance, Disgust or Displeasure. Factor 6, with an eigenvalue of 1.46, explained 2.03% of variance and was defined
as Speaking/Performing in Public/Talking with People in
Authority. Interfactor correlations were moderate (range =
.33 to .60) (see Table 6).

Confirmatory Factor Analysis

101

Table 3. Fit indices of the three tested models

Model 1 Model 2
#Absolute fit

Relative fit

S-B

Model 3

51629.98 12746.49

14706.52

p = .000 p = .000

p = .000

DF

252

237

246

GFI

.70

.91

.89

SRMR

.064

.036

.043

NFI

.95

.99

.99

NNFI

.95

.99

.98

RFI

.94

.99

.98

.95

.99

.99

.12

.063

.066

The results obtained through the exploratory factor analysis

were then subjected to confirmatory factor analysis from the
corresponding covariance and asymptotic variance-covariance matrices of items. Given the type of initial data (ordinal
variables and distributions that did not present multivariate
normality) the robust maximum likelihood (RML) estimation
method was used. The models that we tested included: (1)
single factor, (2) 6 factors, and (3) 6 first-order factors and
one second-order factor. The reasons for including these
models were that some studies have found a single higher-order factor explaining social anxiety (e.g., Mattick & Clarke,
1998; Osman et al., 1996) even with Spanish samples (Olivares et al., 2004) while others have found from three to 6
factors (e.g., Baker et al., 2002; Connor et al., 2000; Davidson
et al., 1997; Safren et al., 1999). Given that a 6-factor structure was found in our analyses, the 1-factor, 6-factor, and
combined models were tested. Following the recommendations made by Bentler (1995), a comparison of robust and
nonrobust estimation factors suggested that neither the kurtosis nor the skewness of distributions affected the results.
Multivariate kurtosis tests offered the following results: Srivastavas test: b2p = 3.9672; N(b2p) = 106.583; p = .000.
Mardias test: b2p = 787.3477; N(b2p) = 254.7749; p = .000.
When the analyses were applied to the transformed scores,
the results did not differ significantly in the three models. The
statistical programs SAS v. 9.1.3 (The SAS Institute, 2006),
PRELIS, v. 2.3 and LISREL, v. 8.8 (Scientific Software International, 2006a, 2006b) were used to perform the various
analyses.
Given that the number of items (72) was very high for
conducting a confirmatory factor analysis, we decided to
use the parceling procedure (Bandalos, 2002; Coffman &
McCallum, 2005; Nasser-Abu Alhija & Wisenbaker, 2006;
Sass & Smith, 2006). Each parcel was formed by the sum
of three items selected at random from every factor. Thus,
a total of 24 parcels were defined as indicators of the 6
latent variables. Before forming the parcels, the unidimensionality of each factor was verified. Furthermore, the reliability estimates (Cronbach ) for every group of items
of the hypothesized 6 factors were good, F1 = .92, F2 = .92,
F3 = .91, F4 = .86, F5 = .88, and F6 = .91.
The hypotheses tested can be summarized for the three
models as follows: (1) observed responses can be explained
by 1, 6, or 6 first-order factors and 1 second-order factor;
(2) each of the indicators has a loading that is statistically
different from 0 (i.e., t values higher than 2.58) in the hypothesized factor and zero loadings in the remaining fac-

tors, and (3) measurement errors associated with the indicators are not correlated with each other. The results of the
contrast comparisons of the three models are summarized
in Table 3.
As can be seen in Table 3, Models 2 (6 correlated factors) and 3 (6 first-order factors and one second-order factor) showed a good overall fit, suggesting that the restrictions we specified for the models were correct. However,
the fit of Model 2 was slightly better: the RMSEA index
was .063 in Model 2 and .066 in Model 3; indices SRMR
(.036 vs. .043), GFI (.91 vs. .89), NNFI and RFI (.99 vs.
.98) were also better for Model 2. Other indices comparing
the fit of Models 2 and 3, such as composite reliability and
average variance extracted (AVE) indicated a similar fit for
both models, although again slightly better for Model 2
than Model 3 (see Table 4). The average interitem correla-

2010 Hogrefe & Huber Publishers

European Journal of Psychological Assessment 2010; Vol. 26(2):95107

Noncentrality CFI
based fit
RMSEA

RMSEA 90% (.12;.12) (.062; .064) (.065;.067)

.000
.000
.000
PCLOSE
Note: RMSEA (root mean square error of approximation): Values less
or equal to .05 indicate close approximate fit; values between .05 and
.08 suggest reasonable error of approximation, and values higher or
equal to .10 suggest poor fit. SRMR (standardized root mean square
residual): values less than .10 are generally considered favorable; the
smaller the SRMR, the better the model fit. GFI (goodness of fit index), CFI (comparative fit index), NNFI (nonnormed fit index, Tucker-Lewis index), and RFI (relative fit index): values higher than .90
indicate good fit. NFI (normed fit Index): values higher than .95 indicate good fit (see Kline, 2005, for a review of all these indices).

Table 4. Composite reliability and average variance extracted of the three models
Model 1

Model 2

Model 3

Compos- AVE
ite reliability

Composite AVE
reliability

Compos- AVE
ite reliability

Factor 1 .963

.522

.903

.699

.903

.699

Factor 2

.912

.721

.913

.724

Factor 3

.886

.660

.886

.660

Factor 4

.839

.567

.840

.568

Factor 5

.869

.624

.868

.622

Factor 6

.883
.654
Note. AVE = Average variance extracted.

.883

.654

102

V. E. Caballo et al.: Measuring Social Anxiety in 11 Countries

tion was 0.486 for Factor 1, 0.487 for Factor 2, 0.436 for
Factor 3, 0.337 for Factor 4, 0.382 for Factor 5, and 0.442
for Factor 6. The total average interitem correlation was
0.337. Interfactor correlations were from moderate to relatively high (range = .64 to .84) (see Table 6).
In order to determine discriminant validity, the average
variance extracted (AVE) was compared with the coefficient of determination (R2) for each couple of latent variables. All the comparisons (10) carried out showed an AVE
greater than R2. This can be considered as a clear evidence
of discriminant validity since each latent construct must
explain the measures composing it rather than other constructs measures.
Composite reliability of each of the latent variables
(construct reliability) was calculated through the formula:

where are the loadings and is the indicator of error

variances. As Table 4 shows, the composite reliability for
latent variables in Model 2 was very similar to that of Model 3. These results were derived by calculating the average
variance extracted using the following formula:

In Models 2 and 3, the 6 factors showed an AVE greater

than 0.50, so we can therefore conclude that a high amount
of the indicator variance in both models is captured by the
construct.

Study 2: Development of the Final

Scale
Based on the analysis with the initial scale, we further examined the psychometric properties of the 72-item scale.
For this purpose, we constructed the Social Anxiety Ques-

tionnaire for Adults Revised (SAQ-AR) (Cuestionario de

Ansiedad Social para Adultos Revisado; CASO-AR),
which included the derived 72 randomly distributed items
on a 7-point (17) Likert rating scale. Administration instructions were the same as in the former version. The Pearson correlation of the SAQ-A (516 items) with the SAQAR (72 items) was r = .98.

Participating Countries and Researchers

The same countries from Study 1 participated in this second
study. However, the number of participating researchers
and subjects differed slightly: The total group of researchers in this second study consisted of 103 collaborators from
the same 11 countries. The numbers of researchers (and
groups of research) per country were as follows: Argentina
= 13 collaborators (3 groups of research); Brazil = 13 collaborators (5 groups of research); Chile = 6 (3 groups of
research); Colombia = 14 (8 groups of research); Costa Rica = 3 (2 group of research); Spain = 14 (8 groups of research); Mexico = 24 (12 groups of research); Paraguay =
3 (1 group of research); Peru = 5 (5 groups of research);
Uruguay = 3 (1 group of research); and Venezuela = 5 (3
groups of research).

Procedure
The procedure was similar to the first study. Collaborators
from each country revised each item of the SAQ-AR to fit
the everyday language of their country and culture. There
was no option to add new items. No significant changes
were made to the 72 items composing the CASO-AR. Data
collection took place over a period of 1 year.
In order to calculate additional psychometric properties
of this new questionnaire, such as consistency, validity, and
reliability, we selected some self-report instruments usually
employed to assess social phobia/anxiety, such as the SPAI

Table 5. Correlations (Pearson) among the SAQ-AR and its 6 factors with other self-report measures of social anxiety
Questionnaires for assessing social phobia/anxiety
SAQ-AR and its factors

SPAI
96 items

SPAI
Sp Ag

LSAS
Anxiety

LSAS
Avoidance

SPIN

F1. Awkward behavior in social embarrassing situations

.64

.59

.59

.43

.59

F2. Interactions with the opposite sex

.62

.58

.58

.45

.58

F3. Interactions with strangers

.75

.75

.62

.44

.64

F4. Criticism and embarrassment

.69

.64

.62

.51

.60

F5. Assertive expression of annoyance, disgust or displeasure

.49

.44

.50

.39

.48

F6. Speaking/performing in public/ Talking with people in authority

.62

.55

.55

.44

.56

Sum of factors score (SAQ-AR)

.78
.74
.72
.56
.69
Note: All correlations significant at p < .0001. SPAI = Social Phobia and Anxiety Inventory; LSAS = Liebowitz Social Anxiety Scale; SPIN =
Social Phobia Inventory. SPAI 96 items = Sum of the score on the 96 items of the Social Phobia Subscale without averaging the items with four
subitems; SPAI SP-Ag = Typical scoring procedure of the questionnaire, Social Phobia subscale score Agoraphobia subscale score.
European Journal of Psychological Assessment 2010; Vol. 26(2):95107

2010 Hogrefe & Huber Publishers

V. E. Caballo et al.: Measuring Social Anxiety in 11 Countries

Table 6. Interfactor correlations for exploratory and confirmatory factor analysis of the abbreviated version
of the SAQ-A
Interfactor correlations for exploratory factor analysis
F1

F2

F3

F4

F5

F1

1.00

F2

0.56

1.00

F3

0.50

0.58

1.00

F4

0.42

0.37

0.33

1.00

F5

0.51

0.49

0.46

0.46

1.00

F6

0.50

0.60

0.51

0.45

0.49

F6

1.00

Interfactor correlations for confirmatory factor analysis

F1

F2

F3

F4

F1

1.00

F2

0.78

1.00

F3

0.72

0.83

1.00

F4

0.80

0.65

0.64

1.00

F5

0.79

0.72

0.74

0.76

F5

F6

1.00

F6 0.83
0.84
0.81
0.77
0.78
1.00
Note: F1. Awkward behavior in social embarrassing situations; F2.
Interactions with the opposite sex; F3. Interactions with strangers; F4.
Criticism and embarrassment; F5. Assertive expression of annoyance,
disgust or displeasure; F6. Speaking/performing in public/Talking
with people in authority.

103

a) Social Phobia and Anxiety Inventory (SPAI; Turner,

Beidel et al., 1989), a 45-item self-report instrument designed to measure social phobia. Each item is rated for
frequency on a 7-point scale ranging from 0 (never) to 6
(always). The inventory consists of 2 subscales: social
phobia (32 items) and agoraphobia (13 items). However,
18 items of the social phobia subscale have 4 subitems
each, 2 items have 5 subitems each, and 1 item has 3
subitems.
b) The Liebowitz Social Anxiety Scale (LSAS; Liebowitz,
1987) is a 24-item self-report instrument that assesses
fear and avoidance of specific social situations. Respondents are asked to rate fear on a 4-point scale ranging
from 0 (none) to 3 (severe) and avoidance on a 4-point
scale ranging from 0 (never) to 3 (usually).
c) The Social Phobia Inventory (SPIN; Connor et al., 2000)
is a 17-item questionnaire that assesses symptoms of
social phobia. Each item contains a symptom that is rated
by the respondent based on how much he or she was
bothered by the symptom during the prior week on a
5-point scale ranging from 0 (not at all) to 4 (extremely).

Results
Confirmatory Factor Analysis

As noted above, three self-report measures of social phobia

were used, together with the SAQ-AR, to obtain concurrent
validity ratings. The measures were:

The univariate and multivariate normality of indicators

were analyzed using the program PRELIS 2.3 (Scientific
Software International, 2006). As the data did not meet the
condition of multivariate normality (Skewness-z = 79.114,
p = .000; Kurtosis-z = 98.164, p = .000), confirmatory factor analysis was implemented on variance-covariance and
asymptotic covariance matrices through the robust maximum likelihood estimation method (RML). The same parceling procedure used in Study 1 was implemented in this
Study 2.
Goodness of fit was verified through different absolute,
relative, and noncentrality indices, such as GFI, SRMR,
NFI, NNFI, RFI, CFI, and RMSEA. Acceptable fit was defined by the following criteria: GFI > .90; SRMR < .08;
NFI > .95; NNFI > .95; RFI > .95; CFI > .95; and RMSEA
( < .06 90% CI < .06). Multiple fit indices were used because they provide us with varied information about model
fit, and, when used together, they provide us with a more
conservative and reliable evaluation of the solution.
The analysis of the SAQ-AR indicated that two models
should be tested: (1) Model 2, with 6 correlated factors, and
(2) Model 3, with 6 first-order factors and 1 second-order
factor. Consistent with the previous analyses of the SAQ-A,
the 6-factor model (GFI = .94; SRMR = .038; NFI = .99;
NNFI = .99; RFI = .99; CFI = .99; RMSEA = .052) presented a better fit overall than the hierarchical model (GFI
= .88; SRMR = .072; NFI = .98; NNFI = .98; RFI = .98;
CFI = .98; RMSEA = .072).
All freely estimated unstandardized parameters (range
from .64 to .88) were statistically significant (p values <

2010 Hogrefe & Huber Publishers

European Journal of Psychological Assessment 2010; Vol. 26(2):95107

(Turner, Beidel et al., 1989), the LSAS (Liebowitz, 1987),

and the SPIN (Connor et al., 2000).

Participating Subjects
A total of 11,026 subjects participated in the second study.
The mean age of the total sample was 25.65 years (SD =
10.63) and consisted of 5,935 women (mean age = 24.79;
SD = 10.51) and 5,091 men (mean age = 25.81; SD =
10.74). The minimum age for subjects was 16 years, but
there was no upper age limit. Table 1 shows the sex, age,
and number of subjects in the participating countries. The
participants had different levels of education (students,
workers, etc.). Specifically, 22% were psychology students, 39.5% were university students with other majors,
14.7% were workers with a university degree, 6.9% were
workers with no university degree, 4.9% were school students, and 6.4% could not be included in any of the former
categories. No data were obtained for the remaining 5.6%
of participants. Missing data were addressed using listwise
deletion, as in the first study, so that the final number of
subjects for factor analysis was 10,118.

Instruments

104

V. E. Caballo et al.: Measuring Social Anxiety in 11 Countries

.001). Factor loadings estimates showed that indicators

were strongly related to latent variables (R2 ranged from
.41 to .77), whereby the indicators used are reliable measures of the 6 dimensions composing social anxiety in this
study. Finally, as hypothesized, the 6 factors showed positive and moderately high intercorrelations.

Discussion

Some of the most widely used questionnaires for measuring

social phobia/anxiety were administered together with the
SAQ-AR, including the Social Phobia and Anxiety Inventory (SPAI; Turner, Beidel et al., 1989), the Liebowitz Social Anxiety Scale (LSAS; Liebowitz, 1987), and the Social
Phobia Inventory (SPIN; Connor et al., 2000). A total of
511 university subjects participated in this part of the study
(135 men with a mean age of 23.00 years and an SD of
6.41, and 376 women with a mean age of 21.55 years and
an SD of 4.68). The average interitem correlation for the
LSAS-Anxiety was 0.27 and the Cronbach = .90, for the
LSAS-Avoidance .23 and the = .87, for the SPAI-Social
Phobia Subscale 0.40 and the = .98, and for the SPIN .38
and the = .91. Table 5 shows the relationships between
the SAQ-AR and its 6 factors with the scores for the other
three measures, specifically the score on the 96 items of the
Social Phobia subscale of the SPAI without averaging the
items with four subitems (SPAI 96 items), the typical scoring procedure of the SPAI (Social Phobia Subscale score
[32 items] Agoraphobia Subscale score [13 items]), the
LSAS Anxiety score, the LSAS Avoidance score, and the
SPIN total score. These correlations are moderately high,
particularly with the overall score of the SAQ-AR (from a
low .56 to a high .78) and with some of the factors usually
found in most of the questionnaires, such as Interaction
with Strangers (from .44 to .75), Criticism and Embarrassment (from .51 to .69), and Speaking/Performing in Public
(from .44 to .62). The LSAS Avoidance showed the lowest
correlations with the SAQ-AR and its factors.

This work presents the development and initial psychometric evaluation of a new questionnaire designed to measure
social phobia/anxiety. Although there are already a number
of anxiety scales in existence, they all suffer from several
notable weaknesses, not the least of which is the manner in
which their items were generated. Existing measures contain items that were adapted from other measures, adapted
from DSM-III or DSM-IV criteria, or by generating items
based on the opinions of experts without secondary confirmation of their validity. By contrast, the present research
applied an objective method to diary-generated items by
asking a large group of diverse participants to record any
social situations that elicited social anxiety during their daily lives. Furthermore, in contrast to existing measures, we
recruited a large and culturally diverse sample from Spanish- and Portuguese-speaking countries.
An extensive first version of the questionnaire, the SAQA, was applied to a large sample of people from 10 LatinAmerican countries and Spain. Objective statistical reduction of the scale produced an instrument with 6 factors and
72 items. Given the item generation procedure and the robust factor structure observed in large samples, we believe
that our measure adequately describes the structure of social anxiety among Latin-American and Spanish-speaking
people. Note also that we followed most of the content validation guidelines proposed by some authors (e.g., DeVellis, 2003; Haynes et al., 1995). It remains to be seen whether the same factor structure will be observed among individuals with social anxiety disorder (social phobia) and
among people from other countries (e.g., Europe, North
America, etc.).
When comparing the factor structure of the SAQ-AR
and the most used social anxiety instruments (SPAI, LSAS,
SPIN, SIAS, SPS, and BPS), some surprising findings
emerged. Only 2 factors were consistently identified,
namely, Interactions with Strangers and Speaking/Performing in Public. Interestingly, the factor Interaction with
the Opposite Sex was not identified in the factor structure
of any of the popular existing measures. This seems surprising, given the reported centrality of this problem in the
lives of individuals with social anxiety/phobia. Yet, existing measures include very few, if any, items dealing with
anxiety about social interaction with the opposite sex. For
example, the SPS and SPIN include no items, and the SIAS
and the LSAS include only one item assessing this problem
area. The SPAI includes 17 of the 96 items. However, these
17 items are really subitems grouped in every case with
other three subitems (fear of strangers, people in authority,
and people in general) to give the mean score of 17 higher items. We understand this to be a limitation of the scoring procedure of the SPAI. A better approach might have
been to consider each subitem as an independent item rather than averaging different items prior to calculating a
score. This method would be more likely to result in a sep-

European Journal of Psychological Assessment 2010; Vol. 26(2):95107

2010 Hogrefe & Huber Publishers

Internal Consistency and Reliability

Internal consistency was calculated via Cronbachs for
every factor (12 items each) and the sum of the factors.
Cronbachs was .89 for Factor 1, .88 for Factor 2, .86 for
Factor 3, .87 for Factor 4, .84 for Factor 5, and .90 for
Factor 6, with an of .97 for the total scale (SAQ-AR).
The split-half reliability of the SAQ-AR was very good
(Guttman split-half reliability = .973). The average interitem correlation was 0.398 for Factor 1, 0.395 for Factor 2,
0.350 for Factor 3, 0.357 for Factor 4, 0.303 for Factor 5,
and 0.420 for Factor 6. The total average interitem correlation was 0.279.

Concurrent Validity

V. E. Caballo et al.: Measuring Social Anxiety in 11 Countries

105

arate factor of Interactions with the Opposite Sex, as we

have found in a recent study with almost 1,000 people (Caballo & Nobre, 2009).
Another factor that only infrequently appears in other
instruments is Assertive Expression of Annoyance, Disgust,
or Displeasure. The SPIN, SPAI, and SPS do not include
any items dealing with this issue, and the SIAS has only
one item. Only the LSAS includes a few items on assertiveness. Again, this is surprising given the centrality of
assertiveness issues in social anxiety. For instance, Caballo,
Olivares, Lpez-Gollonet, Irurtia, and Rosa (2003) found
moderate relationships between social phobia/anxiety
(measured with the SPAI, the LSAS, the SPS, and the
SIAS) and assertiveness (measured with the College SelfExpression Scale, CSES; Galassi, DeLo, Galassi, & Bastien, 1974). Specifically, they found relationships as high
as .57 between the CSES and the SPAI-Social Phobia Subscale, .61 with the SIAS, 0.59 with the LSAS-Anxiety,
and 0.58 with the LSAS-Avoidance.
Finally, factors related to Embarrassing Situations (Factors 1 and 4) are usually overrepresented in existing questionnaires, even if they usually differ in their names. Although the fear of being observed is an important central
concern among individuals with social phobia, existing
measures seem to overemphasize this issue while omitting
other problem areas. For example, most of the items of the
SPS and the SPIN deal with the fear of embarrassing situations.
In addition to the goal of deriving an objectively generated measure of social anxiety, a second goal was to create
a measure with cross cultural relevance to Spain and Latin
America. Indeed, this was necessary because there are certain social situations included in most of the existing selfreport instruments that may not apply to Spain and LatinAmerican countries. For instance, the situation drinking
in public does not seem to be a typical concern among
socially anxious individuals in those countries. The relationship between this single item and the total score of the
questionnaire is one of the lowest correlations (r = .27) for
any of the items on the SAQ-A. People from Latin-American countries spend a significant part of their leisure time
out of their homes at night, sitting outside or in bars, drinking in public. Therefore, drinking in public is rarely a problem, even among socially phobic individuals. A greater
problem in those cultures is not having any friends with
whom to engage in this activity. We do not think items such
as drinking in public reflect a representative behavior of
socially anxious people in the countries participating in this
study. Similar concerns can be raised with a few other
items, such as using public bathrooms. It is our impression
that paruresis is not a significant problem in Spanish and
Latin-American cultures. However, this issue awaits further empirical studies.
With regard to the psychometric data of the new questionnaire, we found high internal consistency (Cronbachs
= .96) and split-half (Guttman = .97) reliability for the
SAQ-AR (72 items). Concurrent validity was also good as

This study was made possible by a grant from Spains Ministry of Science and Technology awarded to the research
project with reference BSO2003-07029/PSCE and cofinanced by the European Regional Development Fund
(ERDF). Stefan G. Hofmann is supported by a grant from
the National Institute of Mental Health (MH075889) and
is a consultant for Organon.
We heartily appreciate the collaboration of all the subjects from the different countries who voluntarily participated in this study.
The CISO-A Research Team, co-author of this article, is
composed of the following researchers: Argentina: G. Bra-

2010 Hogrefe & Huber Publishers

European Journal of Psychological Assessment 2010; Vol. 26(2):95107

shown by high correlations with the SPAI (.74), the LSAS

(.72), and the SPIN (.69). The 6 factors of the SAQ-AR
further showed moderate correlations with these measures,
ranging from .44 to .75. Even factors that did not correspond to any of the existing questionnaires, such as Interactions with the Opposite Sex and Assertive Expression of
Annoyance, Disgust, or Displeasure showed correlations
ranging from .44 to .62, indicating that our scale measured
areas of social anxiety that the other scales might miss. The
lowest correlation was always with the Avoidance scale of
the LSAS. However, as Heimberg et al. (1999) noted, fear
and avoidance ratings do not seem to measure distinct constructs. Furthermore, Oakman et al. (2003) questioned the
distinction between the fear and avoidance subscale.
There are limitations of the present study. For instance,
it might have been useful to report reliability estimates for
the parcels used in confirmatory factor analysis. However,
the small number of items in each parcel does not favor
high reliability. Another limitation may be that, although
we described some of the weaknesses of the habitual measures used in the current assessment of social phobia/anxiety at the beginning of this study, we used several of them
to obtain the concurrent validity of the SAQ-AR. Those
measures of social phobia/anxiety might appraise this construct globally, particularly generalized social phobia (and
this was correct as measures for the concurrent validity of
the SAQ-AR), but they do not capture all the dimensions
of social phobia/anxiety, and some discrete social phobias
may not be correctly identified.
Although the SAQ-AR is highly promising, additional
information is needed, especially in terms of its utility in
clinical samples. The clinical data should help to focus on
discriminant items in order to identify individuals with social anxiety disorders versus those without the disorder. We
are particularly interested in detecting individuals with circumscribed social phobia, a task not well accomplished by
existing self-report measures (Bhogal & Baldwin, 2007).
Further research should also focus on the stability of the
factor structure reported here. Finally, the application of the
SAQ-AR as a pre- or posttreatment measure should further
validate its usefulness in the clinical arena.

Acknowledgments

106

V. E. Caballo et al.: Measuring Social Anxiety in 11 Countries

American Psychiatric Association. (1980). Diagnostic and statistical manual of mental disorders DSM-III (3rd ed.) Washington, DC: Author.
Antony, M. M., Coons, M. J., McCabe, R. E., Ashbaugh, A., &
Swinson, R. P. (2006). Psychometric properties of the Social
Phobia Inventory: Further evaluation. Behavior Research and
Therapy, 44, 11771185.
Baker, S. L., Heinrichs, N., Kim, H., & Hofmann, S. G. (2002).
The Liebowitz Social Anxiety Scale as a self-report instrument: A preliminary psychometric analysis. Behavior Research and Therapy, 40, 701715.
Bandalos, D. L. (2002). The effects of item parceling on goodness-of-fit and parameter estimate bias in structural equation
modeling. Structural Equation Modeling, 9, 78102.
Bentler, P. M. (1995). EQS structural equations software manual.
Encino, CA: Multivariate Software.
Bhogal, K. S., & Baldwin, D. S. (2007). Pharmacological treatment of social phobia. Psychiatry, 6, 217223.
Caballo, V.E., & Nobre, L.A. (2009). Factor analysis, sex differences and psychometric characteristics of the Social Phobia
and Anxiety Inventory (SPAI). Unpublished manuscript, University of Granada, Spain.
Caballo, V. E., Olivares, J., Lpez-Gollonet, C., Irurtia, M. J., &
Rosa, A. I. (2003). Una revisin de los instrumentos para la
evaluacin de la fobia social: Algunos datos empricos [A review of the instruments for the assessment of social phobia:
Some empirical data]. Psicologa Conductual, 11, 539562.
Coffman, D. L., & MacCallum, R. C. (2005). Using parcels to

convert path analysis models into latent variable models. Multivariate Behavioral Research, 40, 235259.
Connor, K. M., Davidson, J. R. T., Churchill, L. E., Sherwood, A.,
Foa, E., & Weisler, R. H. (2000). Psychometric properties of
the Social Phobia Inventory (SPIN): A new self-rating scale.
British Journal of Psychiatry, 176, 379386.
Davidson, J. R., Miner, C. M., De Veaugh-Geiss, J., Tupler, L. A.,
Colket, J. T., & Potts, N. L. (1997). The Brief Social Phobia
Scale: A psychometric evaluation. Psychological Medicine,
27, 161166.
Davidson, J. R., Potts, N. L., Richichi, E. A., Ford, S. M., Krishnan, R. R., Smith, R. D. et al. (1991). The Brief Social Phobia
Scale. Journal of Clinical Psychiatry, 52, 4851.
DeVellis, R. F. (2003). Scale development: Theory and applications (2nd ed.). Thousand Oaks, CA: Sage.
Fresco, D. M., Coles, M. E., Heimberg, R. G., Liebowitz, M. R.,
Hami, S., Stein, M. B. et al. (2001). The Liebowitz Social Anxiety Scale: A comparison of the psychometric properties of
self-report and clinician-administered formats. Psychological
Medicine, 31, 10251035.
Galassi, J. P., DeLo, J. S., Galassi, M. D., & Bastien, S. (1974).
The College Self-Expression Scale: A measure of assertiveness. Behavior Therapy, 5, 165171.
Haynes, S., Richard, D., & Kubany, E. (1995). Content validity
in psychological assessment: A functional approach to concepts and methods. Psychological Assessment, 7, 238247.
Heimberg, R. G., Horner, K. J., Juster, H. R., Safren, S. A., Brown,
E. J., Schneieer, F. R. et al. (1999). Psychometric properties of
the Liebowitz Social Anxiety Scale. Psychological Medicine,
29, 199212.
Heinrichs, N., Rapee, R. M., Alden, L. A., Bgels, S., Hofmann,
S. G., Oh, K. J. et al. (2006). Cultural differences in perceived
social norms and social anxiety. Behavior Research and Therapy, 4, 11871197.
Hofmann, S. G., & DiBartolo, P. M. (2000). An instrument to assess self-statement during public speaking: Scale development
and preliminary psychometric properties. Behavior Therapy,
31, 499515.
Hofmann, S. G., DiBartolo, P. M., Holoway, R. M., & Heimberg,
R. G. (2004). Scoring error of Social Avoidance and Distress
Scale and its psychometric implications. Depression and Anxiety, 19, 197198.
Johnson, H. S., Inderbitzen-Nolan, H. M., & Anderson, E. R.
(2006). The Social Phobia Inventory: Validity and reliability
in an adolescent community sample. Psychological Assessment, 18, 269277.
Kline, R. B. (2005). Principles and practice of structural equation
modeling. New York: Guilford.
Liebowitz, M. R. (1987). Social phobia. Modern Problems in
Pharmacopsychiatry, 22, 141173.
Mattick, R. P., & Clarke, J. C. (1998). Development and validation
of measures of social phobia scrutiny and social interaction
anxiety. Behavior Research and Therapy, 36, 455470.
Nasser-Abu Alhija, F., & Wisenbaker, J. (2006). A Monte Carlo
study investigating the impact of item parceling strategies on parameter estimates and their standard errors in CFA. Structural
Equation Modeling: A Multidisciplinary Journal, 13, 204228.
Olivares, J., Garca-Lpez, L. J., Hidalgo, M. D., & Caballo, V.
(2004). Relationships among social anxiety measures and their
invariance: A confirmatory factor analysis. European Journal
of Psychological Assessment, 20, 172179.

European Journal of Psychological Assessment 2010; Vol. 26(2):95107

2010 Hogrefe & Huber Publishers

gagnolo, A. Ciliberti, M. Correche, L. Gmez, R. Gmez,

M. Granero, M. Milanesio, M. Pinto, F. Rivarola, P. Robles,
S. Savoini, L. Tapia, A. Villafae. Brazil: V. Casella, M. de
Freitas, M. de Oliveira, J. Dias, M. Dias, B. Donato, A. dos
Santos, F. Gauy, L. Loureno, R. Lopes, E. Silvares, M.
Oliveira, M. Wagner. Chile: F. Bono, I. Correa, M. Donoso,
C. Garcs, A. Prieto, C. Soto. Colombia: D. Cceres, G.
Cajiao, H. Castaeda, G. Ceballos, J. Gmez, R. Mazo, A.
Meneses, A. Orozco, X. Palacios, A. Prez, M. Restrepo,
B. Ruiz, M. Varela. Costa Rica: Y. Alfaro, E. Bravo, A.
Soto. Spain: P. Bas, E. Calvete, M. Fernndez, M. Garca,
S. Lago, J. Moriana, I. Orue, J. Piedra, C. Rausell, A. Rego,
J. Sevill, C. Pastor, L. Garrido. Mexico: S. Anguiano, M.
Ascencio, I. Blanquel, V. Crdenas, I. Cortes, N. Dvila, G.
Garca, F. Gil, C. Gonzlez, M. Gonzlez, L. Hernndez,
R. Landero, A. Lpez, F. Lpez, J. Olvera, F. Pez, B. Ramos, M. Ros, R. Robles, I. Silva, R. Tenorio, M. Vzquez,
G. Vila, V. Vega. Paraguay: A. Caballero, R. Estigarribia,
S. Martnez. Peru: A. Barreda, J. Montero, M. Salazar, C.
Segura, C. Velsquez. Uruguay: M. Golberg, M. Lagos, M.
Portela. USA: R. Acierno. Venezuela: Y. Alfonso, L. Feldman, Z. Lugli, J. Pellicer, E. Vivas.
The first author mentioned is the Director of the research
team and the person responsible for the CISO-A Research
Team; he also holds all the rights for this team.

References

V. E. Caballo et al.: Measuring Social Anxiety in 11 Countries

Olivares, J., Garca-Lpez, L. J., Hidalgo, M. D., Turner, S. M., &

Beidel, D. C. (1999). The Social Phobia and Anxiety Inventory: Reliability and validity in an adolescent Spanish population. Journal of Psychopathology and Behavioral Assessment,
21, 6778.
Osman, A., Barrios, F. X., Aukes, D., & Osman, J. R. (1995). Psychometric evaluation of the Social Phobia and Anxiety Inventory
in college students. Journal of Clinical Psychology, 51, 235243.
Osman, A., Barrios, F. X., Haupt, D., King, K., Osman, J. R., &
Slavens, S. (1996). The Social Phobia and Anxiety Inventory:
Further validation in two nonclinical samples. Journal of Psychopathology and Behavioral Assessment, 18, 3547.
Radomsky, A. S., Ashbaugh, A. R., Saxe, M. L., Ouimet, A. J.,
Golden, E. R., Lavoie, S. L. et al. (2006). Psychometric properties of the French and English versions of the Social Phobia Inventory. Canadian Journal of Behavioral Science, 38, 354360.
Safren, S. A., Heimberg, R. G., Horner, K. J., Juster, H. R.,
Schneier, F. R., & Liebowitz, M. R. (1999). Factor structure of
social fears: The Liebowitz Social Anxiety Scale. Journal of
Anxiety Disorders, 13, 253270.
SAS Institute. (2006). SAS, v. 9.3 [Computer program]. Cary, NC:
The SAS Institute.
Sass, D. A., & Smith, P. L. (2006). The effects of parceling unidimensional scales on structural parameter estimates in structural equation modeling. Structural Equation Modeling: A
Multidisciplinary Journal, 13, 566586.
Scientific Software International. (2006a). LISREL, v. 8.8 [Computer program]. Lincolnwood, IL: Author.
Scientific Software International. (2006b). PRELIS, v. 2.3 [Computer program]. Lincolnwood, IL: Author.
Slavkin, S. L., Holt, C. S., Heimberg, R. G., Jaccard, J. J., & Lie-

2010 Hogrefe & Huber Publishers

107

bowitz, M. R. (1990, November). The Liebowitz Social Phobia

Scale: An exploratory analysis of construct validity. Paper presented at the annual meeting of the Association for the Advancement of Behavior Therapy, Washington.
Statsoft. (2006). Statistica, v. 7.1 [Computer program]. Tulsa,
OK: Author.
Turner, S. M., Beidel, D. C., Dancu, C. V., & Stanley, M. A.
(1989). An empirically derived inventory to measure social
fears and anxiety: The Social Phobia and Anxiety Inventory.
Psychological Assessment, 1, 3540.
Turner, S. M., McCanna, M., & Beidel, D. C. (1987). Validity of
the social avoidance and distress and fear of negative evaluation scales. Behavior Research and Therapy, 25, 113115.
Turner, S. M., Stanley, M. A., Beidel, D. C., & Bond, L. (1989).
The Social Phobia and Anxiety Inventory: Construct validity.
Journal of Psychopathology and Behavioral Assessment, 11,
221234.
Watson, D., & Friend, R. (1969). Measurement of social-evaluative anxiety. Journal of Consulting and Clinical Psychology,
33, 448457.

Vicente E. Caballo
Faculty of Psychology
University of Granada
E-18071 Granada
Spain
Tel./Fax +34 958 125 927
E-mail vcaballo@ugr.es

European Journal of Psychological Assessment 2010; Vol. 26(2):95107