Você está na página 1de 8

RESEARCH

Original Research

Meets Learning Need Codes 3000, 3010, 3020, 4000, and 4050. To take the Continuing Professional
Education quiz for this article, log in to ADAs Online Business Center at www.eatright.org/obc, click the
Journal Article Quiz button, click Additional Journal CPE Articles, and select this articles title from a
list of available quizzes.

Validation and Calibration of a Semiquantitative


Food Frequency Questionnaire Designed for
Adolescents
MARINA CAMPOS ARAUJO, MS; EDNA MASSAE YOKOO, PhD, MPH; ROSANGELA ALVES PEREIRA, PhD, MPH

ABSTRACT
Background Epidemiologic studies must have a reliable
method for evaluating food intake; therefore, valid, precise, and practical instruments are essential.
Objective To assess the relative validity and estimate the
calibration factors of a semiquantitative food frequency
questionnaire (FFQ) for adolescents.
Design Validation and calibration study.
Subjects/settings This study enrolled 169 adolescents from
Rio de Janeiro, Brazil. A set of three food records was
used as the reference method.
Statistical analyses performed The relative validity was analyzed according to weighted values for the quartile categorization of energy and nutrient intakes, Pearson correlation coefficients, and the Bland-Altman method. To
estimate the calibration factors for the FFQ, linear regression models, including the food record means as dependent
variables and the FFQ estimations as independent variables were developed for boys and girls.
Results The weighted values ranged from 0.28 to 0.44 for
the raw data and from 0.16 to 0.39 for the deattenuated
and energy-adjusted data. The Pearson correlation coef-

M. C. Araujo is a nutritionist, master in nutrition, and


R. A. Pereira is an assistant professor, Department of
Nutrition, Federal University of Rio de Janeiro, Rio de
Janeiro, Brazil. E. M. Yokoo is an assistant professor,
Department of Community Health, Fluminense University, Rio de Janeiro, Brazil.
Address correspondence to: Rosangela Alves Pereira,
PhD, MPH, Department of Nutrition, Federal University
of Rio de Janeiro, Brazil, Av Carlos Chagas Filho, 373
Centro de Cincias da Sade Bloco J, 2 andar, Cidade
Universitria 21.941-902 - Rio de Janeiro, RJ Brazil.
E-mail: roapereira@ufrj.br
Manuscript accepted: March 1, 2010.
Copyright 2010 by the American Dietetic
Association.
0002-8223/$36.00
doi: 10.1016/j.jada.2010.05.008

1170

Journal of the AMERICAN DIETETIC ASSOCIATION

ficients ranged from 0.33 to 0.46, and the mean agreement varied from 62% to 143%. The calibration factors
estimated for boys ranged from 0.15 to 0.48, and the
factors estimated for girls ranged from 0.14 to 0.47. The
mean energy and nutrient intakes estimated by the calibrated FFQ were similar to the means estimated by the
food records; however, the standard deviations were
smaller for the calibrated FFQ estimations.
Conclusions The tested FFQ is a suitable tool for ranking
energy and nutrients intake in the studied group. Calibration factors are needed to estimate energy and nutrient intake means and should be used to correct raw data
as well as association measurements based on FFQ data.
J Am Diet Assoc. 2010;110:1170-1177.

o investigate the relationship between diet and an


outcome, epidemiologic studies must have a reliable
method for evaluating food intakevalid, precise, and
practical instruments are essential (1). It is still difficult to
accurately estimate the food intake of individuals or populations, and studies have focused on refining the methods
and techniques used to evaluate food intake (2).
The tools applied in dietary intake assessment are subjected to diverse sources of errors. For example, the limitations of food frequency questionnaires (FFQs) are linked to
a fixed list of foods, and to the conceptual abilities related to
the recalling of the frequency and the amount of food intake
patterns during an extended period. Food record drawbacks
are related to participants motivation and literacy. Twentyfour hour recalls rely on the participants memory and abilities in describing foods and quantities; moreover, as with
other methods based on interviews, they also can be subjected to the interviewers bias (3,4).
Nevertheless, FFQs are a convenient and informative
way to evaluate diet (5). An FFQ analyzes past diet (either recent or remote) to estimate the usual intake of
specific foods. An FFQ should be specific to the population
under study, since the provided list of foods should be
based on the populations dietary habits (4).
The estimated energy and nutrients intake originated
from FFQs can be over- or underestimated, as happens

2010 by the American Dietetic Association

with other methods (6,7). Therefore, to correctly interpret


epidemiologic studies based on an FFQ, the instrument
must be validated and calibrated (8). This process allows
researchers to recognize the degree of error included in
the FFQ estimation and to adjust epidemiologic measurements as needed (3).
In Brazil, Slater and colleagues (9) validated a semiquantitative FFQ containing 76 food items that was designed for adolescents living in So Paulo (State of So
Paulo), and Carvalho and colleagues (10) developed an
FFQ with 34 foods items for adolescents from Teresina
(State of Piau); also, two studies have calibrated FFQs
designed for adolescents (11,12). However, in the Rio de
Janeiro area, only one study validated a semiquantitative
FFQ for adults (13).
This study analyzes the relative validity and estimates
the calibration factors of a semiquantitative FFQ designed to estimate the usual food intake of adolescents
from the metropolitan area of Rio de Janeiro, Brazil.
METHODS
Design
The study was conducted to evaluate an FFQs validity
and reproducibility and to estimate calibration factors to
be used in the correction of population energy and nutrients intake means. The reference method was a set of
three food records administered on nonconsecutive days
in-between two administrations of the FFQ (designated
the test and retest). The first food record was administered on the following day to the first FFQ. The time
interval between each food record was 1 week. The test
and retest administration of the FFQ were, in average, 20
days apart. The first FFQ application was included in the
validation and calibration studies.
Subjects
The study included adolescents (aged 12 to 19 years) from
different areas and socioeconomic strata of the Rio de Janeiro metropolitan area. The research was approved by the
Institutional Review Board of the University of the State of
Rio de Janeiro on January 13, 2005; study participants
older than 18 years old or the parents/legal guardians of
younger adolescents signed informed consent forms. The
data were collected in individual interviews administered
by nutritionists and trained college students.
A total of 169 adolescents completed the FFQ and the
three food records. Students from a public school in the
municipality of Niteroi were interviewed between May
and July 2005, adolescents residing in the municipality of
Duque de Caxias were interviewed between January and
April 2006, and students from a private school located in
the city of Rio de Janeiro were interviewed between October and November 2006. This way the data allowed
capturing the expected seasonal variation in the offering
of fruits and vegetables. In Brazilian urban areas, for a
large number of fruits and vegetables this variation is
mostly due to price fluctuations.
In the public school in Niteroi, 140 consent forms were
distributed to students in five sixth- and seventh-grade
classes; 66 students (47%) completed the FFQ and the three
food records. The adolescents residing in Duque de Caxias

were selected from a sample included in a population-based


study (14). Because it is a low income area where only one
fourth of families received more than one official minimum
wage per capita per month (14), the selection process oversampled adolescents from families whose monthly income
was above three minimum wages (about $400). This strategy intended to select individuals supposedly able to acquire
the food items listed in the FFQ. In this area, 110 adolescents were selected, and 65 (59%) completed the FFQ and
the three food records. Finally, in the private school in Rio
de Janeiro, consent forms were distributed to 94 high school
students. Of these, 38 (40%) completed the FFQ and the
three food records.
The Development of the Food Frequency Questionnaire
The food items listed in the semiquantitative FFQ were
based on the food records of 430 public school students
between ages 12 and 18 years from Niteroi. The most
cited items (at least 15 times) and those that together
summed 95% of the reported energy, macronutrients,
cholesterol, iron, calcium, and vitamins A and C intakes
were included in the questionnaire (15). The food list
included 90 items. Typical reference portions were used
for items like bread, bananas, oranges, apples, or eggs; for
other items, the portion sizes used in the FFQ were the
most frequent measurements (mode) reported in the food
records.
The FFQ was designed in the vertical format and included the following options to reporting the frequencies
of food intake: less than once a month or never, 1 to 3
times per month, once a week, 2 to 4 times per week,
5 to 6 times per week, once a day, 2 to 3 times per
day, and four or more times a day. To avoid overreporting of food intake, food frequency choices varied across
the questionnaire items according to the mean intake
frequencies observed in an earlier population-based study
of adolescents in Rio de Janeiro (16). All options were
associated to 14 food items that were typically consumed
in multiple occasions in a day, for example, milk, coffee,
rice, and beans. Eighteen food items that were consumed
once or twice a day, for example yogurt, cheese, and egg,
were connected to seven frequency choices varying from
less than once a month or never to 2 or more times per
day. The remaining 58 foods items were linked to five
frequency options that varied from less than once a
month or never to five or more times per week. Food
intake was reported for the 6 months preceding the interview. The FFQs reproducibility had already been analyzed and the FFQ was considered to be precise (17).
Food Records
The food records required the adolescents to write down
every food they consumed on 3 nonconsecutive days: 2
weekdays and 1 weekend day. Their notes also included
dishes, recipes, ingredients, measurements based on
household utensils or volume and mass units, and the
time and place of meals. To improve the quality of the
recorded information, on the following day of the register,
the records were reviewed by a nutritionist along with the
adolescents. The reviewing process included questioning
about commonly missing items (like sugar, butter, can-

August 2010 Journal of the AMERICAN DIETETIC ASSOCIATION

1171

dies, and beverages), clarifying about the kind of milk


(full fat or fat free), bread (whole or refined flour) or soft
drinks (regular or low energy), plus clearing up illegible
or nonacknowledged terms.
Data Treatment
The data reported in the FFQ were transformed into daily
frequencies; the frequency option once a day was coded
as one and the other options were proportionally associated with the unit. For example, for the items that were
consumed 2 to 3 times a day, the daily frequency was
2.5 (which is obtained from the mean of 2 and 3); for items
consumed 2 to 4 times per week, the daily frequency
was 0.43 (estimated by the formula: {[(24)/2]/7}).
Nutwin software (version 1.5, 2005, Center of Studies
on Informatics in Health, Federal University of So
Paulo, So Paulo, Brazil), which is based on the US Department of Agriculture nutrient database, was used to
estimate the daily intake of energy, carbohydrates, protein, lipids, fiber, and selected micronutrients (calcium,
iron, and phosphorus). If a food was not included in the
softwares food composition table, the centesimal composition was obtained from the Brazilian Food Composition
Table (18). Standard recipes and serving sizes were used
to estimate the nutritional composition of preparations
that were not included in the software database.
Data from the three food records were used to estimate
within- and between-person variances according to the
method proposed by Iowa State University (19,20) using
the Software for Intake Distribution Estimation (PC-Side
version 1.0, Iowa State University, Department of Statistics, Center for Agricultural and Rural Development,
Ames). The mean energy and nutrients intake estimated
from the food records were deattenuated considering
within- and between-subjects variances ratio. The estimated intake of nutrients was adjusted for total energy
intake by the residual method for both FFQ and food
record (21). Calibration regression models were performed with raw food record data.
The Kolmogorov-Smirnov test was used to verify data
symmetry. Log transformations were performed for
Bland-Altman analyses (22).
Validation. For the validation study, the energy and nutrient intakes estimated by the FFQ and the food record
were categorized into quartiles. Weighted values were
calculated to evaluate the agreement between the two
methods. The is a measurement of the agreement between two observations on the same subject. The
weighted is calculated when there are more than two
categories for each observation under analysis; in this
case there were four categories (quartiles). To evaluate
the degree of agreement, the values could be interpreted by taking into account the following limits: 0.00 to
0.20slight agreement, 0.21 to 0.40fair agreement,
0.41 to 0.60moderate agreement, 0.61 to 0.80substantial agreement, and 0.81 to 1.00almost perfect agreement (23,24).
The proportion of subjects categorized in the same
quartile by both methods (agreement), in contiguous
quartiles (adjacent agreement), and in opposite quartiles
(disagreement) were estimated. Pearson correlation coefficients (r) were estimated for the energy and nutrient

1172

August 2010 Volume 110 Number 8

intakes obtained by both methods, after transforming the


data to achieve nonskewed distributions.
To apply the Bland-Altman method (22), the difference
(FFQ minus food record) and the average [(FFQfood
record)/2] of the two methods were calculated. The mean
difference provided the mean agreement, which is given
as a percentage (since the exponential of a difference is a
ratio). The limits of agreement (LOA) were calculated
(mean difference1.96the standard deviation of the difference distribution) (Equation 1). The LOA define the
boundaries within which 95% of the difference between
the methods is expected to fall, and were calculated as:
Equation 1LOAD1.96SDD
Where: Dthe mean difference between the two methods
(FFQ minus mean of the three food records), and
SDDstandard deviation of the differences between the
two methods.
If there is complete agreement between the methods, the
mean agreement is 100%. A mean agreement of 120% indicates that, on average, the FFQ estimations were 1.2 times
greater than the food record estimates. In addition, limits of
agreement equal to 50% to 200% indicate that 95% of all
subjects FFQ estimates were between half and two times
their food record estimates. In this study, limits of agreement between 50% and 200% were considered suitable (25).
To examine whether the agreement between the methods varied with the magnitude of energy and nutrient
intake, the differences between the methods were plotted
against their mean. In addition, linear regression models
formally tested this supposition by fitting the regression
line of the differences. A regression model was performed
having the differences between the methods as the dependent variables and the FFQ and food record means as
independent variables (For example, the equation for energy intake is: FFQenergyFood Recordenergy). A significant slope () in the regression line (H00; .05)
indicated that agreement between the methods varied
according to the magnitude of intake (25,26).
The analyses were performed for both raw data and for
energy-adjusted and deattenuated data.
Calibration. Calibration involves the estimation of a factor
that associates the data obtained from the reference
method (food record) with the data obtained from the
tested instrument (FFQ) (27). A multivariate linear regression was performed in which the energy and nutrient
intakes estimated by the food record were the dependent
variables and the estimates based on the FFQ were the
independent variables. The regression models also included sex and age as independent variables. To perform
the calibration, the intake of nutrients was adjusted by
energy intake by the residual method (21). The regression
constant () and the regression line slope () coefficients
were estimated, and the latter coefficient was identified
as the calibration factor. The calibrated values for each
nutrient were estimated based on and coefficients
using Equation 2:
Calibrated valueQ
Where Q is the estimated energy and nutrients intakes
according to the FFQ.

Table 1. Agreement (weighted ) and cross-classification of quartiles of energy and nutrient


intakes of adolescents living in the metropolitan area of Rio de Janeiro, Brazil, during
2005-2006 (N169), estimated from food frequency questionnaire and the mean of three
food records (raw, deattenuated and energy-adjusted data)

Nutrient
Raw data
Energy
Protein
Total fat
Carbohydrate
Calcium
Iron
Phosphorus
Fiber
Mean
Deattenuated
Protein
Total fat
Carbohydrate
Calcium
Iron
Phosphorus
Fiber
Mean

Weighted
(95% confidence
interval)

Exact
agreement
(%)

0.43 (0.30-0.56)
34
0.38 (0.25-0.52)
40
0.27 (0.13-0.41)
29
0.44 (0.31-0.57)
37
0.40 (0.27-0.53)
40
0.28 (0.15-0.42)
32
0.35 (0.22-0.48)
36
0.33 (0.19-0.47)
35
0.36
35
and energy-adjusted data
0.19 (0.04-0.34)
35
0.26 (0.12-0.40)
31
0.16 (0.02-0.31)
32
0.44 (0.33-0.55)
31
0.19 (0.05-0.33)
28
0.20 (0.05-0.35)
34
0.39 (0.26-0.52)
34
0.26
32

Means and standard deviations were calculated for raw


energy intake and for the energy-adjusted nutrient intakes estimated from the mean of the three food records,
the FFQ, and the calibrated FFQ.
For the validity and calibration estimators analyzed,
the evaluation of statistical significance considered the P
values (0.05) and the 95% confidence intervals.
RESULTS
The study group included 69 boys (41%) and 100 girls
(59%) between the ages of 12 and 19 years. The mean age
was 15.41.94 years and 44% (n74) of the adolescents
were younger than age 15 years. There were no difference
in the age distribution of boys and girls.
Normal distribution was observed for raw energy and
energy-adjusted nutrient intakes, with the exception of
iron intake.
Validation
The weighted values for the raw data ranged from 0.28
(iron) to 0.44 (carbohydrates) and were statistically significant for energy and nutrients intakes. The weighted
values reduced after data deattenuation and adjustment
for total energy intake, except for calcium and fiber. Nevertheless, the statistical significance was maintained
(Table 1).
For the raw data, the exact agreement in the quartile
categorization varied from 29% (total fat) to 40% (protein
and calcium) (mean 35%); the exact agreement added to
adjacent agreement varied from 71% (iron) to 86% (car-

Exact
agreementadjacent
(%)

Disagreement
(%)

82
78
76
86
78
71
75
76
78

18
22
24
14
22
29
25
24
22

69
72
66
80
69
72
79
72

31
28
34
20
31
28
21
28

bohydrates) (mean 78%); and the disagreement mean was


22%. For deattenuated and energy-adjusted data, the
exact agreement mean was 32% and the exact agreement
added to adjacent agreement mean was 72% (Table 1).
For raw data, the mean agreement according to the
Bland-Altman method ranged from 120% (protein) to
179% (fiber) and the agreement between the two methods
estimations varied according to the intake magnitude for
energy, protein, and carbohydrates. For deattenuated
and energy-adjusted data, the mean agreement ranged
from 62% (iron) to 143% (calcium) and varied with the
intake magnitude for carbohydrates and iron. For raw
data, the Pearson correlation coefficients ranged from
0.33 (total fat) to 0.46 (carbohydrates) and from 0.17
(iron) to 0.47 (calcium and fiber) for deattenuated and
energy-adjusted data. The correlation coefficients were
significant except for deattenuated and energy-adjusted
iron intake (Table 2).
Calibration
Because coefficients related to sex were significant in the
linear regression models, the calibration analyses were
performed separately for boys and girls. Coefficients for
age were not significant in the regression analysis and
the models did not include this variable.
Table 3 shows the coefficients of the calibration regression according to sex. For boys, the calibration factor
ranged from 0.15 (energy) to 0.48 (protein); for girls, it
ranged from 0.14 (iron) to 0.47 (calcium and fiber) (Table 3).
The calibrated means estimated for the FFQ were similar to

August 2010 Journal of the AMERICAN DIETETIC ASSOCIATION

1173

Table 2. Mean agreement and 95% limits of agreement (LOA) and Pearson correlation
coefficients between the food frequency questionnaire (FFQ) and the mean of three food
records (raw, deattenuated, and energy-adjusted data) submitted by adolescents living in the
Rio de Janeiro, Brazil, metropolitan area during 2005-2006 (N169)
Nutrienta

Mean agreement
(%)

Raw
Energy
137
Protein
120
Total fat
132
Carbohydrate
150
Calcium
163
Iron
139
Phosphorus
141
Fiber
179
Deattenuated and energy-adjusted
Protein
124
Total fat
110
Carbohydrate
140
Calcium
143
Iron
62
Phosphorus
133
Fiber
129

95% LOAb
(%)

P valued

62-304
52-277
49-353
67-335
47-571
56-343
59-334
66-482

0.270
0.221
0.117
0.226
0.084
0.179
0.189
0.085

0.007
0.043
0.292
0.017
0.406
0.100
0.074
0.401

85-181
78-155
111-178
70-289
45-87
90-198
72-231

0.124
0.129
0.344
0.043
1.152
0.117
0.128

0.293
0.257
0.004
0.639
0.001
0.310
0.176

re
0.41
0.34
0.33
0.46
0.40
0.34
0.36
0.41
0.26
0.31
0.27
0.47
0.17*
0.29
0.47

The intake of energy and nutrients intake were log-transformed for the agreement analysis.
LOA determined as mean difference1.96standard deviation of the differences.
c
Slope of average of methods regressed on difference between methods (Ho: 0; .05).
d
Statistical significance of .
e
Pearson correlation coefficients.
*P0.05.
b

Table 3. Calibration parameters ( and ) and 95% confidence


interval (95% CI) for energy and nutrient intake according to sex, for
adolescents living in the Rio de Janeiro, Brazil, metropolitan area
during 2005-2006
Nutrienta
Boys (n69)
Energyc (kcal)
Protein (g)
Total fat (g)
Carbohydrate (g)
Calcium (mg)
Iron (mg)
Phosphorus (mg)
Fiber (g)
Girls (n100)
Energy (kcal)
Protein (g)
Total fat (g)
Carbohydrate (g)
Calcium (mg)
Iron (mg)
Phosphorus (mg)
Fiber (g)

(95% CI)

b (95% CI)

1,994 (1,442-2,547)
34 (10-58)
35 (15-56)
113 (35-223)
270 (92-448)
8.6 (4.9-12.3)
744 (443-1,045)
7.3 (3.6-10.9)

0.15 (0.02-0.31)
0.48 (0.21-0.76)
0.31 (0.07-0.55)
0.36 (0.12-0.60)
0.32 (0.12-0.52)
0.17 (0.08-0.42)
0.14 (0.09-0.38)
0.28 (0.08-0.47)

1,233 (945-1,521)
55 (33-78)
37 (24-50)
194 (129-260)
193 (65-321)
8.9 (5.5-12.4)
483 (245-720)
3.3 (0.1-6.7)

0.25 (0.16-0.34)
0.24 (0.02-0.49)
0.30 (0.15-0.45)
0.18 (0.04-0.33)
0.47 (0.33-0.60)
0.14 (0.10-0.38)
0.36 (0.18-0.54)
0.47 (0.28-0.67)

a
Energy-adjusted nutrients intakes estimated from the mean of three food records
(dependent variable) and from the food frequency questionnaire (independent variable).
b
Calibration factor.
c
Raw energy intake estimated from the mean of three food records (dependent
variable) and from the food frequency questionnaire (independent variable).

1174

August 2010 Volume 110 Number 8

the means estimated for the food record but showed considerably reduced standard deviations (Table 4).
DISCUSSION
Various analytical procedures were used to estimate the
relative validity of an FFQ designed for adolescents living
in Rio de Janeiro. The results showed that the tested FFQ
is suitable as a tool for ranking adolescents energy and
nutrient intake. Also, the calibration factors should be
used to correct the estimation of food and nutrient intake
means as well as the relationship between dietary intake
and health outcomes. Although the study sample was not
probabilistically selected, adolescents from different geographic regions and socioeconomic strata were included,
suggesting that the sample probably represents the characteristics of adolescents from the Rio de Janeiro metropolitan area.
The agreement in quartile categorization was similar
to that observed by Slater and colleagues (9) and lower
than the estimated by Vereecken and Maes (28) in their
evaluation of the validity of an FFQ designed for Belgian
adolescents. As observed for the correlation coefficients,
the suffered reduction after data deattenuation and
energy-adjustment. The presence of systematic errors
cannot be excluded (4) because adolescents have difficulty
estimating the amount of food intake. Therefore, the use
of aiding tools, like photographs and models, is recommended to reduce bias in the reporting of food intake
portions.

Table 4. Meanstandard deviation (SD) for energy and nutrients intake estimated from the
food frequency questionnaire (FFQ), 3 days of food records (FR), and calibrated food frequency
questionnaire (calibrated FFQ) according to sex, for adolescents living in the Rio de Janeiro,
Brazil, metropolitan area during 2005-2006
Energya/nutrientsb

FFQ

Boys (n69)
Energy (kcal)
Protein (g)
Total fat (g)
Carbohydrate (g)
Calcium (mg)
Iron (mg)
Phosphorus (mg)
Fiber (g)
Girls (n100)
Energy (kcal)
Protein (g)
Total fat (g)
Carbohydrate (g)
Calcium (mg)
Iron (mg)
Phosphorus (mg)
Fiber (g)

4 meanSD 3
3,1851,205
2,456821
2,472181
8511
7514
755
8512
6213
624
45537
27839
27713
833271
536242
53687
14.72.3
11.02.4
11.10.4
1,288198
930189
92428
18.24.3
12.33.7
12.41.2
2,9501,260
8713
9013
44138
892333
14.42.5
1,300215
16.74.3

FRc

1,980648
7817
6410
27629
608272
10.93.0
946208
11.24.7

Calibrated FFQ

1,970315
763
644
2747
612156
10.90.3
95177
11.22.0

a
Calibration regression for energy: raw energy intake estimated from the mean of three food records (dependent variable)
and from the FFQ (independent variable).
b
Calibration regression for nutrients: energy-adjusted nutrients intake estimated from the mean of three food records
(dependent variable) and from the FFQ (independent variable).
c
Mean of three food records.

The FFQ overestimated the food intake when compared


to the food record estimates. The mean agreement and
the LOA between the tested FFQ and the reference
method obtained according to the Bland-Altman procedures were acceptable for deattenuated and energy-adjusted data, except for iron. Lietz and colleagues (29)
analyzed the validity of an FFQ for Scottish adolescents
and found that the FFQ overestimated the intake of nutrients when compared to the mean of 7 days of weighed
food records; the authors concluded that the agreement
between the two instruments was poor.
In our study, for deattenuated and energy-adjusted
data, the agreement estimated by the Bland-Altman
method varied with the intake magnitude for energy and
iron. Cullen and colleagues (30) validated an FFQ for
American adolescents, although the authors found that
the agreement estimated by the same procedure was
independent of the intake magnitude.
The correlation coefficients between the FFQ and the
reference method (the mean of three food records) showed
an average reduction of 22% for deattenuated and energy-intake-adjusted data. Slater and colleagues (9) also
found that the correlation coefficients were reduced by
50% when the data were adjusted for energy intake.
Usually, in FFQ validation studies, the estimated correlation coefficients vary from 0.5 to 0.7 (31). Molag and
colleagues (32) reviewed 42 FFQ validation studies and
found that for most nutrients, the correlation coefficients
were significantly higher when the reference method was
applied for 8 to 14 days rather than for 1 to 7 days.
Applying the reference method for 15 days or more pro-

vided no improvement in the correlation coefficients (32).


In our study, the reference method was replicated for 3
nonconsecutive days; therefore, it is possible that dietary
intake estimates would be more accurate if a greater
number of food records were collected over a longer period
of time.
Watson and colleagues (33) tested the validity of an
FFQ developed for children and adolescents aged 9 to 16
years having as reference a set of four food records. The
authors obtained Pearson correlation coefficients ranging
from 0.03 (retinol) to 0.56 (magnesium) for transformed,
energy-adjusted, and deattenuated nutrient data. The
agreement for the categorization in quintiles was ascertained by the weighted , which varied from 0.12 (retinol)
to 0.45 (iron). In that study, the agreement between FFQ
and the food record assessed by the Bland-Altman procedures also varied with the intake magnitude for most of
nutrients. The authors conclude that the tested FFQ
overestimated nutrients intake and is not appropriate to
assess absolute intake; however, the FFQ ranked individuals with reasonable accuracy.
The FFQ calibration revealed the extent of the instruments measurement errors. The process also offered the
coefficients necessary to correct both the mean nutrient
intake values that were estimated by the FFQ and measures of the association between disease and diet. The
calibration of an FFQ designed for adolescents allowed us
to obtain mean energy and nutrient intakes that were
similar to those obtained from the reference method; however, there was a considerable reduction in the data dispersion. The calibration factor is close to one when the

August 2010 Journal of the AMERICAN DIETETIC ASSOCIATION

1175

means obtained through the FFQ are similar to those


estimated with the reference method. Yet, the calibration
factor is usually attenuated and 1 (31). According to
Kaaks and Riboli (34), a calibration factor is useful in
correcting biases in food intake estimates, particularly
when dietary intake is the exposition variable in a study
of the association between diet and disease.
This is the first study that calibrates dietary intake
data from an FFQ designed specifically for adolescents
living in the Rio de Janeiro metropolitan area. Other
studies that calibrated FFQ for adolescents in Brazil obtained similar results. Voci and colleagues (12) estimated
calibration factors varying from 0.07 (iron) to 0.40 (vitamin C), and Slater and colleagues (11) estimated calibration factors for energy (0.89), carbohydrates (0.41),
total fat (0.22), and protein (0.20). The reduction in the
distribution dispersion of the calibrated data has been
reported by other researchers (11,12,31,35,36); this reduction is related to the linear relationship between the
FFQ and the dietary records.
It is often assumed that the reference method should
reflect the true intake and that the FFQ measurement
errors (which are mainly systematic errors) should be
independent of the reference method (food record) errors (37). Actually, according to Kaaks and colleagues
(37), the assumption of independent errors between the
methods of dietary assessment does not occur. The
authors report that subjects with obesity tend to underestimate their food intake in comparison to subjects
with normal weight and observe that this underestimation is considered a random error that can have an
effect on the quality of FFQ, food records, and 24-hour
recall intake data (37).
Studies on the calibration of dietary intake estimates
have been used in different settings, particularly in the
European Prospective Study into Cancer and Nutrition
(38-42). These studies examined the risk of cancer associated to dietary factors in a multicenter cohort study
carried out in 10 European countries. Data on dietary
intake were collected by validated country-specific FFQ
and calibration studies allowed the correction of the estimated food intake and hazard ratios.
Underreporting is the main bias that interferes with
the food intake data obtained from food records (6,7).
Chinnock (6) analyzed data from 60 Costa Rican adults
and observed that food intake data estimated from food
records were underestimated for energy, nutrients, and
food groups when compared to the results obtained from
a weighed food record. Anderson and colleagues (7) observed that energy intake estimates were 34% lower than
the estimated energy expenditure among Norwegian adolescents. Possible underreporting in the food records
and/or overreporting in the FFQ may have influenced the
results observed in this study.
Despite being recognized as methods that provide rich
information on food intake, the food records and the 24-hour
recalls inadequately measure foods episodically consumed
(35). On the other hand, recent proposals for improving
dietary intake assessment methods have recommended
the use of 24-hour recalls combined with an FFQ (43).
This new statistical modeling that associated 24-hour
recalls and FFQ (44) brought new light on the FFQ, which
can be applied in national wide surveys. Therefore, it is

1176

August 2010 Volume 110 Number 8

still important to study the advantages and the limitations of FFQ in assessing food intake in the diverse population groups. Furthermore, FFQ is the method of choice
in case-control studies because it is appropriate for retrospective information. Finally, in cohort studies, information on specific foods intake obtained with FFQs are
comparable over an extended period.
As other studies have shown (45-47), this study indicates that the tested FFQ can be used to rank food and
nutrients intake (except iron) of adolescents living in the
investigated area. Furthermore, the estimated FFQ calibration factors allow the correction of energy and nutrient intakes and are needed to estimate population intakes means.
STATEMENT OF POTENTIAL CONFLICT OF INTEREST:
No potential conflict of interest was reported by the authors.
FUNDING/SUPPORT: The study was funded by the
National Council of Technological and Scientific Development (grant no. 506336/2004-2).
ACKNOWLEDGMENTS: The authors thank Rosely
Sichieri, PhD, for her support in the study conception, and
for data analysis and interpretation. The authors also thank
Gloria Valeria da Veiga, PhD, for providing access to the
dietary data used in the elaboration of the FFQ.
References
1. Beaton GH. Approaches to analysis of dietary data: Relationship
between planned analyses and choice of methodology. Am J Clin Nutr.
1994;59(suppl):253S-261S.
2. Margetts BM, Nelson M, eds. Design Concepts in Nutritional Epidemiology. Oxford, UK: Oxford University Press; 1997.
3. Gibson RS. Measuring food consumption of individuals. In: Gibson
RS, ed. The Principles of Nutritional Assessment. Oxford, UK: Oxford
University Press; 2005:41-64.
4. Willett WC. Nutritional Epidemiology. 2nd ed. New York, NY: Oxford
University Press; 1998.
5. Barbosa KB, de Lima Rosado LE, Franceschini Sdo C, Priore SE.
Dietary records used for food consumption evaluation in adolescents:
Comparison among methods. Arch Latinoam Nutr. 2007;57:43-50.
6. Chinnock A. Validation of an estimated food record. Public Health
Nutr. 2006;9:934-941.
7. Andersen LF, Pollestad ML, Jacobs DR Jr, Lovo A, Hustvedt BE.
Validation of a pre-coded food diary used among 13-year-olds: Comparison of energy intake with energy expenditure. Public Health Nutr.
2005;8:1315-1321.
8. Carroll RJ, Pee D, Freedman LS, Brown CC. Statistical design of
calibration studies. Am J Clin Nutr. 1997;65(suppl):1187S-1189S.
9. Slater B, Philippi ST, Fisberg RM, Latorre MR. Validation of a semiquantitative adolescent food frequency questionnaire applied at a
public school in Sao Paulo, Brazil. Eur J Clin Nutr. 2003;57:629-635.
10. Carvalho CMRG, Nogueira AMT, Teles JBM, Paz SMR, Sousa RML.
Food consumption by adolescents enrolled in a private high school in
the city of Teresina, Piau, Brazil. Brazil J Nutr. 2001;14:85-93.
11. Slater B, Marchioni DM, Voci SM. Use of linear regression for correction of dietary data. Rev Saude Publica. 2007;41:190-196.
12. Voci SM, Slater B, Silvia MV, Marchioni DML, Latorre MRDO. Calibration Study of Food Frequency Questionnaire for Adolescents
(AFFQ). Revista Cincia & Sade Coletiva Web site. http://www.
abrasco.org.br/cienciaesaudecoletiva/artigos/artigo_int.php?id_artigo
3161. Accessed March 26, 2010.
13. Sichieri R, Everhart JE. Validity of a Brazilian food frequency questionnaire against dietary recalls and estimated energy intake. Nutr
Res. 1998;18:1649-1659.
14. Salles-Costa R, Pereira RA, Vasconcellos MT, Veiga GV, Marins VM,
Jardim BC. Association between socioeconomic factors and food insecurity: A population-based study in the Rio de Janeiro metropolitan
area, Brazil. Brazil J Nutr. 2008;21:99-109.
15. Block G, Hartman AM, Dresser CM, Carroll MD, Gannon J, Gardner
L. A data-based approach to diet questionnaire design and testing.
Am J Epidemiol. 1986;124:453-469.

16. Andrade RG, Pereira RA, Sichieri R. Food intake in overweight and
normal-weight adolescents in the city of Rio de Janeiro. Cad Saude
Publica. 2003;19:1485-1495.
17. Araujo MC, Ferreira DM, Pereira RA. Reliability of a semi-quantitative food frequency questionnaire designed for adolescents from the
Rio de Janeiro Metropolitan Area, Brazil. Cad Saude Publica. 2008;
24:2775-2786.
18. Tabela Brasileira de Composio de Alimentos - TACO [Brazilian
Food Composition Data]. Campinas, So Paulo, Brazil: NEPA/UNICAMP (Ncleo de Estudos e Pesquisas em Alimentao/Universidade
Estadual de Campinas [Nucleus of Studies and Researches of Food/
State University of Campinas]); 2006.
19. Nusser SM, Carriquiry AL, Dodd KW, Fuller WA. A semi-parametric
transformation approach to estimating usual intake distributions.
J Am Stat Assoc. 1996;91:1440-1449.
20. Nusser SM, Fuller WA, Guenther PM. Estimating usual dietary intake distributions: Adjusting for measurement error and no normality
in 24-hour food intake data. In: Lyberg L, Biemer P, Collins M, Leeuw
ED, Dippo C, Schwarz N, Trewin D, eds. Survey Measurement and
Process Quality. New York, NY: Wiley; 1997:689-709.
21. Willett WC, Howe GR, Kushi LH. Adjustment for total energy intake
in epidemiologic studies. Am J Clin Nutr. 1997;65(suppl):1220S1228S.
22. Bland JM, Altman DG. Statistical methods for assessing agreement
between two methods of clinical measurement. Lancet. 1986;1:307310.
23. Fraile BB. Epidemiologa clnica aplicada a la toma de decisiones en
medicina [Clinical epidemiology applied to decision making in medicine]. Anales del Sistema Sanitario de Navarra [Annals of Navarra
Health System]. Government of Navarra, Spain, Department of
Health; 2001. http://www.navarra.es/NR/rdonlyres/B31A650F-39524E42-A4C2-1439ED42F72D/147800/Epidemiologia_clinica.pdf. Accessed July 6, 2010.
24. Landis JR, Koch GG. The measurement of observer agreement for
categorical data. Biometrics. 1977;33:159-174.
25. Ambrosini GL, van Roosbroeck SAH, Mackerras D, Fritschi L, de
Klerk NH, Musk W. The reliability of ten-year dietary recall: Implications for cancer research. J Nutr. 2003;133:2663-2668.
26. Bland JM, Altman DG. Measuring agreement in method comparison
studies. Stat Methods Med Res. 1999;8:135-160.
27. Rosner B, Willett WC, Spiegelman D. Correction of logistic regression
relative risk estimates and confidence intervals for systematic withinperson measurement error. Stat Med. 1989;8:1051-1069.
28. Vereecken CA, Maes L. A Belgian study on the reliability and relative
validity of the Health Behaviour in School-Aged Children food-frequency questionnaire. Public Health Nutr. 2003;6:581-588.
29. Lietz G, Barton KL, Longbottom PJ, Anderson AS. Can the EPIC
food-frequency questionnaire be used in adolescent populations? Public Health Nutr. 2002;5:783-789.
30. Cullen KW, Watson K, Zakeri I. Relative reliability and validity of the
Block Kids Questionnaire among youth aged 10 to 17 years. J Am Diet
Assoc. 2008;108:862-866.
31. Kaaks R, Riboli E, van Staveren W. Calibration of dietary intake
measurements in prospective cohort studies. Am J Epidemiol. 1995;
142:548-556.
32. Molag ML, de Vries JH, Ocke MC, Dagnelie PC, van den Brandt PA,
Jansen MC. Design characteristics of food frequency questionnaires
in relation to their validity. Am J Epidemiol. 2007;166:1468-1478.
33. Watson J, Collins C, Sibbritt D, Dibley M, Garg M. Reproducibility
and comparative validity of a food frequency questionnaire for
Australian children and adolescents. Int J Behav Nutr. Phys Act.
2009;6:62.
34. Kaaks R, Riboli E. Validation and calibration of dietary intake measurements in the EPIC project: Methodological considerations. European Prospective Investigation into Cancer and Nutrition. Int J Epidemiol. 1997;26(suppl 1):S15-S25.
35. Hoffmann K, Kroke A, Klipstein-Grobusch K, Boeing H. Standardization of dietary intake measurements by nonlinear calibration using
short-term reference data. Am J Epidemiol. 2002;156:862-870.
36. Kynast-Wolf G, Becker N, Kroke A, Brandstetter BR, Wahrendorf J,
Boeing H. Linear regression calibration: Theoretical framework and
empirical results in EPIC, Germany. Ann Nutr Metab. 2002;46:2-8.
37. Kaaks R, Ferrari P, Ciampi A, Plummer M, Riboli E. Uses and
limitations of statistical accounting for random error correlations, in
the validation of dietary questionnaire assessments. Public Health
Nutr. 2002;5:969-976.
38. Buckland G, Agudo A, Lujn L, Jakszyn P, Bueno-de-Mesquita HB,
Palli D, Boeing H, Carneiro F, Krogh V, Sacerdote C, Tumino R,
Panico S, Nesi G, Manjer J, Regnr S, Johansson I, Stenling R,

39.

40.

41.

42.

43.

44.

45.

46.

47.

Sanchez MJ, Dorronsoro M, Barricarte A, Navarro C, Quirs JR,


Allen NE, Key TJ, Bingham S, Kaaks R, Overvad K, Jensen M, Olsen
A, Tjnneland A, Peeters PH, Numans ME, Ock MC, Clavel-Chapelon F, Morois S, Boutron-Ruault MC, Trichopoulou A, Lagiou P,
Trichopoulos D, Lund E, Couto E, Boffeta P, Jenab M, Riboli E,
Romaguera D, Mouw T, Gonzlez CA. Adherence to a Mediterranean
diet and risk of gastric adenocarcinoma within the European
Prospective Investigation into Cancer and Nutrition (EPIC) cohort
study. Am J Clin Nutr. 2010;91:381-390.
Bchner FL, Bueno-de-Mesquita HB, Ros MM, Kampman E, Egevad
L, Overvad K, Raaschou-Nielsen O, Tjnneland A, Roswall N, ClavelChapelon F, Boutron-Ruault MC, Touillaud M, Chang-Claude J,
Kaaks R, Boeing H, Weikert S, Trichopoulou A, Lagiou P, Trichopoulos D, Palli D, Sieri S, Vineis P, Tumino R, Panico S, Vrieling A,
Peeters PH, van Gils CH, Lund E, Gram IT, Engeset D, Martinez C,
Gonzalez CA, Larraaga N, Ardanaz E, Navarro C, Rodrguez L,
Manjer J, Ehrnstrm RA, Hallmans G, Ljungberg B, Allen NE, Roddam AW, Bingham S, Khaw KT, Slimani N, Boffetta P, Jenab M,
Mouw T, Michaud DS, Kiemeney LA, Riboli E. Consumption of vegetables and fruit and the risk of bladder cancer in the European
Prospective Investigation into Cancer and Nutrition. Int J Cancer.
2009;125:2643-2651.
Suzuki R, Allen NE, Key TJ, Appleby PN, Tjnneland A, Johnsen NF,
Jensen MK, Overvad K, Boeing H, Pischon T, Kaaks R, Rohrmann S,
Trichopoulou A, Misirli G, Trichopoulos D, Bueno-de-Mesquita HB,
van Duijnhoven F, Sacerdote C, Pala V, Palli D, Tumino R, Ardanaz
E, Quirs JR, Larraaga N, Snchez MJ, Tormo MJ, Jakszyn P,
Johansson I, Stattin P, Berglund G, Manjer J, Bingham S, Khaw KT,
Egevad L, Ferrari P, Jenab M, Riboli E. A prospective analysis of the
association between dietary fiber intake and prostate cancer risk in
EPIC. Int J Cancer. 2009;124:245-249.
Gonzlez CA, Pera G, Agudo A, Bueno-de-Mesquita HB, Ceroti M,
Boeing H, Schulz M, Del Giudice G, Plebani M, Carneiro F, Berrino F,
Sacerdote C, Tumino R, Panico S, Berglund G, Simn H, Hallmans G,
Stenling R, Martinez C, Dorronsoro M, Barricarte A, Navarro C,
Quiros JR, Allen N, Key TJ, Bingham S, Day NE, Linseisen J, Nagel
G, Overvad K, Jensen MK, Olsen A, Tjnneland A, Bchner FL,
Peeters PH, Numans ME, Clavel-Chapelon F, Boutron-Ruault MC,
Roukos D, Trichopoulou A, Psaltopoulou T, Lund E, Casagrande C,
Slimani N, Jenab M, Riboli E. Fruit and vegetable intake and the risk
of stomach and oesophagus adenocarcinoma in the European Prospective Investigation into Cancer and Nutrition (EPIC-EURGAST). Int J
Cancer. 2006;118:2559-2566.
Gonzlez CA, Jakszyn P, Pera G, Agudo A, Bingham S, Palli D,
Ferrari P, Boeing H, del Giudice G, Plebani M, Carneiro F, Nesi G,
Berrino F, Sacerdote C, Tumino R, Panico S, Berglund G, Simn H,
Nyrn O, Hallmans G, Martinez C, Dorronsoro M, Barricarte A,
Navarro C, Quirs JR, Allen N, Key TJ, Day NE, Linseisen J, Nagel
G, Bergmann MM, Overvad K, Jensen MK, Tjonneland A, Olsen A,
Bueno-de-Mesquita HB, Ocke M, Peeters PH, Numans ME, ClavelChapelon F, Boutron-Ruault MC, Trichopoulou A, Psaltopoulou T,
Roukos D, Lund E, Hemon B, Kaaks R, Norat T, Riboli E. Meat intake
and risk of stomach and esophageal adenocarcinoma within the European Prospective Investigation Into Cancer and Nutrition (EPIC).
J Natl Cancer Inst. 2006;98:345-354.
Subar AF, Dodd KW, Guenther PM, Kipnis V, Midthune D, McDowell
M, Tooze JA, Freedman LS, Krebs-Smith SM. The food propensity
questionnaire: concept, development, and validation for use as a covariate in a model to estimate usual food intake. J Am Diet Assoc.
2006;106:1556-1563.
Tooze JA, Midthune D, Dodd KW, Freedman LS, Krebs-Smith SM,
Subar AF, Guenther PM, Carroll RJ, Kipnis V. A new statistical
method for estimating the usual intake of episodically consumed foods
with application to their distribution. J Am Diet Assoc. 2006;106:
1575-1587.
Lassale C, Guilbert C, Keogh J, Syrette J, Lange K, Cox DN. Estimating food intakes in Australia: Validation of the Commonwealth
Scientific and Industrial Research Organisation (CSIRO) food frequency questionnaire against weighed dietary intakes. J Hum Nutr
Diet. 2009;6:559-566.
Hjartker A, Andersen LF, Lund E. Comparison of diet measures
from a food-frequency questionnaire with measures from repeated
24-hour dietary recalls. The Norwegian Women and Cancer Study.
Public Health Nutr. 2007;10:1094-1103.
Sasaki S, Matsumura Y, Ishihara J, Tsugane S; JPHC. Validity of a
self-administered food frequency questionnaire used in the 5-year
follow-up survey of the JPHC Study Cohort I to assess dietary fiber
intake: Comparison with dietary records. J Epidemiol. 2003;13(suppl
1):S106-S114.

August 2010 Journal of the AMERICAN DIETETIC ASSOCIATION

1177