Annual Status of Education Report

2005 T0 2010

KEY FEATURES OF ASER

Rukmini Banerji and Wilima Wadhwa

August 2010: New Delhi

This original version of this paper was written for

the National Seminar for Education Statistics,
NUEPA, New Delhi, March 2008.
 2

TABLE OF CONTENTS

• Setting the stage: Why ASER?

o What needs to be measured?

o How should measurements be done?
o What data is currently available to track progress?
o What level of data is needed?
o How can independent assessments be done?

• The ASER Example

• Key Features of ASER

o Sampling
o Coverage
o Household survey questions
o Assessment tasks for basic learning
o School observations
o Time duration
o Partners

• Conclusion

Annexure 1. Learning assessment tasks: ASER 2005 to 2010

Note: ASER reports from 2005 to 2009 can be downloaded from

www.asercentre.org

For more information, write to: contact@asercentre.org

 3

SETTING THE STAGE: WHY ASER?

Are programs in elementary education leading to desired outcomes? Is public expenditure
resulting in progress towards stated goals? In the last two decades, there have been a number of
large scale education sector programs in India. Through the 1990s, the District Primary
Education Programme (DPEP) was implemented in almost half of India’s districts. The following
decade saw the implementation of Sarva Shiksha Abhiyan (SSA), the national program for
universalisation of elementary education.1 The annual budgetary allocation for SSA has risen
steadily over time, increasing from Rs. 4,754 crores in 2004-05 to Rs.9,53,231 crore in 2009-2010.
Since 2004, citizens of India have been paying a 2% education cess on all central taxes to finance
the commitment to universalize quality basic elementary education.2

Sarva Shiksha Abhiyan has the following stated goals:

• All children (6-14) years in India to be enrolled in school.
• All children to complete five years of primary schooling by 2007 and eight years of
elementary school (through Std 8) by 2010. Universal retention to be achieved by 2010.
• Gender and social gaps to be eliminated.
• All children to receive elementary education of satisfactory quality.
• SSA has a special focus on girl's education and children with special needs. SSA also seeks to
provide computer education to bridge the digital divide.

In order to reach these goals, SSA documents specify several desired processes. These include
habitation level planning and decentralized decision making that take into account local
conditions, community participation and involvement of non government organizations.

An important component of SSA functioning is the annual work plan (AWP) process. Following SSA
guidelines and norms, each district in the country prepares its annual operational plan and budget
for elementary education. This district level planning and implementation can be seen as an
important step in the direction of decentralization and local decision making. District level annual
work plans are reviewed at state and national levels before decisions about allocation of funds are
made for the subsequent year.

With high expenditures and ambitious goals, monitoring progress towards the outcomes of
education programs becomes essential. Internal evaluation and self regulation by the government
are necessary for program implementation, but assessment that is independent of the government
is also critical to ensure transparency and accountability. These broad ideas are now widely
accepted not only by people at large but also by policy makers.

The big question, however, is what should be done and how?

The SSA formulation of goals includes time-bound targets for enrollment and retention. So far,
most of the measurement effort by the government has focused on measuring access, enrollment

1
Sarva Shiksha Abhiyan “is an effort to universalize elementary education by community ownership of the
school system. It is a response to the demand for quality basic education all over the country. The SSA
programme is also an attempt to provide an opportunity for improving human capabilities to all children,
through provision of community owned quality education in a mission mode”. (See www.ssa.nic.in)
2
Extracted from Budget papers, Ministry of Finance. 2% primary education cess in overall 3% education cess.
 4

and coverage, inputs, infrastructure and provisions. From the central government to the last
village, annual data on out of school children and school enrollment, physical provision of schools,
classrooms and teachers is collected by the government. These are closely reviewed at the district,
state and national levels.

RTE: The Right to Free and Compulsory Education Act 2009 was passed by the Parliament in August
2009, and after receiving Presidential assent immediately thereafter, it was notified for
implementation from April 1, 2010. The 86th amendment that provides children of India, in the
age group 6 to 14 years, a fundamental right to free and compulsory education was simultaneously
notified the same day.

With its justiciable clauses that makes it compulsory for the state to provide a neighbourhood
school of a prescribed minimum quality to each child within three years, a designated PTR ratio
that has to be maintained in each school, all teachers to attain nationally designated qualifications
within five years, education that is inclusive, nondiscriminatory and of equitable quality for all
children, including those with disabilities, mandatory School Management Committees consisting
mostly of parents, and designation of local authorities. Delivery mechanisms, teacher recruitment
procedures, training institutions, inter-agency coordination, community and parent participation
and monitoring mechanisms shall have to be put in place rapidly to ensure that violations of the
Act are kept to the minimum, or immediately dealt with. The same applies to sections of the Act
that bans admission tests and other screening procedures for all schools, ban on capitation fees,
ban on corporal punishment and tuitions by teachers.

By contrast, the articulation of learning goals for children is not concrete or precise even at the
national level; nor is there a frequent, nationally comparable measurement of student
achievement or children’s academic progress. The central government’s achievement surveys
(carried out by NCERT), done periodically for Std 3 and above, do not generate district level
estimates that can be used as inputs in the annual process of provisions, review and planning.3

What needs to be measured? Given that enrollment levels especially at the primary stage have
climbed well over 90% in most states, it is essential to look beyond, to educational processes and
outcomes. With respect to processes, it is increasingly important to pay attention to a whole host
of issues inside schools that affect attendance, retention and quality of education, especially those
related to teaching and learning. Inputs and processes together create outcomes. And with respect
to outcomes, although there may be many important outcomes of schooling that could be
measured, learning is certainly one of the key ones.

Estimates of learning are needed to understand how children are doing as compared to
expectations. In order to ensure that the base of universal schooling is built on sound and solid
foundations, it is critical that we know how much children are learning in primary school especially
with respect to basic skills like reading, writing and arithmetic. Unless a child learns to read fluently
and confidently by Std 2 or 3, it will be hard for him or her to make adequate educational progress
in any subject and therefore early tracking is needed.

3
The NCERT tests are school based pen and paper achievement tests administered periodically to a national
sample of children in Std III, V and VIII. Publicly available data on midterm assessment results from NCERT
tests report state level not district level results.
 5

ASER uses a simple definition of universalization of elementary education: “Every child in school
and learning well”. Every year, the ASER exercise ensures that we keep the spotlight on children
who are out of school in each district. The “last mile” is often the hardest to cover. Independent
assessments that annually focus on this last group of children out of school will help ensure that
action is taken to educate them. Incidentally, nationwide estimates generated by the independent
study commissioned by the Ministry of Human Resource Development in 2005 closely matched the
ASER 2005 estimates of out of school children for that year.4

In order to bring learning to the centre of all discussions of educational planning and
implementation, ASER focuses on basic learning, especially on the ability to read simple text (up to
Std 2 level) and the capacity to do basic arithmetic operations (at Std 3 or 4 level). To bring
learning to the centre of the stage, it is important not only to have data but also to disseminate
and discuss the findings widely within different layers of the government and among citizens more
broadly. For widespread dissemination and discussion, indicators need to be simple, easy to
understand and interpret. ASER aims to provide such information annually.

How should measurements be done? Unlike in other sectors such as banking, power or telecom
where regulatory authorities have been set up, in the education sector, ongoing monitoring from
the central level is possible but not sufficient.5 Metrics, methods and mechanisms for independent
monitoring have to be generated that allow us to see whether the allocated resources are
adequate, whether they are being applied well and how outlays are translating into outcomes.
Further, these metrics must be comparable and consistent across contexts and across time, so that
citizens can see whether progress is being made. Given constraints of time and resources, the
architecture of the mechanism must be simple; the design scientific; the results that are generated
must be useful locally but should also allow aggregation to appropriate levels.

What data is currently available to track progress? As mentioned earlier, the government
regularly gathers information on enrollment, infrastructure and other inputs from schools and
from household surveys. In a very welcome development, the government now has individual
DISE9 school report cards as well as school indicators aggregated to cluster, block, district and state
level. For a country the size of India it is a big achievement to have data publicly available online
for over 1 million schools in the country.6

The DISE school report cards covers:

• General school information
• Teachers information
• School infrastructure information
• Enrollment and Repeaters
• Incentives
• School Funds ( Received and Expenditure )
• Examination results

This data is available for 2005 onwards. The government also commissions independent

4
See ssa.nic.in/research for details of All India Survey of Out of School Children in 2005. This study was
conducted by SRI (a specialist unit of IMRB).
5
Beginning during DPEP, biannual joint review missions are undertaken by the Ministry of Human Resource
Development, Government of India and donor agencies.
6
http://www.schoolreportcards.in Also see www.dpepmis.org
 6

research studies.7 However, independent, national data is scarce, especially on children’s

achievement.8

What level of data is needed? In elementary education, annual work plans and allocations are
made at the district level. Data on children’s learning at least on basic reading and arithmetic are
needed at district level to plan for improvement. Also for the planning process data is required
starting with habitation level and going to district level.

Districts with high incidence of out of school children are targeted for special focus and provisions.
Similar provisioning and planning is done for districts and blocks where access to schools is poor or
where teachers are still in short supply. The same kind of careful planning needs to be done for
districts where children’s learning levels are particularly low. Data is needed that can be
aggregated upwards from district to state and to national levels.

How can independent assessments be done? Who will participate and at what cost? People of
India are the largest resource that is available. Various initiatives in the area of governance, health,
water, sanitation, which were earlier limited to NGOs are now spreading as “movements” beyond
the confines of small organizations and networks. Successful examples include Loksatta in electoral
reform, Janaagraha in urban governance, CUTS in power reform, MYRADA in watershed programs.
These groups have not only been active in mobilizing grassroots initiatives and piloting programs
on scale but also in significantly shaping policy.

SOCIAL AUDITS :

Social Audit is a tool with which government departments can plan, manage and measure non-
financial activities and monitor both internal and external consequences of the department/
organisation’s social and commercial operations. It is an instrument of social accountability for an
organisation. In other words, Social Audit may be defined as an in-depth scrutiny and analysis of
the working of any public utility vis-à-vis its social relevance Social Audit gained significance
rd
especially after the 73 Amendment of the Constitution relating to Panchayat Raj Institution.

THE ASER EXAMPLE

In India, ASER (Annual Status of Education Report) provides a unique and innovative example of
how to engage citizens in assessing progress towards the goals of elementary education. The focus
of this initiative is primarily on enrollment and basic learning. The design of the exercise is such
that using a common set of survey questions and assessment tasks as well as a common sampling

7
See ssa.nic.in/research
8
Over the past few years, assessment activity in elementary education has been on the rise. In recent years,
states are instituting state wide assessments even in primary grades. For example, Karnataka, Gujarat and
Orissa are among states that have developed an ongoing assessment and tracking system for analyzing
children’s academic progress. Nongovernment organizations like Azim Premji Foundation and Education
Initiatives are also conducting assessment studies in urban and rural schools.
9 DISE: District information system for education. http://www.dise.in/
 7

frame, local groups in each district carry out the data collection and dissemination.9 This exercise
has been conducted every year since 2005.

ASER was conceptualized and designed as a large scale rapid assessment to be done by ordinary
citizens. Therefore the design of ASER incorporates the idea that the exercise must be both simple
and fast, so that any interested citizen of India can participate. The methods and measurements
cannot be very complex, time consuming or costly. Training is imparted to all ASER volunteers.
Typically the duration of training is two days and the actual survey also takes two days.

The biggest challenge for ASER has been how to keep activities simple and fast but at the same
time generate reliable and robust data.

KEY FEATURES OF ASER

A. Sampling:

ASER uses a two stage sample design. In the first stage villages are selected from the Census 2001
village listing. Since villages vary in size, PPS (probability proportional to size) is used to select the
villages to be sampled. In the second stage, households are picked randomly in the selected
villages. The district is the stratum and the sample size is 600 households per district. In the first
stage 30 villages are selected in each district and in the second stage 20 households are sampled
from each selected village.

Due to constraints of time and resources, household listing is not done prior to household
selection in the village. Instead, the village is mapped including all hamlets, landmarks and
sections. Local participation in the mapping process is essential. If the village is a continuous
habitation, then the village is divided into 4 sections. If the village is comprised of hamlets, then all
hamlets are numbered and four hamlets are randomly selected using chits. Five households are
randomly selected within each hamlet or section by visiting every fifth household from the centre
of the hamlet or section. All children in the household between the ages of three and sixteen are
surveyed.

ASER also employs a rotating panel of villages so as to increase the precision of the measurement
of temporal changes. In 2005, in each district, 20 villages were randomly selected using the Census
2001 village list as the sampling frame. Standard PPS (Probability Proportional to Size)
methodology was used to randomly select villages in each district. In 2006, in each district, the 20
villages from 2005 were retained and 10 new randomly selected villages were added. In 2007, 10
randomly chosen villages were retained from 2005 and 10 from 2006, and 10 new villages were
added. The 2008 sample had 10 villages from 2006, 10 from 2007 and a new set of 10 villages. The
analogous process was carried out in 2009 and will be continued in 2010.

9
India has approximately 600 districts of which about 575 are rural districts. ASER is carried out in all rural
districts each year.
 8

B. Coverage:

Each year ASER aims to reach every rural district in the country. The ASER design keeps in mind
the need to keep costs low in terms of time and people. Given these constraints, the challenge is
to generate reliable district level estimates. In 2005, 20 villages were randomly picked from each
district. In each sampled village, 20 households were randomly selected for the survey. All children
in these sampled households were surveyed and tested. From 2006, the ASER sample size was
increased to 30 villages per district and 20 households per village. This expanded sample size has
been used from 2006 onwards.

The table below shows the coverage that has been achieved in each year.
Table 1. Coverage of ASER, 2005 – 2009
Coverage 2005 2006 2007 2008 2009
No of districts covered 485 549 567 583 583
No of villages covered 9,521 15,610 16,054 16,198 16,291
No. of villages per district 20 30 30 30 30
No. of households surveyed 191,057 318,761 319,239 335,966 338,027
No. of children surveyed (total) 332,971 758,028 720,397 704,241 691,734
Children aged 3-5 -- 146,422 124,856 118,325 117,679
Children aged 6-14 332,971 518,551 527,266 510,985 489,954
Children aged 15-16 -- 93,055 68,275 74,931 84,101
No. of schools observed 9,252 -- 13,355 -- 9,302
Note: ASER reports from 2005 to 2009 are available on www.asercentre.org.

C. Household survey questions:

In each sampled household, the child’s age and sex is noted down. Since 2006, some
information about mothers is also recorded. Except for educational information of each child, all
other data collected from households is minimal. Given the vast scale of this exercise and the
diversity in the types of volunteers who conduct the survey, the household survey format is limited
to one page.10

In 2007, we asked each household whether children attended tuition class, defined as any paid
class outside of regular school. In 2009, information about father’s education was also collected.
Both these indicators are included in ASER 2010 as well. In 2010, a simple test of mothers’
numeracy has also been included which involves dialing a telephone number on a mobile phone.

Since 2008, for each surveyed household, a basic set of household characteristics has been
collected. This includes information about the type of house (construction – permanent/”pucca” or
temporary “kachcha”), livestock, electricity in the household, phones and televisions. Similarly,
since 2008, for each sampled village, information on basic village facilities like road connectivity,
availability of schools, health care, banks, electricity etc has been collected.

10
The survey format is piloted repeatedly each year before the actual survey to ensure ease of
administration. Instructions to surveyors are also reviewed and modified each year to improve on the ease
with which they can be understood. Survey formats and instructions are available in most regional
languages. Since 2007, an animated presentation is used to enable trainers to summarize the key points of
the survey. In 2009, short quizzes were used to “test” knowledge of master trainers and trainers.
 9

D. Assessment tasks for basic learning:

The primary focus of ASER has been on enrollment and learning. Children are assessed in the
household.

Why are children surveyed at home? There are several reasons for why the assessment is done at
home. First of all, since the primary purpose of ASER is to generate a district level of estimate of
schooling and learning, it is important that sampled children should be representative of all
children in the district whether they are children who are in government schools, private schools,
or out of school. The sampling frame is generated from the village lists in the 2001 Census.

Second, if a school based estimate for children’s learning was to be created, then for sampling we
would need a complete list of schools (government, private – aided, unaided, recognized or
unrecognized). While comprehensive data on government schools are available for each district,
the listing of private schools is not as reliable or complete. In any case, with school based
assessment, district level learning estimates would ignore out of school children and dropouts.

What is each child asked? Each child/family is asked if the child is enrolled in school. The child is
also asked if the school that s/he attends is a government school or a private school, or whether it
is a madarssa or any other kind of alternative school. As the survey is done in the household, the
child’s response is corroborated by other household members or neighbours.

Each child aged 5 to 16 in the sampled household is assessed one by one. The common and
comparable tasks each year are for basic reading and arithmetic. Each year additional components
are added (see Annexure 1). There are several samples of each test so that each child in the
household can be given a different but equivalent test.11

Why is the same test given to children of all ages? For basic reading and maths, the same test is
given to children between the ages of 5 and 16. The ASER test is a “floor” level test. It assesses
whether children are able to do basic tasks in reading and arithmetic. For younger children in Std 1
and 2, it is not expected that they will be able to go beyond the first few tasks. However it is
expected that older children in Std 3 onwards will be able to comfortably and confidently go
beyond the simple tasks in the ASER assessment. Since 2005, ASER results have indicated that a
significant proportion of children in Std 3, 4 or 5 are not able to read simple text at Std 2 level or
do basic numerical subtraction problems expected of children in early grades.

E. School observations:

ASER is a household survey of children. All questions and assessment tasks are given to children at
home. However, from time to time, ASER does visit schools. It is important to understand that the
school observation data presented in the ASER reports are not based on a statistically selected

11
ASER Centre has detailed documents on details of tools and the process of their development. These are
available on request. Tools are developed after analyzing state textbooks and in consultation with expert
groups at state and national level. They are then piloted intensively before use to ensure comparability and
consistency across states and over time. Extensive studies have also been carried out to establish the
reliability and validity of the ASER tools as well as their comparability with other similar tests.
 10

school sample. This should be kept in mind when interpreting ASER school data and
when comparing ASER data to other school based data. School visits have been a part of ASER
2005, 2007 and 2009 exercises and will be part of ASER 2010 as well. ASER volunteers visit over
15,000 villages in the country and a total of 30 schools are visited in each district.12

• ASER 2005: ASER volunteers were instructed to visit any government school with primary
grades in the sampled village.
• ASER 2007, 2009 and 2010: ASER volunteers were instructed to visit a government school
with primary grades in the sampled village. However, if there was more than one
government school in the village with primary grades, then they were instructed to visit
the school which had greater enrollment.

The comparison of data for schools visited in 2005, 2007 and 2009 should be done keeping in mind
the differences described above.

Most of the school observations in ASER are for indicators where visual verification is possible. The
data collected includes head count of attendance of teachers and children, and functioning of
basic amenities like toilet, water and midday meal. For example, in the case of water, the surveyor
is asked to check if there is a tap or hand pump and also if there is water coming out of it on the
day of the visit. No observations are made on teaching learning processes.

Another important aspect which was added to ASER school observation format is the tracking of
SSA central government funds. This section is tracking the 3 main SSA central government funds
(School Maintenance Grant, School development grant and Teacher learning material grant)
coming to the school. This is to understand whether the schools have received the grants? If yes,
then what was the amount received? Whether they have spend it or not? And who is aware about
the grants coming to the schools (HM, teacher, para-teacher).

ASER is in the field each year in the October-November period. The school observations are usually
made on a Saturday or a Monday as the household survey is typically conducted on a Sunday. In
many states, this is a period of festivals and holidays. School functioning may be affected as a
result of the festival season and the data may reflect this as well.

F. Time duration:

Each year ASER is in the field in October and November. Each year the report is released
in January. From start to finish, the entire exercise takes about 100 days. The timing of ASER is
determined by a number of factors, the primary factor being that the ASER report should be
available before the plans or allocations for the following academic year are made. The budget is
presented to the parliament in end February and the annual work plans for elementary education
are usually finalized in March. In many states, enrollment numbers are finalized by end of August
or September. And, in most states, by October or November, children are in the middle of the
school year. ASER findings therefore provide a midyear benchmark for basic learning and
enrollment.

12
The NCERT study for the mid term assessment is also based on 30 sampled schools selected in proportion
to Std V enrolment. The NCERT selects from four strata – rural & primary, rural and other than primary,
urban & primary and urban and other than primary. 15 children are selected from each sampled school.
 11

G. Partnerships:

One of the most unique features about ASER is decentralized and localized implementation and
dissemination. For each year’s ASER, in each district of India, there is a local organization which
actually carries out ASER. To ensure comparability and consistency, the sample design and the
assessment tools are centrally designed. But the actual work is carried out in each rural district by
the ASER partner for that year. The local partner is involved in data collection and also
in dissemination of the results.

Sarva Shiksha Abhiyan requires that each year detailed work plans for elementary education be
made in each district. These plan documents include reviews of progress made in the previous
year, plan of activities for the current year and budgets. SSA guidelines state that the district’s
annual work plans should incorporate habitation level planning, participatory processes,
community mobilization and participation, collaboration of government and local non-government
groups in the process of developing plans.13

Local ownership is a key element of the architecture of ASER. Local ownership is essential along
with local level efforts for awareness building and action for the improvement of elementary
education. In the ASER context, the district is the “local” unit. The ASER approach is intended to be
aligned with the planning and implementation envisaged in Sarva Shiksha Abhiyan. ASER provides
a representative sample of children at the district level. It is also the only estimate currently
available for basic learning outcomes. The information generated from ASER can be a valuable
input into the annual work plan process. The local group that collects the information for ASER can
potentially be a member of the core group that facilitates the planning process in the district.

Each year since its inception, about 600 local groups have participated in this effort. These include
self help groups, women’s organizations, youth groups, nationally well known NGOs, local
community based organizations, college students and faculty from district colleges, and
universities.

The resources for this large-scale citizens’ participation come from individuals as well as
institutional donors.14

CONCLUSION

With increased public spending in the social sector, it is essential that mechanisms for increased
accountability be introduced. As citizens of India, and as funders of increased social sector
allocations, we need to know how far we have come as a country. Government’s mandate is to
provide elementary education. As citizens, we have the right and the responsibility to participate
in the process that translates outlays into outcomes. ASER is a nation-wide example of how to
make this happen.

13
Sarva Shiksha Abhiyan: A Programme for Universal Elementary Education. Framework for implementation
st
(Amended till 31 March 2008). Ministry of Human Resource Development. Department of School Education
& Literacy. Government of India.
14
Each year’s ASER report lists names of partners and also those of donors.
 Annexure 1. LEARNING ASSESSMENT TASKS : ASER 2005 TO 2010
Used in Year
Target
No Domain Description Details
population
2005 2006 2007 2008 2009 2010

Basic reading tasks (floor level text) : Read letters, common

All children : age
1 Reading words, paragraph with 4 simple sentences (Std 1 level text) X X X X X X
5 to 16
& short story of 10 to 12 sentences (Std 2 level text)
Child asked to read Std 1 level text, then based on the Std 1
Reading & level text ("para"), 2 fact retreival questions were read out to All children : age
2 In all Indian X
Comprehension the child and the child had to answer orally. Same was 5 to 16
languages &
done with the Std 2 level text.
English
Children who
Reading & Child asked to read Std 3 level text. Based on the text, the
3 could read Std 2 X
Comprehension child asked to read questions and then answer orally.
level text fluently
Adult women in
4 Reading Paragraph with 4 simple sentences (Std 1 level text) X X
the household

All children : age

5 Writing One short dictated sentence In own language X
5 to 16

Basic arithmetic tasks (floor level text) : Number recognition

1-9, 11-99. Subtraction : Numerical 2 digit subtraction All children : age
6 Arithmetic X X X X X X
problem with borrowing. Division: Numerical problem : 3 5 to 16
digit number divided by 1 digit number
Child asked to read the word problem and solve it. Child Only to those
could answer orally or in writing. One word problem on children who
7 Arithmetic X
subtraction (2 digits) and another word problem was a could read Std 2
division problem (3 digits divided by 1 digit) In all Indian level text fluently
languages &
Word problems with currency operations with Rs 50: Child All children : age
8 Arithmetic English X
was asked orally. Child could answer orally or in writing. 5 to 16
All children : age
Telling time. Using currency notes X
5 to 16
Applied
Children in Std 5
9 arithmetic & Simple calendar, menu, area and estimation tasks X
and above
everyday tasks
Mothers of
Dialing a mobile number X
children tested

Basic English reading and comprehension tasks: Letter

English : English as a
recognition, reading simple words, saying meaning of simple All children : age
10 Reading & second X X
words, reading basic sentences, saying meaning of basic 5 to 16
comprehension language
sentences.
School observations were done in 2005, 2007 and 2009. They will be done in 2010 also.