Foundational literacy and numeracy (FLN) skills are the most important building blocks for a child to thrive in and navigate today’s world. The existence of alarming learning gaps, especially post-pandemic has led to increased focus on targeted instruction to ensure that a child’s learning level meets the level they are at. This has also thrown light on the need for consistent generation of evidence of children’s learning levels, and the regular need to conduct assessments. The National Education Policy (NEP) 2020 through NIPUN Bharat mission emphasizes on the importance of assessments not just at the unit of a classroom or school, but also at the systemic level through regular large-scale assessments.

To institutionalize the process of a large-scale assessment and to employ data in setting realistic targets, the Government of Telangana administered a sample baseline assessment in the state to:

1. Understand the level of FLN skills among students enrolled in Grade 3
2. Understand how baseline assessment data can aid in setting learning targets at the state and district level to plan for future interventions, as and when needed.

The baseline assessment in Telangana assessed a sample of nearly 6300 Grade 3 children in each of the subjects of Telugu/Urdu, English and Maths, over three days, in an orally administered one-to-one format, where 54 master trainers (MTs) and 1400 field investigators (FIs) were trained. It saw the introduction of technology for data collection, simplifying an otherwise tremendous task by many hours and financial resources. The following are some key learnings that stood out from this process:

(A) A contextualized and piloted assessment tool is key

An assessment tool with well-defined items, aligned with the curricular standards is essential. A three-day in person workshop was organized to ensure a free-flowing and detailed discussion between different subject resource groups and assessment experts. The merits and administration methodology for each item was discussed to contextualize the tool without having to omit any key features. It was ensured that the tool stayed comparable across Telugu and Urdu in both scope and rigor.

A field pilot of all the items was conducted to identify items that provided the most accurate, valid and reliable results. The sample schools chosen for the field pilot resembled the sample for the actual baseline. This field trial helped to:

(i) Conduct a psychometric analysis to evaluate each item on difficulty, reliability, fit- all of which are deciding factors in choosing whether an item should be retained in the final administration or not
(ii) Understand logistics of a large-scale assessment and to be fully equipped to run a hassle-free exercise
(iii) Build a more robust training design based on pilot experiences

(B) Well-designed and implemented training model means the job is already half done

The key to standardization for any large-scale assessment, especially for the ones that involve one-on-one oral testing, is an effective and structured training model that can be successfully implemented without leading to transmission loss. The baseline training implemented the following cascade model that comprised of: 

(i) State-level training of master trainers by the state team
(ii) District-level training of field investigators by master trainers 

The key objective of the state-level training was to prepare master trainers as effective trainers to ably lead training in their respective districts, whereas the objective of the district level training was to prepare field investigators to become thorough surveyors. A group of high school teachers, nominated by State Council of Education Research and Training (SCERT), based on their capability and experience in leading training programs, were identified to be master trainers. They were consciously selected from across all districts of the state to ensure smooth implementation of the next level of training in districts in a fool-proof and cost-effective manner. 

A three-day residential training was conducted for all master trainers, where the design, administration process, prompt and stop rules, as well as the rationale for each of these was distinctly explained. Time was allotted within the training itself for the master trainers to plan for district level training. Similarly, a carefully planned two-day in-person training was conducted for all field investigators. 

(C) Ensuring proper selection and engagement of field investigators (FIs) is key

Field investigators make up most of the force as they are pivotal in the actual roll-out of the assessment, and hence identifying and training them early on helps. After considerable deliberation, students of District Institute of Education and Training (DIET) colleges, College of Teacher Education (CTE), and govt. B.Ed colleges were engaged as field investigators for the following reasons:

(i) An external person from outside the system is ideal; DIET students can act as a neutral force and hence report reliable data from the field
(ii) DIET students are technologically-savvy to administer the assessment
(iii) For teacher trainees, this exercise plays a paramount role in familiarizing them with the current reality of learning levels of children, especially in their respective districts, turning this into a unique field-work opportunity.

(D) Usage of technology is the need of the hour

Technology paves way for more cost-efficient methods of data collection and analysis, increasing efficiency. The baseline assessment in Telangana stood as a pioneer for the state by introducing and leveraging technology as a medium to administer and collect data from the field through a large-scale assessment, using an application called Tangerine. The activity of rendering all the assessment tools in Tangerine proved significantly fruitful, as it streamlined usage and implementation, reducing the printing cost to almost zero. 

Using technology opened doors for real-time capacity building of the system on the usage of technology in assessments, creating comfort around the usage and in accentuating the value in its adoption.

(E) A robust monitoring system leaves no room for error

33 sectoral officers (District Samagra Shiksha Officials; 1 from each district) and ~200 cluster resource persons (6 from each district) were identified and trained extensively on the modalities of the assessment and the aspects to be monitored.

To carry out the field observations, an exclusive monitoring tool was developed through which data was collected and analyzed in real-time, through the three days of the assessment. This enabled us to track implementation and course correct in real-time based on the gaps identified. Each field investigator was also given the details of the CRPs responsible for the schools that they were going to visit, resulting in better coordination and smoother execution.

(F) Usage and dissemination of data is key to closing the loop

Long-term adoption of any process by a state government is dependent heavily on the meaningful applicability of the results. Therefore, it is essential to show how the generated evidence can be used to self-check and improve. To ensure this, Telangana adopted the following:

(i) Alignment of results with the FLN mission monitoring metrics; this included setting goals and ensuring tracking progress on the learning outcomes over time against the baseline
(ii) Incorporating the results in the FLN mission dashboard, which is used by all stakeholders in the system from state to school level

Dissemination activities tailored to meet expectations and needs of different stakeholders were planned at the state, district, mandal, cluster and school level. 

Way Forward

Going forward, the state of Telangana is keen on conducting an endline assessment to track progress achieved by the state in foundational literacy and numeracy through the year, and in identifying further room for improvement. The assessment will be conducted at a similar scale, but with an even finer design and implementation plan, incorporating all learnings from the baseline assessment.