Q1. How can policymakers, stakeholders and academicians at the state and district levels effectively interpret monitoring and evaluation data for education, particularly FLN, and align their priorities to maximise impact in the final two years of the NIPUN Bharat Mission?

It is not the responsibility of policymakers alone to interpret data, but it is incumbent upon those collecting the data to present it in an understandable manner. I think we are trying to address the problem in a reverse and counterintuitive manner when we ask ‘how can data be interpreted to enable decision making for impact.’ The question that should be addressed is – are data systems designed to enable decision making for impact? Data for data’s sake is ineffectual and will always leave decision makers thinking ‘so what?’ Data systems – results frameworks and their indicators – need to account for utility from the point of design – they should and have to be in service of the theory of change. Parallelly, data workers need to present learnings in a way that is understood by those in decision making roles. This means moving beyond complicated statistical terms to breaking that down to implications. For example, if the data shows children struggling with reading comprehension, those responsible for the data need to show what are the competencies before reading comprehension where there might be breakdown and where decision makers need to focus. 

Q2. What are some of the key factors/considerations to be kept in mind while measuring learning outcomes in FLN grades? 

The most important thing to keep in mind while designing large scale assessments is how to generate the most robust data within the resources available. This could mean financial and human resources. Creating a best fit and yet robust assessment framework and instrument is one step, the second is ensuring a robust sampling technique and finally there is training of field investigators for effective deployment of the instrument. Other factors like routine data monitoring and ensuring appropriate technology for the assessment are also important to keep in mind. 

Q3. How well do existing assessment tools like ASER, NAS and state-level assessments capture FLN learning levels?

Each of these assessments have a different purpose and they meet those purposes quite well. 

ASER is not intended to be a holistic or formative assessment but rather a big picture dipstick that tells us where the (rural) school going population is positioned in terms of basic reading and numeracy skills. 

NAS is a grade-appropriate assessment of the learning levels of a state – across various school types, giving us an understanding of some more granular and grade-level concepts around learning. 

State FLN assessments are specifically designed to give us an idea of where the breakdown is in the building of FLN skills – these can be formative and summative – used for both precise planning around curriculum and pedagogy and also as a report card for the efforts made thus far. 

Name	Grades	Frequency	Modality
NAS	3,6 and 9	Once in three years	Paper pencil – OMR
FLS	3	Only conducted once (in 2022)	Paper pencil – OMR
ASER	3,5 and 8	Once a year till 2014; once in two years post 2016	Paper pencil – Survey Booklet
State Assessment: Madhya Pradesh	2 and 3	Every year (2022-23, 2023-24 and 2024-25)	State app-based
State Assessment: Haryana	1, 2 and 3	Two rounds (2023-24, 2024-25)	Tangerine-based
State Assessment: Assam	2	First round (2024-25)	Tangerine-based
State Assessment: Bihar	2	State rounds (2022-23, 2024-25), district (2023-24, 2024-25)	Tangerine-based
State Assessment: Punjab	1 and 2	Second round (2022-23, 2024-25)	Tangerine-based
State Assessment: Odisha	2	Second round (2022-23, 2024-25)	Tangerine-based
State Assessment: Telangana	2	State rounds (2022-23, 2023-24, 2024-25), district (2024-25)	Tangerine-based

Q4. Can you share examples of how M&E-driven insights have influenced state-level FLN policy or interventions?

An important example of assessment-driven insights influencing state-level FLN work is state FLN assessments in Madhya Pradesh (MP). The Government of Madhya Pradesh not only owned the results of the FLN assessment but also used them to plan FLN activities and set targets at a district level. This is an important example of proactive data utilisation. For M&E insights to be used in FLN policy work, governments need to own the entire M&E design and process. For example, for data from classroom observations or teacher interviews to make it into the system, there needs to be system buy-in of those data points, or else any insights from this data will be limited to smaller programmes rather than state level adoption. CSF, for example, is committed to incorporating insights provided by its M&E system into its work at the state and district levels – a lot of which is focused around strengthening state appetite and capacity for data like this. 

Q5. How does CSF support states in setting up robust monitoring and evaluation frameworks for FLN?

As mentioned above, CSF is committed to strengthening state appetite and capacity for robust data systems and we do this in multiple ways. One of these ways is providing technical support for all FLN data-related activities, another is by demonstrating utility of the data and a third is by facilitating capacity building in the data domain. 

About Dr. Akashi Kaul

Akashi leads the Research, Monitoring, Evaluation, Assessment and Learning (RMEAL) team at CSF. She specialises in evaluation methods and designs, with close to 15 years of experience. Her previous roles include leading the education, child protection and equity vertical at a leading Indian RMEL firm, contributing to external evaluations for NIH, NSF, NASA and US Dept. of Labour and State as an evaluation expert and conducting independent research on advancing evaluation methodologies in the global south. She enjoys designing research and evaluation for complex interventions in complex contexts, particularly with an eye towards using innovative methodologies and learning mechanisms. Akashi holds a Bachelor’s Degree in Literature from Punjab University, a Master’s Degree in Communication from SIMC, a Master’s in Education Degree from University of Pennsylvania, and a PhD in Education Policy from George Mason University.