“Without data, you’re just another person with an opinion.“

W. Edwards Deming

This quote emphasizes the importance of data and monitoring in decision-making processes across various fields and education is no exception.  In the context of school education, monitoring plays a critical role in ensuring delivery of quality education and identifying areas for improvement. In the Indian context, while monitoring systems do exist in all states, involving government officials in conducting regular visits and inspections to schools, these visits are often not structured and fail to identify and resolve challenges specific to academic performance.

To address these challenges, the Department of School Education, Telangana in collaboration with the Central Square Foundation developed an Integrated Monitoring System to track and review data across inputs, outputs, and outcomes. 

As part of Telangana’s FLN Mission Tholimettu (First Step), the monitoring system is designed to provide supportive supervision and mentorship to teachers, empowering them with the right tools and inputs to achieve desired learning outcomes. In this article, we present the design framework and outcomes of this monitoring system, which includes the use of easy and effective mobile-based monitoring platforms, data analytics for informed decision-making, and a nuanced understanding of the structural and classroom factors in the education system of Telangana.

Genesis of FLN Monitoring and Mentoring in Telangana- The Journey

In an effort to promote structured pedagogy, Telangana developed teacher learning material (TLM) that includes lesson plans and teacher guides, and conducted training for all teachers throughout the state in a cascade mode at various levels. However, the department recognized the need to ensure that teachers are using the TLM effectively and receiving sufficient support and feedback to improve the teaching-learning process in their classrooms. To address this, the concept of “Monitoring and Mentoring” was seeded and further translated into action, with a focus on identifying capable individuals within the system, establishing strong processes, and leveraging technology to support the initiative. 

The monitoring system’s design framework is based on a nuanced understanding of the structural and classroom factors in the education system of Telangana. The following components were considered to improve governance and state capacity:

• Process: designing Key Performance Indicators (KPIs) based on the program’s Theory of change and further developing tools, protocols, and reliability mechanisms to measure KPIs.
• People: low state capacity in terms of inadequate resources and systemic infirmities contribute significantly to ineffective monitoring.This is something the state has addressed by creating a dedicated team (cadre in government) of monitoring officials from the Headmasters (HMs).
• Technology: leveraging technology to facilitate monitoring and assessment processes, including the use of the mobile application/platform and dashboards.
• Capacity Building: strengthening officials’ monitoring and mentoring abilities through training on the monitoring model adopted and data collection mechanisms. Further capacity building was done on data-driven decision making at cluster, mandal (block) and district level to make appropriate course corrections in the approach.

Leveraging Technology

The monitoring system’s data architecture includes modules created by State Council for Educational Research and Training (SCERT), Telangana, with CSF’s support, which form the core of the monitoring framework. These modules include:

• A student learning tracker for teachers to track learning levels of individual students against goals set on a monthly basis.
• A classroom observation tool for mentors/monitoring officials to conduct in-class evaluation of Teaching Learning Process (TLP) and further provide appropriate constructive feedback.
• A spot assessment module for mentors/monitoring officials to conduct assessments on a sample of students to continuously track learning levels against the goals set, thus using this learning data to support the teachers and facilitating course corrections in the program.
• A review tracker to document proceedings of the reviews conducted making them more effective.

Digitalization of the above processes through a mobile application can enable real-time data visualization in an online dashboard across the state at all levels, which further enables comprehensive, data-driven decision-making. The use of a standardized visit proforma in the mobile application also provides focus and allows visiting officers to identify state’s priorities. 

Therefore, a mobile app was designed and developed with all the above modules incorporated. A think deep, start quick and scale fast strategy was adopted while developing the tech systems, wherein a Minimum Viable Product (MVP) was launched first and then scaled up later. Tangerine, a mobile assessment platform developed by RTI International was used by further contextualizing it to meet the state’s requirements. Google Looker Studio, a free dashboard tool, was used to create dashboards for data. Over a period, this was further refined based on the feedback and a Master App was developed and launched by the Department of School Education, Telangana.

Of the above, the Master App currently covers the following critical components:

• Student Learning: tracks performance of students  with respect to FLN outcomes, with focus on triangulation of learning data from National Achievement Survey (NAS), State Level Achievement Survey (SLAS), monthly student learning tracker and spot assessments.
• Teaching Learning Process: tracks performance of teachers in FLN classrooms, with special focus on lesson plans, using TLM effectively and assessing FLN accurately, among others.
• Governance: tracks effectiveness of mentoring/monitoring visits and review meetings. 
• Critical Enablers: this looks at ensuring all basic FLN inputs are in place in  school such as functional libraries, textbooks, etc.

Impact

During a 4-month period from September 2022 to January 2023, approximately 41,000 classroom observations were conducted in 15,633 schools, and 28,625 teachers had at least one visit. Learning levels of all the 11.3 Lakh primary students are tracked on a monthly basis and  a total of 1.5 Lakh students were tested through spot Assessments. 1500 review meetings’ data was also captured across the state. 

The introduction of the monitoring system with real-time dashboards has been a game-changing innovation and has raised the bar for the department in terms of using technology for monitoring. This has led to an increased support for the program.

Way Forward

The most important thing we can do to ensure high data quality and to improve the system over time is to ensure everyone uses it.  The same applies to the Integrated Monitoring System in Telangana as well. While the idea is definitely to improve data  reliability and to further integrate with other data systems, the state is moving towards institutionalization of the FLN monitoring system and thus enabling data driven governance. 

While we acknowledge that the Monitoring system is not a silver bullet for all the problems ailing school education in India, however, what can be said confidently is that a well-designed integrated system can aid the state in streamlining implementation of programs and improving learning outcomes. Experiences of Telangana have been positive so far and with high acceptance and ownership of the state apparatus, it is evolving and moving in the right direction towards a visible change. 

We are positive and confident that the initiative will continue to thrive and bring about positive change in the way monitoring is envisaged as an enabler for improving learning outcomes.