The Covid-19 pandemic has highlighted the importance of integrating technology in learning and teaching. EdTech solutions are enabling children to learn at home in sync with what is being taught in schools. They are also helping children learn at their own pace and levels with the help of games and activities that are fun and engaging. Gamification in learning goes a long way in harnessing children’s natural curiosity and their proclivities to experiment, explore, problem-solve, and stay engaged in meaningful activities^[1]. The National Education Policy (NEP) 2020 too stresses that learning should be holistic, integrated, enjoyable and engaging. Technology can play a catalytic role in delivering these experiences to children at their homes.

Chimple, a non-profit EdTech organization based in Bangalore, has been an early-believer in technology-led playful learning. Founded in 2015, Chimple is a game-based app focused on improving core foundational literacy, numeracy, and digital skills for students in grades 1-3. The team was a finalist for the Global Learning XPRIZE (a global innovation challenge for education with a $15mn grand prize), won a $1M grant to further develop their solution, and test it via an evaluation in Tanzania, where they were able to demonstrate impressive learning gains in both math and language.^[2]

Chimple is now embarking on a journey of rigorous evaluation via a randomized controlled trial (RCT) of its teacher-led, at-home learning model in India, so that it is better prepared to be scaled up. There is an urgent need to build evidence on technology-based home-learning solutions, particularly in low-income contexts, so that we can understand what works, how it works, and whether it works at scale. Chimple’s evaluation will help inform the interventions needed for such at-home learning models contextualized for India 2 and 3, especially for children in foundational grades.

There is an urgent need to build evidence on technology-based home-learning solutions, particularly in low-income contexts, so that we can understand what works, how it works, and whether it works at scale.

Before the evaluation takes place, a coalition of researchers conducted a pilot study to understand how children and teachers can meaningfully engage with the app. CSF was part of this coalition. This article presents key findings from the study on what worked to boost engagement with Chimple.

Chimple’s approach

Chimple’s theory of change is organized along two pillars: engage the child and involve the learning agents (teachers and parents).

This two-pronged approach serves to improve uptake, adoption, engagement, and retention in two ways – one, by creating in-app engagement hooks geared towards young children, and two, by creating an interface through which the teacher or parent can meaningfully connect the game to classroom instruction, and truly blend learning.

Since 2019, Chimple has been working to contextualize its offering to serve the Indian landscape via translations in language and content. It has pivoted its model from school-based to home-based during Covid-19 to reach as many students as it can via the Android PlayStore, and is now piloting innovative WhatsApp-based learning in partnership with the Nashik Municipal Corporation and Rocket Learning.

Preparing for the Chimple evaluation

To prepare the model for evaluation, Chimple worked with a research coalition – consisting of Central Square Foundation, Chimple, Bharti Foundation’s Satya Bharti schools, and researchers from the University of Virginia and Wellesley College – to pilot the solution in a handful of schools (as part of the Bharti Foundation’s Satya Bharti Schools program) in academic year 2021-22, create feedback loops on the ground to learn from and refine the implementation, and set up the right processes for scale-up and research in the next academic year. By taking a phased approach to evaluation and by piloting the solution in a few schools first, the coalition was able to develop a strong product and program to achieve the desired learning time on the solution to best set it up for impact evaluation. The research coalition reviewed metrics from the product data weekly around these three key areas:

• Engagement: the amount of time the child spent on the solution, how many games they played, etc
• Retention: : how many users were active, and whether they were coming back to the solution on a weekly basis
• Learning progression: how much of the curriculum was being covered, how many activities/games were played across subjects

Chimple also conducted interviews with stakeholders on the field, including teachers and parents, to get qualitative insights into the health of the program. Despite strong initial engagement due to the novelty and excitement around the technology, they saw retention and engagement begin to dip over time. The iterative process allowed them to learn-as-they-went , and Chimple uncovered many interventions that worked to increase engagement with technology solutions, and many that did not. Many lessons were learned along the way, both on the product and program side, with implications for at-home and blended EdTech implementations in India and across the world.

Key findings from the pre-evaluation study

1. Strong curriculum integration is instrumental in supporting teachers:
   When the content and language in EdTech solutions is familiar to teachers, their uptake and engagement with it increases.One of the early pieces of feedback from teachers was that the Chimple curriculum does not speak to their prescribed curriculum as the games and activities in the app were termed differently than what they see in their textbooks. To solve this, Chimple revamped its UI/UX to display the names of the games/activities in the way that they are termed in NCERT textbooks. This increased familiarity and led to a significant increase in teacher engagement.Chimple also worked with the Bharti Foundation team to find ways in which the app’s activities could be integrated into regular teaching and learning activities that teachers were familiar with, such as lesson plans, to build their capacity to inform instruction using data from Chimple. By seeing Chimple as homework, and by including it within the lesson planning framework, teachers were better able to map student learning and accordingly augment instruction.
2. Parental buy-in is key to encouraging learning at home via technology:
   When talking to parents, Chimple found that the following two things were key to getting their buy-in and encouraging them to share their smartphones with their children for the learning activities:a) information around what EdTech is and how it can help their child in learning, andb) easy-to-follow and simplified directives on how they can engage with their child over technology.For example, to improve engagement levels, Chimple piloted a series of parent onboarding sessions via ‘Chimple PTMs (parent-teacher meetings)’ to help parents understand the benefits of Chimple and inform them of their role in driving usage at home. They used the catchphrases such as “Khel-khel mein seekhna” (learn through play) and “Har din Chimple se padhai, sirf 10 minute ke liye” (practice via Chimple, for only 10 minutes everyday) to help teachers and parents build familiarity with the program and its objectives. In schools where these PTMs were implemented, a higher number of students started becoming active on the app, and engaging on the solution for an increased amount of time. In follow-up surveys, Chimple saw parents and teachers use the same campaign catchphrases from the program to describe how Chimple can benefit them and their students.
3. Students love to play – so let them!
   Campaigns and contests were also found to playan important role in generating excitement about the program.For example, to leverage the additional time children were spending at home during the winter vacation, Chimple and Bharti Foundation launched a “Chimple ki Nayi Saal ki Pratiyogita” (Chimple New Year contest), which incentivized students to engage on the solution via gamification. Every week, top students and parents would get highlighted via a leaderboard, and the top five students from each school were given a Chimple certificate and a small gift to commemorate their achievement. Gamification and incentives such as these were surprisingly successful in ways – one child even spent over 10 hours learning on Chimple during the four-week competition period! Campaigns like these were found to be extremely effective in improving awareness and excitement about Chimple. During school visits, students were able to immediately recognize Chimple’s logo and its mascot Chimpanzee on one of the Chimple team members’ shirts, calling it their “dost” (friend).

The above learnings were incorporated into the final program design for the impact evaluation, which was launched in April 2022. Chimple will be evaluated by the same research coalition via a clustered RCT in 34 schools across five districts in the state of Haryana, covering close to ~1,400 children in treatment and control groups. This is going to be one of the first rigorous evaluations of an EdTech solution for foundational literacy and numeracy in India. The results from this study are intended to inform policy and intervention design for other blended and at-home learning solutions in India going forward.