Designing a low-tech digital education RCT during a pandemic: tools, tips and tricks

2021-09-12 africa evidence week 2021 blog highlights informs learns news
Designing a low-tech digital education RCT during a pandemic: tools, tips and tricks

Blog originally published on the Young 1ove website

In March 2020, the onset of COVID-19 and school closures in Botswana presented an opportunity to pivot our education programs to remote instruction and venture into the unknown: digital education programming. In response, we designed and implemented digital adaptations of our education and health programs. Six months later, we’ve successfully completed one of the world’s first randomized trials on remote education during the crisis: a low-tech RCT that used phone calls and SMSes to deliver learning content to approximately 10,000 students across Botswana.   

Our midline results, published in collaboration with Oxford University’s Center for the Study of African Economies (CSAE) are striking: at midline, we observed a 52% drop in the percentage of students who were unable to perform any mathematical operation at the start of the intervention, among students who received a weekly phone call and SMS. These findings illuminate the high potential of low technology (low-tech) as a safe, effective and widely accessible means to remotely deliver education content and improve learning outcomes during crises and periods of school closure. Below, our team shares the tools, tricks and tips we’ve utilized over the past several months, to successfully implement low-tech digital education programs during the COVID-19 pandemic. Tools: Phone Call and SMSes, SurveyCTO and WhatsApp Employing effective digital tools to administer educational content was key to our RCT’s success. Phone calls and SMSes allowed us to reach students across the urban/rural and digital divide in 9 regions across Botswana. SurveyCTO served as the right tool to collect robust data through facilitator administered student assessments and surveys via phone call. Given that our entire field team was working remotely, WhatsApp was used to facilitate rapid communication among field staff countrywide and to train facilitators. 

Tricks: Leveraging existing curricula and cost-effectively training facilitators remotely through WhatsApp Given the need to act swiftly and decisively due to rapid school closures, we leveraged our existing curriculum to build a low-tech digital curriculum: simplifying our original content to be more accessible for students and parents to engage with. For the Low-Tech Education program, Young 1ove’s Associate Curriculum Manager, Colin Crossley notes that “we had a strong maths curriculum ready to go - all that was left to do was simplify and condense instruction to fit into a 320 character SMS and a 12-15 minute phone call’. We used an accessible and cost effective platform to train our team of 70+ facilitators remotely: WhatsApp. ‘Hand emojis’ were used to carry out roll calls, and trainees were sent voice notes accompanied by short pdf documents to deliver important training concepts. The overall result was a fun, yet effective, fully remote training experience for our facilitators.  

Tips: Advice for organizations seeking to develop and deliver low tech learning interventions 1. Make virtual facilitator trainings feedback and rapport driven to keep trainees engaged Navigating training facilitators virtually may seem like a daunting task for organizations.  However, building trust and engagement between trainers and trainees will ensure that the process is a smooth one. As noted by our curriculum and Training Manager, Ms. Thato Letsomo “Both the trainers and the participants should be very committed. Paying attention to feedback, questions and comments from trainees is key.” 2. Pilot, pilot, pilot. From a curriculum development perspective, teams considering implementing a digital education curriculum should undergo a process of testing and iteration that will ensure that their curriculum is right-fit for beneficiaries. Concrete examples include conducting pilot calls with test candidates to observe whether the learning content translates well with beneficiaries via SMS and phone call instruction. In our case, we conducted and piloted test questions and calls with a small subset of test beneficiaries, prior to the official launch of our RCT. 3. Keep it simple Instruction via phone and SMS will likely be something new to beneficiaries. Our pilot process revealed it is best to keep calls and SMSes simple, short and digestible to make learning content easier to engage with, and help students and parents adapt to low-tech instruction. Pilot calls helped us gauge the receptiveness of parents to the intervention, evaluate the optimum length of instruction calls and SMSes, and tailor learning content to suit the learning needs of our students, all of which helped us relay learning content to beneficiaries in a simple yet effective manner. 

For more insight into our digital education RCT and conducting phone-based learning assessments, read our recently published article in BMJ Global Health, and watch our July Webinar held in conjunction with SurveyCTO. If you are interested in implementing a similar low tech intervention, get in touch with us at info@young1ove.org

Follow us on LinkedIn, Facebook, Twitter and Instagram and visit our website for first hand access to the results of our digital education RCT, and to learn more about our programs.

Disclaimer

The views expressed in published blog posts, as well as any errors or omissions, are the sole responsibility of the author/s and do not represent the views of the Africa Evidence Network, its secretariat, advisory or reference groups, or its funders; nor does it imply endorsement by the afore-mentioned parties.