With the adoption of the Millennium Development Goals (MDGs) in 2000, especially MDGS 4 and 5,
the world has witnessed an accelerated reduction in both under-five and maternal mortality. In
2014, Rwanda was among 10 low and middle income (one of only two countries in the sub-Saharan
region) with a high maternal and child mortality burden that in 2012 were on track to achieve MDGs
4 and 5a. This success has resulted from the use of several programs to improve maternal and child
health in the country.
One of the strategies that Rwanda employed over this time period was the use of an mHealth
system called RapidSMS. The platform was implemented to facilitate communication between
Community Health Workers (CHWs) and the broader health system, including the ambulance
system, health facilities, and the central government. As part of this program, CHWs were equipped
with mobile phones that enabled them to collect and use real-time data on key maternal, neonatal,
and child health indicators. This was collected during the first 1000 days of life: from the start of
pregnancy until 2 years of age.
The RapidSMS system was first piloted in Musanze District starting in 2009, then scaled-up
nationwide starting in 2013. A pilot study found an increase in the proportion of births taking place
in a health facility in the district.1 After completing this pilot, in addition to scaling up the RapidSMS
program nationwide, UNICEF provided additional support in 10 Districts, including CHW training and
quarterly supervision meetings, the provision of essential equipment to health facilities, and
community nutrition initiatives.
Through a stakeholder engagement session and qualitative interviews, we considered three
mechanisms through which RapidSMS could have influenced maternal and child health. The first was
the RED Alert notification system that could have led to a decreased response time and earlier
intervention in emergency situations. Second, through improved tracking of pregnancies, newborns,
and children, we hypothesized that RapidSMS could have impacted processes of care including the
number of ANC visits, and the proportion of deliveries that took place in health care facilities.
Similarly, better tracking of newborns through the first 1,000 days of life could have resulted in
increased PNC visit rates and follow-up visit rates, allowing the earlier identification of issues that
might have led to infant mortality and child mortality. Finally, the improved data on maternal and
child health services could have led to more informed decision-making by policymakers.
This study used longitudinal quantitative data and comprehensive qualitative interviews to evaluate
the impact of the RapidSMS program on a number of key indicators and outcomes. Our mixed
methods approach assessed the relevance, effectiveness, impact, efficiency, equity, and
sustainability of the RapidSMS program. Our quantitative analysis used program data from
RapidSMS, along with administrative health data from Rwanda’s Health Management Information
System (HMIS) between January 2012 and June 2016. We used Interrupted Time Series Analysis
(ITS), one of the strongest quasi-experimental research designs, to study longitudinal changes in
several outcomes of interest. Our qualitative study leveraged key informant interviews from
stakeholders and program recipients along with focus groups of CHWs, mothers, and fathers, in 4
Districts (Ngororero, Nyanza, Gasabo, and Ngoma). All interviews and focus groups were conducted
using a semi-structured interview guide, were transcribed, and were coded using ATLAS.TI software.
Finally, we also reviewed all available program manuals and documents made available to us.
In terms of program use, we found that over the course of the study period, CHWs sent more than
9.3 million text messages, most commonly for community-based nutrition and child health visits. In
terms of completeness, it appears that RapidSMS is only capturing a portion of births: compared to
8
HMIS and Census estimates, only about 56% of births were recorded using RapidSMS. This varied
substantially by district, with UNICEF supported districts showing much higher rates. The overall rate
of message use was similar in UNICEF supported Districts and other districts. The use of RED Alert
emergency messages was low, which may have been driven in part by the fact that CHWs reported
ambulances not responding more often than they came.
Our quantitative analysis found little impact of the start of RapidSMS use in health centers on most
antenatal care outcomes, including registrations, 1st trimester visits, and the number of mothers
completing 4 standard visits. In contrast, we did find an impact on facility delivery rates, with UNICEF
supported districts showing an 18% increase at one year over what would have been expected
based on existing trends. However, we found no evidence of such a change in non-supported
districts. Similarly, we found increases in postnatal care registration, visit numbers, malnutrition
screening, and immunizations delivered in UNICEF supported districts; for example, a 100% increase
at 1 year in total postnatal care visits. However, we found either smaller on non-existent changes in
these same indicators in non-supported districts. Unfortunately, unanticipated data quality issues
precluded our ability to rigorously assess the impact of the program on maternal or neonatal
mortality.
Our interviews showed broad support for the RapidSMS program, and a widespread belief that the
program is improving maternal and child health care and outcomes. Participants shared the program
objectives are well-aligned with District, national, and international targets, and that it has reached
most of the intended beneficiaries. Most respondents felt the financial and human resources
dedicated to the program were sufficient to ensure its effective operation. However, program
sustainability was a concern given the ongoing dependence on funding from developing partners.