Objective To evaluate the impact of the community-based newborn care package (CBNCP) on six essential practices to improve neonatal health.

Methods CBNCP pilot districts were matched to comparison districts using propensity scores. Impact on birth preparedness, antenatal care seeking, antenatal care quality, delivery by skilled birth attendant, immediate newborn care and postnatal care within 48 hours were assessed using Demographic and Health Survey (DHS) and Health Management Information System (HMIS) data through difference-in-differences and multivariate logistic regression analyses.

Findings Changes over time in intervention and comparison areas were similar in difference-in-differences analysis of DHS and HMIS data. Logistic regression of DHS data also did not reveal any significant improvement in combined outcomes: birth preparedness, adjusted OR (aOR)=0.8 (95% CI 0.4 to 1.7); antenatal care seeking, aOR=1.0 (0.6 to 1.5); antenatal care quality, aOR=1.4 (0.9 to 2.1); delivery by skilled birth attendant, aOR=1.5 (1.0 to 2.3); immediate newborn care, aOR=1.1 (0.7 to 1.9); postnatal care, aOR=1.3 (0.9 to 1.9). Health providers’ knowledge and skills in intervention districts were fair but showed much variation between different providers and districts.

Conclusions This study, while representing an early assessment of impact, did not identify significant improvements in newborn care practices and raises concerns regarding CBNCP implementation. It has contributed to revisions of the package and it being merged with the Integrated Management of Neonatal and Childhood Illness programme. This is now being implemented in 35 districts and carefully monitored for quality and impact. The study also highlights general challenges in evaluating the impacts of a complex health intervention under ‘real life’ conditions.