DDI_GHA_2010_MCC-WS_v01_M Millennium Challenge Corporation 2014-10-03 NADA Version 1.1 (October 2014) Version 2.0 (May 2015). Edited version based on Version 01 (DDI-MCC-GHA-WASH-2010-v1.1) that was done by Millennium Challenge Corporation. Water and Sanitation 2010 MCC-WS 2010 GHA_2010_MCC-WS_v01_M Notre Dame Initiative for Global Development (NDIGD) National Opinion Research Center (NORC) NADA Millennium Challenge Corporation Monitoring and Evaluation Division Independent Impact Evaluation Ghana Rural water Impact evaluation propensity score matching Health water sanitation Ghana water sanitation impact evaluation The Ghana Community Services Activity was designed to complement the Agriculture Project by providing educational, water and sanitation and rural electrification infrastructure in the Intervention Zones and by enhancing the capacity of local governments to deliver the related services. These interventions were part of a larger effort by the Government of Ghana to expand the provision of basic community services throughout Ghana, and were specifically expected to enhance the sustainability of the Agriculture Project by providing the necessary infrastructure to improve health of communities, to enhance skill development through access to education, and to facilitate small-scale post-harvest processing of agricultural products. The water and sanitation component of the Community Services Activity was designed to provide improved water systems to 137 selected communities in the Afram Basin Zone, Southern Horticultural Belt, and the Northern Agricultural Zone. Over the course of the compact, a total of 392 water points were constructed, including boreholes, small town water systems and pipe extensions. The Millennium Development Authority (MiDA), MCC's partner in Ghana, contracted the National Opinion Research Center (NORC) to design a rigorous evaluation of the impact of the improved water supply services and to conduct the baseline survey. Following closure of the Ghana Compact, MCC hired the University of Notre Dame Initiative for Global Development (NDIGD) to complete the endline survey and evaluation using the same methodology developed by NORC to estimate the impact of the water improvement project. The evaluation was designed to answer the following six hypotheses: 1. The incidence of diarrhea will decline, particularly in children 5 years old and younger. 2. Time devoted to acquiring water will fall significantly. 3. The price paid for drinking water will decline significantly, where water has been previously purchased. 4. Households will consume a greater quantity of water for domestic purposes. 5. Household consumption expenditures will increase. This evaluation employs a pretest-posttest with pair matched comparison groups. In order to assess the impacts of the water points, the research design uses a difference-in-difference estimate of the program impacts. These estimates account for changes over time (before and after the intervention) in the treatment and control groups and for differences in those changes between the treatment and control groups. The measure of the program impact is thus a double-difference estimate. In this study, difference-in-difference estimates of the program impacts will be obtained for each of the main outcomes of interest as defined in the evaluation design (i.e., incidence of diarrhea amongst children 5 years and younger, time savings in acquiring water, water expenditures, water consumption, and changes in household assets as a proxy for consumption). Ghana 100 communities (50 treatment, 50 control) within MiDA program areas in Ghana The main unit of analysis for the household survey is a household. Some of the questions in the household survey collected information at the individual household-member level, and other data is collected for each water source used by the household. The unit of analysis for the community survey is communities. Data is also collected at the water source level and school level. The population that is being studied under this evaluation are the beneficiaries of the water activity. A total of 153,000 people living in 27,407 households in 30 districts of Ghana were affected by the water activity. This evaluation will be conducted with 600 households in treatment communities. There are similar numbers from the control communities. Sample survey data [ssd] Panafields A two-stage survey design was used, in which the first-stage sample of communities was selected and a second-stage sample of households was selected from within each sample community. During the first stage, out of 137 program communities (where the water project was implemented), 50 communities were selected randomly during the baseline. These 50 communities served the treatment communities. The treatment communities were selected using variable probabilities after stratifying on key variables, and control communities were selected using matching based on variables obtained from MiDA and from vendors of geographic information system (GIS) data. Further, in the baseline to allow for nonresponse, two features were incorporated into the design: (1) the sample size was increased from the desired number of 50 communities for each group (treatment before, control before) to 60 for each group; (2) for each treatment unit in the sample, two matching control units were selected. Hence, the complete sample consists of a set of 60 matched triplets, of which the first 50 acceptable comprise the desired sample. If for any reason it is not possible to observe a particular treatment community in a matched triplet, that triplet is discarded and a replacement matched triplet is selected. For controls, the procedure was a little different. If the first-selected control unit was acceptable, then it was used as the matching control. If it was not acceptable, then the second-selected control unit was used. If both controls were unacceptable, then an unused control from the same district is used. The second-stage sample of households was selected using a starting point and pre-determined step interval to select a random sample of 12 households in each of the 100 communities. If the interview in a selected household was unsuccessful, the immediate next household in the interviewer's journey plan was used as a replacement. During the baseline, 1,200 households were selected to provide room for possible attrition. Several communities needed to be replaced during the data collection period. These replacements were due to inclement weather encountered during the rainy season except in the case of Dungu and Yong. Dungu was replaced because it was no longer receiving the MiDA intervention, Yong was its associated control community. The communities that needed to be replaced were: --Santa (treatment) --Tuvuu (control) --Sodzikope (treatment) --Akroso (control) --Dungu (treatment) --Yong (control) These villages were replaced with the following treatment and control pairs: --Nyanyano (treatment) --Gomoa Lome (control) --Tong (treatment) --Kpalguma (control) --Apaah (treatment) --Adidwan (control) This study employed two questionnaires, both of which were written in English and translated into local languages (Danmgbe, Twi, Dagbani, Ewe, Hausa) for use in the field: 1. The Community questionnaire, administered once in each of the 100 communities, and 2. The Household questionnaire, administered to 12 households per community. NORC's terms of reference instructed, to the extent possible, to use the questions from the fifth round of the Ghana Living Standard Survey (GLSS5+) as adapted for use in gathering baseline information. Their questionnaire replicated questions from the GLSS5+ in the household survey and the community survey. NORC also drew on three other sources in developing the questionnaire. First, they consulted the questionnaire employed in the baseline data collection for the impact evaluation of the Millennium Challenge Corporation (MCC)-supported water improvement project in Lesotho. Second, they studied the structure and the precise wording of questions employed in water usage surveys conducted by UNICEF, the Development and Health Survey, and the Environmental Health Project (USAID) that have set international standards and conventions. Third, they employed some questions from the Core Welfare Indicators Questionnaire-II (CWIQ). Questions on water supply, cost and usage differ in several areas from those in the GLSS5+ in order to obtain more accurate and comprehensive information on time spent acquiring water, the quantity acquired, the source of water by the purpose for which it was used, and hygiene related to water usage. The total interview time budgeted was 90 minutes, of which the majority is allocated to the all-important water sector questions. Throughout the process of questionnaire development, NORC continuously consulted with and sought feedback from MiDA. The household surveys includes modules on: -Introduction: Respondent consent and name(s), language, housing, occupancy status. -Household Members: Basic information on each household member, including age, gender, marital status, and other topics. -Household Education: Education information on each household member. -Household Health in the Last Two Weeks: Information on each household member's health in the last two weeks, including illness type, symptoms, treatment, length, severity, and other information. -Activity Status, Primary and Secondary Occupations - Last 12 Months: Information on each household member's occupations in the last 12 months, including job type, payment, amount of work done, and other information. -Water Use and Sources: Information on the household's use of different types of water sources, including the amount of water fetched, the time spent fetching water, cost, and uses. -Toilets and Hygiene: Information on the household's use of toilets and hygiene practices. -Household Unit, Durables, Fuel, Poverty Predictors: Information on materials used for household construction, ownership of durables, fuel use, and several poverty predictors. The community survey includes modules on: -Screening: Information on the community's history of water system improvement. -Public Water Facilities: Information on the facilities available to households within the community, including type of facility, number in the community, condition, and builder of the facility. -Schools Within a 30 Min. Walking Travel Time or 5KM Radius of Village: Information on all schools in or near the village, including name, type, year of establishment, and levels taught. These surveys were conducted as part of an impact evaluation, with the intention of conducting follow-up surveys in the future to measure changes in communities and households. NORC's contract with MiDA covered only baseline data collection, and Notre Dame will conduct the follow-up. Six field teams were employed, consisting of four interviewers plus one supervisor and one editor. Teams were organize to insure that all interviewers had a command of the local language in each community. Data collection took place over a period of 16 days, from September 23, 2010 to October 10, 2010. Although NORC initially estimated that it would take 2 hours to administer the surveys, the actual average administration time was closer to 70 minutes. The data is not weighted. After accepting the data entry files from Panafields, NORC exported all five data sets - four parts for the household survey and one for the community survey - to SPSS, then merged the four parts of the household survey into one dataset. This resulted in one Household dataset containing data from the 1200 household interviews and one Community dataset containing data from all 100 community-level interviews. To ease cleaning and analysis, the household dataset was then further split up into four additional relational SPSS datasets based on the primary unit of analysis for each question: one dataset for questions asking about the household as a whole; one for questions asking about individual household members; one for questions asking about different types of water sources; and one for questions about water collectors. Data processing consisted of the coding and input of data and in cleaning the database. Data cleaning was carried out for detecting and correcting, removing, or flagging incorrect data, errors in format, incomplete data, inconsistent data, etc. During data processing, two issues with the data that warranted further investigation were uncovered. The first issue was identified as a mistake in the data entry program and was corrected when discovered. The second issue was a misunderstanding by enumerators on how to code 'legitimate skips' in the questionnaire in reference to multiple household members fetching water from one source. This issue was resolved during data cleaning. Both issues were correctable and neither issue impacted the final quality of the dataset. The overall response rate for the household survey was very high at 91.6% (91.9% for treatment and 91.3% for control). The main reasons for non-response were: --No household member at home or no competent respondent at home at time of interview (5.5%) --Entire household absent for extended period (1.4%) --Refusal (0.8%) --Dwelling vacant or address not a dwelling (0.3%) Millennium Challenge Corporation Millennium Challenge Corporation http://data.mcc.gov/evaluations/index.php/catalog/112 Cost: None