Millennium Challenge Corporation 2015-01-21 NADA Version 1.0 (January 2015) Version 2.0 (April 2015). Edited version based on Version 01 (DDI-MCC-MAR-IND-ME16LOT3-2013-v1) that was done by Millennium Challenge Corporation. Functional Literacy 2013 MCC-FL 2013 Alphabétisation fonctionnelle MAR_2013_MCC-FL_v01_M Transtec-Attitudes Conseil Consortium NADA Millennium Challenge Corporation Monitoring & Evaluation Division Independent Performance Evaluation literacy training education adult education Agriculture and Irrigation This performance evaluation uses a pre-post methodology to evaluate the functional literacy subactivity of the Artisan and Fez Medina Project. The evaluation followed the Kirkpatrick model trying to assemble a set of qualitative and quantitative evidence. To do so the evaluation team questioned 6 main criteria : relevance, coherence, efficiency, effectiveness, impact and sustainability and also assessed the program gender and environment approaches. 17 focus groups, 500 respondents to a survey and 70 semi-structured interviews contributed to collect primary and secondary sources of information so to triangulate findings. Morocco Marrakech, Souss Massa Draa, Tanger-Tétouan, and Fez Boulemane Individuals There were 70 semi-structured interviews, 17 focus groups, and a survey of 500 beneficiaries in the four intervention regions. The objective was to include the maximum number of direct beneficiaries in each of the three data collection events. The overall size of the sample was kept to 1,297. Sample survey data [ssd] Data Ingénierie In order to meet the objectives and guarantee the quality of the survey results, the Consultant opted for stratification of the target populations to form homogeneous groups by variable, correlated with the variables of interest. Effective stratification not only yields better representativeness of the population as a whole but increases the accuracy of the results for homogeneous groups. Notwithstanding, this method will not consist of a uniform comparison of the stratification variables, but instead, differ by level, due to the small size of the sample to be surveyed. The aforementioned Methodology Note submitted at the conclusion of Phase 1 contains the definition of the sampling criteria (stratifications), which respect the sites indicated in the TOR for the functional literacy interventions. These criteria include a specification over and above the specifications in the initial proposal to better respond to the evaluation objectives, adapt to the concentration of beneficiaries in the selected regions, and guarantee the total number of interviews to conduct according to the TOR, while respecting the need to include the gender dimension in the evaluation. In preparing the survey plan, contractual obligations were respected, along with the survey constraints, notably the available resources and the delays that had occurred. In choosing the sampling technique, both bias and variance had to be minimized. Given the rich sampling base that was available (exhaustiveness and reliability) and the evaluation context, the Consultant opted for a systematic survey with equal probability within the strata constructed. This type of survey is very common, as it offers better-quality results and requires no adjustments of any type; however, the sample is widely distributed geographically, which had direct implications for the cost of collecting data in the field. The beneficiaries will therefore be classified in a very specific order (by the different types of stratification variables), and the sample will be drawn systematically with a probability ranging from "equal" to "irregular." The starting unit is drawn through simple random selection among the numbers from 1 to "not drawn." The units to survey are then drawn automatically using appropriate software. The size of the total sample for the AFM and FLVT evaluations, originally set at 1297 individuals in the TOR, was cut to 770 following APP’s decision, communicated on 4 July 2013, to eliminate the Vocational Training and PEAQC activities from the Transtec-AC Consortium evaluation project. The final sample wovered 4 regions instead of 14. Four data collection tools were used in the evaluation of the functional literacy subactivity: -A pre- and post-activity document review -Field surveys of beneficiaries -Three types of focus groups (with beneficiaries, literacy workers, and functional literacy service providers) -Semi-structured interviews with all categories of stakeholders in the functional literacy program Data Ingénierie and the Statistician expert led a team of 3 moderators, 7 supervisors and interviewers. The following variables were used to stratify data: sector (fishery, handicraft, agriculture), implementers, region, zone, gender, age CSPro also enables users to set up a data entry control module to monitor proper entry of questionnaire data, automatically checking the consistency of the information inputted. The software allows users to create validation rules to ensure correct data entry through well-defined rules or error messages in the case of logical inconsistencies. Millennium Challenge Corporation Millennium Challenge Corporation http://data.mcc.gov/evaluations/index.php/catalog/145 Cost: None Transtec-Attitudes Conseil Consortium. Final Evaluation of the "Artisan and Fez Medina" Project and the "Functional Literacy and Vocational Training." November 2013.