Vacancy: Consultants , Ex-Post Evaluation of the USAID/Mali Out of School Youth Project (PAJE-NIETA) Evaluation

Job Description

CALL FOR EXPRESSIONS OF INTEREST

Ex-Post Evaluation of the USAID/Mali Out of School Youth Project (PAJE-NIETA) Evaluation

INTRODUCTION

USAID/Africa Bureau (AFR) in collaboration with USAID/Mali and United States Department of Agriculture (USDA) has commissioned the Analytical Support Services and Evaluations for Sustainable Systems (ASSESS) to conduct an ex-post evaluation of the USAID/Mali Out of School Youth Project (PAJE-NIETA).  The evaluation will examine the sustainability of the USAID/Mali Out of School Youth Project outcomes, six years after project completion.  Refer to Annex 1 for additional project background information.

EVALUATION PURPOSE AND USE

1. Evaluation Purpose

The purpose of this ex-post evaluation is to assess the sustainability of project-supported interventions 6 years after the project ended. The evaluation seeks to understand the extent to which PAJE-NIETA sustainability strategy was effective and is still relevant in addressing the education and business needs of out-of-school youth. The evaluation will examine the extent to which local structures, networks, linkages, and systems established or strengthened by PAJE-NIETA continue to function and contribute to out-of-school youth education, enterprise creation, and civic engagement. In addition, the evaluation will examine the extent to which the holistic and integrated out-of-school youth learning model empowered youth in creating and sustaining their own enterprises and civic engagement initiatives and how it can be replicated in similar activities. Finally, the evaluation will identify the factors behind the effectiveness and sustainability PAJE-NIETA interventions as well as lessons learned and best practices that can be broadly shared and replicated to improve the sustainability of future out of school programs.

b. Audience and Uses of the Evaluation

The primary intended audience for this evaluation is USAID/Mali, which will use the results to inform the design of an upcoming youth activity as well as to plan for future programs. The findings of the evaluation will also be beneficial for the Government of Mali, other donors, education stakeholders, the Africa Bureau and other USAID/Washington Bureaus which will acquire a better understanding of the sustainability of alternative education interventions catering to youth and use findings and lessons learned to improve the support and increase impact of future interventions. The ex-post evaluation findings will also contribute to the overall Agency’s learning agenda questions focusing on program sustainability which is one of the key USAID programming principles. Other Agency and Africa Bureau programming priorities and approaches the evaluation contributes to include resilience and multisectoral programming.

EVALUATION QUESTIONS

USAID AFR and USAID/Mali would like to investigate and understand the following key questions and lines of inquiry related to the effectiveness of the sustainability strategy and implementation approach employed by PAJE-NIETA. 

Evaluation Question 1: To what extent and in what ways PAJE-NIETA sustainability strategy contributed to maintaining jobs and enterprises for youth and still supports the creation of jobs and youth enterprises in Mali?

Evaluation Question 2: To what extent and in what ways were PAJE-NIETA basic education, work readiness, financial and business skills programs were effective in empowering youth to create their own enterprises and become more civically engaged in their communities? To what extent are these programs replicable in similar activities? 

TASKS AND RESPONSIBILITIES

The evaluation team will be required to conduct fieldwork in Mali. Fieldwork activities will include interviewing project stakeholders including USAID/Mali staff with direct knowledge of the project, PAJE-NIETA implementing partner staff, government representatives, representatives of youth association and youth volunteers,  savings groups, youth employers, as well as a non-representative sample of youth beneficiaries that can be located or others who live in the communities that were targeted by the project and are knowledgeable about the project. Interviews with other donors funding similar activities may also be considered.

DELIVERABLES

1. Draft Evaluation Design Proposal

Fifteen days following receipt of the final evaluation SOW, the evaluation team will deliver an evaluation design proposal that describes the conceptual framework for the evaluation and the justification for selecting that approach. USAID/AFR must provide its approval of the draft evaluation design proposal before the evaluation team begins in-country data collection.

b. In-Briefing with AFR and USAID/Mali

The in-brief meeting between USAID and the evaluation team will allow both parties to clarify evaluation expectations. Within 2 working days of arrival in Mali, the evaluation team will meet with USAID/Mali and virtually with the Africa Bureau in Washington for introductions and to discuss the team’s understanding of the assignment, initial assumptions, evaluation questions, methodology, and work plan, and/or to adjust the SOW, if necessary.

c. Final Evaluation Design/Work Plan

Within five days of commencing in-country work, the evaluation team will deliver a final evaluation design proposal that incorporates any changes or updates reflecting new information  obtained on the ground, a solid understanding of the project’s design and implementation approach based on discussions with Mission and other stakeholder staff who are familiar with the project, a review of additional data sources, and possible light revisions to the evaluation questions and methodology, including the sampling strategy.  The final evaluation design proposal will include all the elements listed in the draft evaluation design proposal, the final pre-tested data collection instruments, as well as the final work plan that details key evaluation tasks, timeline, and data collection instruments. 

d. Interim Meeting

The evaluation team will organize one virtual briefing session at mid-point of data collection to provide USAID/Mali updates on data collection progress and discuss potential challenges and emerging opportunities.

e. Outbrief/Presentation of Preliminary Findings

The team will make a presentation of key preliminary findings of the evaluation to USAID/Mali and USAID/Washington at the close of fieldwork and before the team departs Mali. The debriefing must include a discussion of preliminary findings, conclusions and recommendations. The evaluation team leader will share the draft PowerPoint slides for USAID’s review prior to the presentation.

f. Draft Evaluation Report

Within 7 days of the presentation of preliminary findings, the evaluation team will submit the draft evaluation report to USAID. The report should separately and comprehensively address each of the objectives and evaluation questions listed in the Scope of Work as well the findings, conclusions, and recommendations which should be clearly supported by the collected and analyzed data. Findings should be presented graphically where feasible and appropriate, using graphs, tables and charts.

The draft report should not exceed 30 pages in length (not including appendices, lists of contacts, etc.). Annexes should include the Scope of Work, description of the methodology used, lists of individuals and organizations consulted, data collection instruments (i.e., questionnaires and discussion guides, etc.), a bibliography of documents reviewed, and a table mapping all Findings/Conclusions/Recommendations.  

g. Final Evaluation Report

Following receipt of all USAID comments on the draft evaluation report, the evaluation team will have 5 days to prepare and submit a final version that incorporates and responds to USAID feedback. A translated French version of the report should also be submitted to USAID along with the English version.

h. Final Presentation

The final report is to be accompanied by PowerPoint presentation that aims to debrief select USAID/Washington stakeholders on the findings, conclusions and recommendations from the ex-post evaluation. A draft of the final deck should be submitted to USAID/Washington prior to finalization and the virtual presentation.

PERIOD OF PERFORMANCE

The anticipated period of performance is from January 24, 2022 - April 14, 2022. 

CONDITIONS OF REMUNERATION AND REIMBURSEMENT OF EXPERTS

The evaluation team will be remunerated at an agreed fixed price per day.

PLACE OF PERFORMANCE

The place of performance is Mali. The extent of travel will be determined by the evaluation design and data needs as agreed upon between ASSESS and USAID. A designated contact person from USAID Africa Bureau will serve as the primary Point of Contact and Contracting Officer’s Representative (COR) for the Ex-Post Evaluation.

ROLES AND RELATIONSHIPS

The consultants will be contracted under the USAID/WA ASSESS mechanism. ASSESS will be responsible for the administrative and technical management of their respective contracts. This will include monitoring of LOE and payments. This will include approval of the consultants’ work plan, assigning tasks within the scope of the contract, reviewing technical outputs and providing feedback.

ANNEX 1: PROJECT BACKGROUND

1. Mali Out of Youth Project Background


2. Introduction

The USAID/Mali Out of School Youth Project, also known as PAJE-NIETA (Project d’Appui aux Jeunes Entrepreneurs), was a five-year, $25 Million project implemented from 2010-2015 by the Education Development Center, Inc. (EDC) in collaboration with Catholic Relief Services (CRS), Swisscontact, Association Jeunesse Action (AJA) and for the Timbuktu region, Association Malienne pour la Survie au Sahel (AMSS). The project was jointly funded by the Education, Economic Growth, and Democracy and Governance Teams of USAID/Mali.

The project began with a Rapid Assessment of Malian youth conducted in April and May 2010 which recommended that primary attention be given to youth living in rural villages and peri-urban areas in an attempt to reduce migration from rural areas to urban centers.  The Rapid Assessment revealed that youth wanted to earn a living and achieve successful livelihoods in their home villages. It also revealed that youth are entrepreneurial and able to identify the inputs and knowledge they need to reach their goals, which, for rural youth, are predominantly in the agro-pastoral sector.

In order to address that need, USAID/Mali designed the Mali Out of School Youth Project with the goal to enable 10,000 rural youth between the ages of 14 and 25 who had either never been to school or who had dropped out in the early grades to become better educated, more economically productive, civically engaged and empowered to improve their lives and those of their families and communities.

1. Project Objectives/Intermediate Results

The project addressed four intermediate results of USAID/Mali’s strategic plan:

IR 1: Increased access to relevant basic education: The first project Intermediate Result was to enable youth to acquire basic academic skills, knowledge and attitudes needed to enter the job market. Youth were provided with work readiness, entrepreneurship and basic education training focused on literacy and math skills in local language as well as a functional level of literacy in French to negotiate the marketplace. This also included information needed to become better  educated, more economically productive and civically engaged through training in life skills, leadership and conflict resolution.  It was anticipated that at the end of the project, targeted youth would be better equipped with basic education and work readiness skills needed to earn a livelihood.

IR 2: Increased access to relevant skills training: The project’s second intermediate result was to enable youth to acquire technical skills needed to earn a living within rural and peri-urban settings. Rural youth would engage primarily in agro-enterprise development training while peri-urban youth may focus more on ancillary vocational education skills training related to the agricultural sector, such as fabrication of agricultural tools. Related to this, the project sought to build the capacity of rural vocational training providers, particularly with respect to their ability to train out-of-school and illiterate youth. It was anticipated that at the end of the project, targeted youth would be better equipped with vocational education and technical skills needed to earn a livelihood while local skills providers have improved capacity to prepare youth for work.

IR 3: Increased entrepreneurship opportunities and job networking: Under this Intermediate Result, PAJE-NIETA was tasked to enable youth to acquire entrepreneurship and work readiness skills needed for self-employment and job networking. The project provided youth with entrepreneurship and work readiness skills training and liaised with micro-finance institutions, local businesses, producers, and ateliers to increase economic opportunities for youth via self-employment or jobs.  At the end of the project, it was expected that targeted youth would gain viable employment opportunities (self-employment or other) in the agricultural and vocational education sectors.

 IR 4: Increased civic engagement: The project’s fourth intermediate result was to increase civic engagement at community, regional and national levels.  Youth development activities reinforced the notion of community and regional and national level civic engagement. The project supported youth in building the capacity of existing youth associations.  Youth associations were guided to collect data, analyze community needs, and plan and execute community service projects with assistance from the project in the form of small grants.

1. Project implementation approach and Sustainability Strategy  

Implementation Approach

EDC and its partners developed an integrated holistic model of the youth development cycle to be used as the main guide for project implementation.  The model is summarized in the diagram below:

The following figure depicts the Mali Youth learning pathway which implicitly conveys the project’s Theory of Change.

The project targeted a total of 10,000 rural and peri-urban youth, almost an equal number of boys and girls, aged 14-25 who were unschooled or out-of-school and had little or no literacy skills.  It covered the regions of Kayes, Koulikoro, Sikasso, and Timbuktu, and followed a cohort-model consisting of three consecutive youth cohorts.  Lessons learned from each cohort were used to improve interventions for the next cohorts.

• Cohort 1 (June 2011-June 2013) targeting 5,740 in 100 villages in the regions of Kayes, Koulikoro, and Sikasso


• Cohort 2 (February 2013-July 2014) targeting 1,255 youth in 21 villages (26 centers) in the Sikasso region


• Cohort 3 (June 2014-November 2015) targeting 7,858 youth in 107 villages (178 centers) in the regions of Kayes, Sikasso, and Timbuktu.

Key project interventions included providing basic education training for youth covering literacy, numeracy and life skills, upgrading the quality of technical skills for stakeholders involved in youth vocational training, providing accompaniment in the form of work readiness, financial, and business skills training as well as links to credit and savings programs to facilitate the purchase of start-up equipment. The project integrated information technology to support various interventions.

Sustainability Strategy

The project was designed with sustainability in mind. Key elements of PAJE-NIETA sustainability plan included:

1. Strengthening the capacity of youth serving institutions and a local training network:  The project provided organizational capacity building training to members of youth associations, all serving as volunteers.  Of particular note, the project incorporated in its design an innovative Mali Youth Volunteer Corps activity under which 200 educated, unemployed youth who were recent graduates, were recruited and trained to serve as community-based literacy, work readiness, and youth leadership trainers. Youth were also mobilized to design and implement community service projects in their villages.


2. Introducing the Private Sector Provider (PSP) Approach: The project initiated in its last year a private sector provider approach to enhance the sustainability of the project. Under this approach, communities selected a representative to serve as PSP candidate who received intensive training and get certified and prepared to provide services to community members after the project ends. PSPs receive payments for these services.


3. Partnering with government institutions including:
   
   
   
   • Ministry of Employment and Vocational Training
   
   
   • The Fonds d’Appui a la Formation Professionnelle et L’Apprentissage (FAFPA/ Professional Training and Support Fund) in order to ensure the stability and sustainability of the Mali Youth Corps.
   
   
   • The Ministry of Youth, Ministry of Agriculture and the Ministry of Education.
   
   
   • At the commune level: The mayor’s office, ministry representatives, community development councils, elders, and youth associations
   
   
   
   


4. Youth linkages to the private sector:
   
   
   
   • Support to ateliers for additional and improved skills standards, training modules, and implication in the accreditation process. Accreditation and certification will motivate youth learners in agro-enterprise and in ateliers to continue to use these local systems for skills training in the future.
   
   
   • Linkages to employment networks and market information system via mobile phones; fostering linkages between local telecoms and service providers (agricultural market data, employment services) to ensure that these services will remain available after the close of the project.
   
   
   
   


5. Linkages created for agro-entrepreneurs, such as improved access to new markets (through opportunity identification analysis), improved access to agricultural market data and new processing and marketing methods will allow producers to be sustainable after the project intervention cycle is complete.

d. PAJE NIETA Key Results

The project achieved impressive results under each of the above Intermediate Results.  These results are documented in the various project documents, including the final project report.   One of the key results worth highlighting is that over the life of the project, 88% of the youth who completed technical training and received an income generating activity (IGA) starter kit actually started their microenterprise and, of these, 94% were self-employed.  Most importantly, the project’s sustainability strategy proved to have been successful.   As of November 2015, 70% of youth in Cohorts 1 and 2 were still operating their microenterprises, with 64% of these as self-employed, even 24 months after the close of Cohort 1 development cycle and 12 months after the close of Cohort 2 development cycle. Between 83% (Cohort 1) and 85% (Cohort 2) of youth still in operation reported an increase in profits.  Given the great success of the project not only in generating important outcomes for beneficiaries but also in ensuring those benefits continued overtime, it’s important to assess the sustainability strategy and program implementation approach utilized by the project to see if they are viable models for replicating in similar activities and specifically for sustaining out-of-youth outcomes in the current Malian context.

II.  Existing Project Information and Data Sources

The following are the existing project information documents that might be useful in answering the key evaluation questions. USAID/Mali will share these and any additional documents with the evaluation team before the implementation of the evaluation.

1. Mali Youth Assessment Report


2. Project Description, Approval, and Modification Documents


3. Performance Monitoring Plan


4. Quarterly and Annual Reports


5. Project Baseline and Endline Surveys


6. PAJE NIETA Mid-Term Evaluation


7. Mali Out of School Final Project Report


8. PAJE-NIETA Final Evaluation Design (Power Point Slides)

Required Skills or Experience

EVALUATION TEAM COMPOSITION AND QUALIFICATIONS

USAID/WA has commissioned the Analytical Support Services and Evaluations for Sustainable Systems (ASSESS) mechanism to conduct the Ex-Post Evaluation of the USAID/Mali Out of School Youth Project (PAJE-NIETA) Evaluation. To this end, ASSESS is seeking to recruit four professionals to form the team to conduct this ex-post evaluation. It is expected that evaluation team members will have relevant prior experience in West Africa, be proficient in both English and French, have familiarity with USAID’s objectives, approaches, and operations, and prior evaluation/assessment experience. In addition, individual team members should have the technical qualifications identified for their respective positions.

The experts include:

1. Senior Youth Workforce Development Specialist who will act as the Evaluation Team Leader;


2. Senior Education Specialist


3. Agro-Enterprise Development Specialist, and


4. Monitoring and Evaluation Specialist/Logistics Coordinator.  

The following are the desired profiles of the team:

1) Evaluation Team Leader / Senior Youth Workforce Development Specialist (International)

The team leader will be primarily responsible for the quality of the evaluation design and its execution, including timely production of high quality deliverables associated with this evaluation. He/she will also be responsible for managing the team and liaising with USAID/Washington and USAID/Mali.  

• Must have a minimum of a Master’s degree in Education or an applicable social science field;


• Must have extensive knowledge of and 8-10 years of experience in youth workforce development, youth entrepreneurship/vocational training, and income generation activities for youth.  Experience in agriculture and finance sectors would be a plus.


• Knowledge of/familiarity with youth employment, and youth enterprise development in  Mali or similar African contexts


• Excellent knowledge of evaluation theories and methods.  Sound, proven knowledge of USAID Evaluation Policy and ADS 201 evaluation quality standards


• At least 7-year experience designing and carrying out evaluations of education in Africa and particularly those focusing on youth workforce development


• Track record of successful oversight, as team leader, of evaluations of complex international technical assistance projects


• Demonstrated experience managing multinational teams and producing high-quality and timely reports for USAID or similar audiences


• Proven experience leading the drafting/finalization of evaluation reports (report samples will be required)


• Extensive knowledge and skills in designing qualitative and quantitative survey research instruments and methodologies; excellent data analysis skills, including relevant data analysis software


• Strong knowledge of experience in positive youth development and gender integration and analysis   


• Excellent analytical and report writing skills


• Fluent in English and French

2) Sr. Evaluation Specialist/Sr. Education Specialist (International)

The Evaluation Specialist/Sr. Education specialist will work in close coordination with the Team Leader and will be actively engaged in efforts to oversee and ensure the quality of data collection activities, ensure that data codebooks are clearly written, and that all data collected can be properly transferred to USAID.  He/she will have the following qualifications:

• Must have a minimum of a Master’s degree in Education or a related social science field


• Extensive experience in out of school youth education, capacity building, and vocational training for youth


• Knowledge of/familiarity with youth employment, and youth enterprise development in  Mali or similar African contexts


• Strong knowledge and application of the positive youth development principles  


• Gender knowledge and experience highly desirable


• Minimum of 7-year experience conducting evaluations of donor-funded out of school youth education or youth workforce development activities


• Excellent quantitative and qualitative research skills and demonstrated ability to undertake content analysis and write a full or components of an evaluation report


• Fluent in both English and French

3) Agro-Enterprise Development Specialist (Local)

The local agro-enterprise development specialist will work under the leadership and supervision of the evaluation team leader who also serves as Youth Workforce Development Specialist. He/she will contribute to the evaluation design and data collection and analysis, particularly bringing his/her sectoral expertise and local knowledge on aspects related to agro-enterprise development, the main source of income generation and youth employment in African rural communities.  He/she will have the following profile:

• Minimum of bachelor’s degree in Agriculture, Economics, or Business Administration; Master’s degree preferred


• Strong knowledge of and experience designing, managing, and/or conducting research or evaluations of agro-enterprise, livelihood, or other income generation activities


• Experience in market linkages and savings and lending schemes in the Mali context 


• Well versed in Mali context and culture; significant experience working with communities and associations in the geographic areas covered by the ex-post evaluation


• Must have detail knowledge of Mali out of school youth issues and key stakeholders involved in addressing these challenges


• At least 5 years’ experience consulting for International Development agencies on Mali youth enterprise activities and developing research products based on quantitative and qualitative data gathering


• Fluent in French and English

4) Monitoring and Evaluation Specialist/Research Coordinator (Local)

The local M&E Specialist will support the evaluation team leader and Sr. Education Specialist in all aspects of data collection and analysis.  He/she will be primarily responsible for reviewing data collection tools and ensuring data collection follows standard M&E protocols.  He/she will also play an important role in sample design, instrument design, and data coding and statistical analysis. In addition, he/she will assist with all data collection logistics.   He/she will have the following qualifications:

• Bachelor’s degree in the field of Statistics, Sociology, Economics, and/or other relevant field;


• Minimum of 7 years of experience in monitoring and evaluation


• Strong understanding of data collection and analysis methodologies applied to program evaluation; including surveys, key informant interviews, and focus-group discussions


• Substantial experience in designing participating in/or contributing to evaluations of international development programs


• Must have good knowledge of USAID programming policies and practices.


• Proficiency in quantitative and/or qualitative analysis software such as SPSS, Stata, NVivo, etc.


• Strong experience managing logistics for research and evaluation activities.