Development of Out-of-school Youth Monitoring and Evaluation System (OSYMES) for USAID Opportunity 2.0 Program

Abstract

The USAID-funded Opportunity 2.0 (O2) program supports the Philippine education system in providing second-chance opportunities for 180,000 out-of-school youth (OSY) to pursue employment, entrepreneurship, or further education. A crucial element of the program is the monitoring and evaluation (M&E) system, which ensures effective program implementation and resource utilization. However, the current M&E system, M&E Insight, developed by the Education Development Center (EDC) in the US, has faced challenges in meeting the evolving data and reporting needs of the program. The local M&E team, with limited expertise in the system’s underlying programming language (C#), and the complex process of change management with the US office, struggled to address these issues promptly. As a result, the team resorted to manual data entry and consolidation through Youth Trackers, which presented challenges in data validation, deduplication, and report generation.

To address these challenges, the Out-of-School Youth Management and Evaluation System (OSYMES) was created to improve data collection, reduce reliance on manual processes, and ensure more accurate and timely reporting. Built with PHP, OSYMES offers a more flexible and user-friendly interface, enabling the local M&E team to quickly adapt to evolving program requirements. The system also incorporates automated deduplication and multithreading to enhance processing efficiency, while leveraging cloud-based services through AWS Lambda and S3 to optimize resource management. Additionally, the inclusion of a reporting dashboard provides stakeholders with real-time reports to support decision-making. By addressing the limitations of M&E Insight, OSYMES represents a significant step forward in enhancing data management and improving the overall effectiveness of the O2 program's efforts to support out-of-school youth in the Philippines.

The development of OSYMES utilized a V-shaped model process. Its prototype was set-up in an AWS environment, which acted as the testing environment for the M&E Team from different O2 project sites to participate in the testing. Testing results showed that OSYMES met the expected requirements, allowing the team to do away with manual data consolidation.

To understand how well OSYMES was received, a survey was conducted to see how useful and easy it was to use. Everyone agreed that OSYMES would significantly boost their productivity and make tasks easier to complete, especially with the deduplication feature, which streamlined a lot of their work. While most users found the system easy to learn and navigate, they suggested that some additional training and a phased rollout would help make the transition smoother. Overall, the team felt that OSYMES was a big improvement over the previous M&E Insight system.