Agency-level Measurement: A Case Study from Catholic Relief Services
https://www.crs.org/our-work-overseas/ict4d
Kelly Scott is a Technical Advisor for Monitoring, Evaluation, Accountability and Learning with Catholic Relief Services.
Within the international development and relief industry, there has been a concerted focus on increasingly sophisticated project-level monitoring and evaluation (M&E) systems. But how does an agency working in many countries, implementing many different types of programs, go about measuring progress at an agency level? How does an agency like Catholic Relief Services (CRS) aggregate information in order to tell our story at the global level? A new journal article titled Measuring Agency-Level Results: Lessons Learned from Catholic Relief Services’ Beneficiary and Service Delivery Indicators responds to these questions by presenting a case study of CRS’ agency level measurement system. This article was recently published in the peer-reviewed Journal of Multi-Disciplinary Evaluation.
The article chronicles the development and evolution of CRS’ Beneficiary and Service Delivery Indicators (BSDI) initiative. The BSDI initiative provides CRS with an aggregated and global view of who we serve and what services we provide. Now, with over five years of BSDI experience under our belts, we wanted to share our reflections with peer organizations, in the spirit of collaborative learning.
The article opens by describing the demand on and response from international agencies to strengthen M&E systems and approaches, both at the project and agencies levels. The article then dives into CRS’ case study – providing a chronological account of the evolution of the BSDI initiative – documenting its development, as well as bumps and successes along the way. The case study examines the many factors that influenced the success of system and its adoption at CRS, and closes with a straightforward section that lays out a concise set of key lessons learned and accompanying recommendations, which include:
- Starting small before scaling up
- Providing detailed guidance, training and tools to help staff collect and analyze the data
- Choosing technology platforms that are easy for local staff to access and use
- Ensuring high-level administrative and leadership support
- Not underestimating the amount of labor needed
CRS is committed to making continued improvements to the BSDI initiative in the coming years. We are currently examining how to improve data utilization at all levels of the agency, as well as how we might expand the types of data we will collect in future.
We invite our peers and colleagues to read the journal article with the hope that our experience and learning may inform your own journey towards developing agency-level measurement systems.