It has been a year and a half since Alison Vicary, Sean Turnell, and Wylie Bradford of the Burma Economic Watch Team published their output-based analysis of the Three Diseases Fund. They promised to continue to examine the issue of evaluation of the Three Diseases Fund but have not to the moderator's knowledge done so.
Finally, though, the mid term evaluation of the Three Diseases Fund has been done and the report published.
It is a rigorous evaluation and there is something in it for everyone. You can read the full report on the Three Diseases Fund website or can download it at http://www.hivinfo4mm.org/blog/_archives/2009/12/1/4394414.html#attachments The executive summary is below.
Read.
[him] moderator
******************************
1. The overall conclusion of this Mid Term Evaluation (MTE) of the Three Diseases Fund (3DF) is that the Fund has performed well and has made a significant contribution to the containment of the three disease epidemics. The supporting donors, Fund Board (FB) and Fund Management Office (FMO) have worked well to mobilise resources and operationalise the Fund speedily and generally to good effect.
2. However, donors and the Government of Myanmar should recognise that a reduction in the overall disease burden will require sustained health system support with significant increases in funding nationally and internationally. Within the current constrained political environment, this is not possible and the 3DF should continue to provide its focused humanitarian support.
3. The findings with respect to the impact of the 3DF are that:
x the Fund has functioned satisfactorily in a constrained political and operating environment where other attempts to support humanitarian responses to the HIV, tuberculosis and malaria epidemics in Myanmar failed;
x there is broad appreciation of the Fund and the FMO by most stakeholders and partners and overall the FMO has functioned effectively;
x whilst the evaluation was only able to identify modest directly attributable outcomes, had the Fund not been in place, service provision and access would have been considerably lower;
x there needs to be closer attention paid to processes, procedures, and performance particularly by the FMO if impact is to improve;
x scaling up responses and improving performance at this stage requires focus by all partners and attention to planned consistent progress that builds on success;
x given the context, the 3DF should accept its role as providing limited humanitarian support and avoid overambitious attempts to extend its reach and function; the governance arrangements simply spell out who does what within the institutional framework; they need to be maintained and confusion reduced by the Fund Board (FB) giving greater clarity and explanation where necessary.
4. The 3DF has had the following impact on the three diseases:
x The fund has been successful in averting deaths and reducing illness due to the diseases (see para 3.3 ñ 3.5 and table 1 page 13).
x Fund performance shows that 5-10% of need has been met by services provided with Fund support (see para 3.2).
x The Fund has contributed 30-50% of achievement towards the identified national targets and possibly even greater contribution to nationwide outputs (para 3.2).
x The level of effect of the Fund is significant in the Myanmar context, especially given the level of need compared to available resources (para 3.6).
x The Fund has helped ensure the epidemics have at least been contained (para 3.7).
x However, the level of effect is unlikely, on its own, to have contributed to reduced mortality and morbidity nationally (para 3.8).
5. In addition, the MTE identified the following indirect impact of the 3DF (para 3.9):
x The Fund has demonstrated that it is possible to successfully provide health services to vulnerable groups in Myanmar.
x There is better understanding of the epidemics through the funding of national surveys and research by the Fund.
x Through the TSGs and other partner interaction, there is shared understanding of the issues and who is doing what.
x The MoH has taken responsibility for the TSGs; they are functioning better and there are some examples of coordination.
6. Programme Performance
The programme of activities delivered by Implementing Partners and funding by the 3DF is based on a sensible mix of prevention, care and support. Interventions are generally congruent with the national strategies. Choices of the level and extent of support seem well founded and are intended to be complementary to public health.
7. Overall targeting of geographic areas and identified high risk groups has been appropriate. However, coverage is low relative to need so that at scale impact is limited.
8. Monitoring, Evaluation and Impact The M&E system has too many disparate and disconnected roles and functions and too much information is generated through partner monitoring. Information generated does not seem to be used to monitor strategically.
9. Impact assessment needs to be improved, focusing on qualitative monitoring, operations research and analysis of outputs to reflect more strategic thinking and oversight.
10. Fiduciary and Operational Arrangements
The 3DF disbursed some $43.5m during the bridging period and the first two years of Round I. The FMO has established good financial management procedures and staffing levels are appropriate for maintaining standards. Audit reports were ësatisfactoryí and give a good basis for follow up.
11. IPs value the accessibility and approachability of the FM finance staff ensuring that problems can be resolved as they arise. The issues of overhead costs needs to be addressed.
12. The absence of the intended three year costed operational plans developed by the TSGs and approved by the MoH has weakened the planning and financial planning base.
13. Institutional Arrangements
The Cycle of Fund investment based on a focused assessment of operational plans was never established. As a result there has been no incentive for MoH to convene the Coordinating Body and frustration in TSGs.
14. The TSGs have developed role clarity and focus that extends beyond GFATM and 3DF, a particularly positive development in the institutional arrangements for health planning for the three diseases.
15. MoH has increased ownership and responsibility though pressure from above, and challenges below need understanding. IPs continue to have a mixed appreciation of the wider context and purpose of the Fund and, at times, of the governmentís role.
16. Governance
The fundamental principles of the governance structure have been maintained and are still appropriate. Governance structures are continuing to function largely as foreseen. However, there is insufficient clarity over roles and functions throughout the structure. The central role of the Coordinating Body (now the Country Coordinating Mechanism) and the relationship between the MoH and the FB should be reviewed, and a clearer distinction is required between the role of the FB as policy lead focusing on the big picture and the role of the FMO focusing on implementation.
17. There remains confusion over the position of the FMO as the independent office of the 3DF responsible to the Fund Board and its management by UNOPS. Role clarity here is essential for fund governance, and the Fund Board ñ UNOPS ñ Fund Management governance relationship remains a significant risk factor that needs to be addressed.
18. On a day to day basis, fund administra
tion is being carried out effectively and the FMO staff team under the leadership of the CEO has administered the 3DF well under difficult circumstances. The interface with MoH, UN agencies and other IPs needs to be based on a clearer appreciation of the boundaries and policy limits.
Main Recommendations
Performance
1. The Fund Board reconfirm with the MoH the agreements underpinning the Fundís operation, including its humanitarian brief and the role of the MoH in leading the planning process and programme development.
2. A more comprehensive approach to beneficiary accountability is developed with a broader sample of IPs involved..
3. The FMO streamline current M&E systems and processes by refocusing the M&E to undertake partner performance (rather than partner activity monitoring); facilitate assessment of Fund performance and impact assessment at a strategic level; and deepen the linkages between analysis of disease trends and epidemiology.
Operations
4. The 3DF FMO engage with the chairs and secretariat to provide support to re-stimulate the planning and budgeting capacity of the TSGs to operate on a more dynamic basis to produce three year rolling plans.
5. The 3DF should abandon the present reward system based on fund reallocation.
6. Capacity Building for partners (including CBOs) is not a core function of the FMO and these activities should be discontinued and capacity development built into contracts.
7. As the 3DF moves towards a possible second phase, the Fund Board should give consideration to moving fund allocation to a model based on the 3DF as commissioning agent and IPs as service providers, forming the middle tier between national plans and service delivery plans of IPs and linking strategic planning with decisions on resource allocation to programmes.
8. The Fund Board should reverse the decision to pilot FMO implementation of fund flow mechanisms to township level and defer any discussion of future modalities until the WHO review is complete and after commissioning its own in-depth fiduciary risk assessment of the financial management arrangements at township level.
Governance
The Fund Board should:
9. clarify and strengthen the separation of roles between the Board (policy and oversight) and the Fund Manager (implementation and monitoring);
10. appoint an independent Board Secretary to be responsible for minutes and progress chasing of Board actions;
11. follow up and close on all audit findings and recommendations; and
12. review the operating relationship with UNOPS including regular meetings with the Board chair, UNOPS Regional Director and 3DF CEO to ensure UNOPS fulfils its commitment to support the Fund Board and provide an enabling and responsive service.
Management
The CEO should:
13. implement a staff development process to ensure clear understanding of roles and functions by the FMO staff and provide a systematic learning mechanism within the FMO;
and 14. undertake a review of staffing levels, skills and structures and recommend to the Fund Board possible changes in the structure for the FMO to ensure a focus on the core functions.
http://3dfund.org/midtermEvaluation
