Alison Vicary, Sean Turnell, and Wylie Bradford of the Burma Economic Watch Team here use the Fund for HIV/AIDS in Myanmar as a case study in looking for that Holy Grail of economists: cost effectiveness.

The tone of this report can be described by the [him] moderator as 'snarky'. College sophomores use the same style. So the paper would not be accepted for publication in a peer reviewed journal. But then the Burma Economic Watch has never presented itself as one.

[him] moderator

******************************

NOT MUCH BANG FOR THE AID BUCK
FUND for HIV/AIDS IN MYANMAR (FHAM)
Burma Economic Watch Team
Alison Vicary, Sean Turnell, Wylie Bradford
Thursday, July 10, 2008

The handing over of money to the international NGOs, UN agencies, Burma’s government (I use the term loosely) and the Burmese NGOs does not mean that the resources are used effectively and efficiently for the people of Burma. Uncovering the performance of aid - that is its cost effectiveness and its impact on the intended recipients is not necessarily an easy task. It is also a task made more difficult by the poor quality of the information generally provided by the donors and the recipient organisations.

We can not hope to address the performance of all the aid provided to Burma, but let’s have a brief look at the use of funds supplied by the Fund for HIV/AIDS in Myanmar (FHAM) to the Burmese NGOs. FHAM provided monies to the international NGOs, Burmese NGOs, the UN agencies and the Burmese ‘government’ between 2003/04 till 2007/08. The funds were provided by Norway, Sweden, the United Kingdom, Australia and the Netherlands. It was the first multi-donor health fund established in Burma and distributed around USD27m to the different organisations. The Global Fund came to Burma in 2004/05, but only hung around for 1 year having disbursed around USD11m. After the closure of FHAM and the exit of the Global Fund, those donors that had established FHAM set up the 3 Diseases Fund. FHAM only provided monies for HIV/AIDS projects, whereas the 3 Diseases Fund (as with the Global Fund) provided money for TB and malaria in addition to HIV/AIDS projects.

So lets have a brief look at what FHAM says was the performance of the Burmese NGOs. By Burmese NGOs I meant the following organisations, each with the differing reputations and agendas:

1) Myanmar Medical Association (MMA)
2) Myanmar Nurses Association (MNA)
3) Myanmar Health Assistance Association (MHAA)
4) Myanmar Red Cross Society (MRCS)
5) Myanmar Anti-Narcotics Association (MANA)
6) Myanmar Business Council on Aids (MBCA)
7) Pyi Gyi Khin (PGK)

The MMA, MNA and the MHAA have been around for a long period of time, as has the MRCS. The history or background of these organisations is for discussion elsewhere. MANA is a more recent invention of the regime and its cohorts, parading as an NGO. The advertised mission of the NGO is to reduce the problems associated with drug use. One could be cynical and think that one of the missions of the organisation is to obtain aid money, given the popularity of funding HIV/AIDS projects, but lets just see what they did with the money from FHAM. MBCA was allowed to register in 2000, but due to delays (unspecified) it took them 3 years to get started. The impetus for the organisation came from their namesake in Thailand. Then there is the mysterious new NGO, Pyi Gyi Khin, which translates roughly as the Mistresses of the Big Country. The ridiculous name might give some indication of the origins of the organisation, which has been reported to have begun in 1997. This organisation, which has no profile, does not appear in the regime’s press, managed to establish itself as a ‘national’ NGO, such that it has been able to attract large amounts of funds (in the context of Burma) from foreign donors. Maybe the Myanmar Maternal Child and Welfare Association was too much on the nose for the Western donors (though not for the Japanese or the UN agencies) to be able to be given funds, without complaints from their constituencies at home. Since the UN agencies have no constituencies they don’t have to worry. We have no idea, what the Japanese public thinks about the use of their aid funds. It is difficult to imagine that given the severe constraints that exist on the capacity to organize in Burma that a new organisation could suddenly appear, without being backed by the regime. Maybe they are doing good job?

It should be noted that the per capita expenditure on healthcare in Burma is only USD4. Most of this is expenditure is undertaken by the individual out of their pocket, but a small proportion of this comes from foreign donors. Given this very low expenditure on healthcare it is important that aid funds be allocated efficiently and effectively.

So let’s get to what FHAM says about the performance of these organisations. The record keeping of the NGOs supported by FHAM was precise, with the number of people attending sessions, condoms distributed meticulously counted. The attention and resources allocated to counting must have been considerable. Unfortunately, all the false precision makes it resemble an article from the New Light of Myanmar. Even more unfortunately the information is from a report funded by the governments of some of the Western democracies.

Myanmar Medical Association (MMA) - USD21,390

The MMA received the smallest amount of funds from FHAM, at USD21,390. To be fair to the organisation they only received the funds to tide them over after the Global Fund left. For this money the MMA (or members of the organisation) apparently achieved the following:

    * tested 591 people for sexually transmitted diseases (STD). The number of those testing positive was not disclosed and whether they were treated effectively was also not provided. It is not clear if this was the total number tested or the increase in the numbers tested (by whom exactly is not clear)
    * treated 1,487 people with HIV/AIDS for opportunistic infections. No information was provided about the nature of the treatment, the infections or the results. It is also not clear, if this was the total number of people treated or the extra amount that could be treated given the funds provided
    * referred 76 people for HIV tests

Assuming that referrals don’t entail the use of additional resources, and that the above figures represent the number of patients attended for the money provided, then it cost the MMA around USD10 to test for STD and treat people for HIV/AIDS. The per capita expenditure on healthcare is USD4, so if the above is correct, then the may have done a reasonable job. At least the MMA utilized their funds the most cost effectively of all the Burmese organisations (on the basis of the information provided by FHAM).

Myanmar Nurses Association (MNA) – USD91,000

The MNA received around USD91,000 for projects in 2003/04 (the first funding round). Despite receiving this money they did not manage to impact on any of the so-called performance indicators developed by FHAM. The MNA was not directly funded in the following rounds and maybe their lack of performance was the reason, but became part of a ‘consortium’ headed by Save the Children (UK). So based on the information provided by FHAM the USD91,000 added nothing to improving the health of the people of Burma.

Myanmar Health Assistants Association (MHAA) – USD42,000

The MHAA as with the MMA only received the funds at the end of the FHAM, to cover them after the Global Fund stopped funding projects in Burma. For the 42,000 the MHAA made the following contributions:

    * 38, 206 (very precise) condoms were distributed. Burma seems to be awash with condoms. The issue is whether people are using them regularly, not the number being given away.
    * Conducted 28 health education sessions on HIV/AIDS for 15,148 young people (of which 604 received counseling). So on average there were 541 people at these gatherings, rather large for health education on such sensitive issues. There is no information that such gatherings are successful in communicating information about HIV/AIDS, and more importantly in affecting behaviour. There is no information about where the education sessions were conducted, advertised and the reasons people attended. Incentives are an important determinant of behaviour and new information, even if properly understood, does not necessarily change behaviour.
    * 13,249 pamphlets were distributed. Presumably people attending the education sessions were provided with some pamphlets. Not that there is anything wrong with pamphlets, but there mere existence provides absolutely no indication that behaviour of those reading pamphlets changes. How many smokers have seen “DON’T SMOKE” and continue to do so. No NGO working in a Western country could get up and state we distributed blah number of anti-smoking pamphlets. People would ask, but yes what has happening to number of people smoking.
    * 190 peer educators trained. Again this tells us nothing about impact. There is no information even on the quality of the training, the length of the training, and what the peer educators learnt from the training. What were the incentives of those undertaking the training? Did they have nothing else to do? How active are they in the communities? Training people is one thing, but does not tell us anything about quality and what the educators are doing.
    * 310 people were given HIV tests
    * 195 people referred for HIV tests

If we accept that HIV tests can be provided for USD10, as based on our estimates from MMA use of monies, then the MHAA could have tested the 310 people for a total of USD3,010. As the condoms and pamphlets were provided free and distribution costs should be minimal, any costs associated with this activity are ignored. It is also assumed that referring people for tests is undertaken in the course of other activities, hence entail no additional expenses. Based on this and assuming that the costs of training a peer educator and providing health education sessions for the population incur equivalent costs, then each training or workshop was provided by the MHAA at around USD178. If the costs are calculated on a per person basis, i.e. the number of people reached by the organisation then, then it cost the MHAA around USD2.50 to provide each person with HIV/AIDS information.

Myanmar Red Cross Society (MRCS) – USD245,000

The MRCS seemed to engage in lots of meetings and distributed lots of condoms. Not much for a ½ million USD. Lets look a bit more at what they did

    * 411,378 condoms distributed - again did they put them on?
    * 6 Mass Awareness Sessions held by MRCS. It seems that 21,689 young people were educated at these sessions. They must have been big events with more than 3,5000 young people attending each. Of the 21,689 young people reached, 3,259 were reportedly provided with counseling. FHAM reports this as video shows, TV spots, public talks, festivals. Possibly unfairly, but mass awareness sessions in Burma brings to mind having to listen to mind-numbing speeches given by mind-numbed government apparatchiks to people, who have been rounded up and forced to listen. We might be cynical, but if any of the poor participants were left awake at the end, maybe they picked up something about HIV/AIDS, but who knows what.
    * 30,917 pamphlets or HIV/AIDS educational materials distributed, possibly at the mass rallies. At least local the printeries must be doing well. (Wonder, who owns them?) Am I being cynical to suggest that there might be lots of pamphlets lying around in some near empty MRCS warehouse? Hopefully, they were put to better use pasted on the walls of people’s homes to improve insulation.
    * 2 Workshops for healthcare providers and 91 peer educators trained
    * 72 people referred to services for sexually transmitted diseases. Maybe after the mass meetings or a discussion at the tea-shop, the educators tell their peers “Hey you better go and get a test. Could have caught something”
    * Lots of Meetings
          o 46 advocacy meetings
          o 48 multi-sector meetings

Does anything more need to be said about this? The New Light of Myanmar comes to mind. The meeting was held and advice was given.

Well it appears that for ½ a million US dollars, the MRCS held some mass rallies, and handed out a lot of condoms and pamphlets, but provided no healthcare. Condoms and pamphlets are provided free and the costs of distribution in a local area would be very small. Referring people for testing is also costless and presumably arose out of the counseling sessions. If we assume (for simplicity) that the costs of putting a workshop, a mass rally, sessions, and training peer educators are the same, then it cost the MRSC around USD5,800 to provide each service. As expenditure on healthcare is only USD4 per capita, the costs incurred by the MRCS to produce their services is outrageously cost inefficient. If the costs are calculated on the number of people reached by the education provided by the MRCS, then the NGO spent more than USD11 for each person. Again a ridiculous allocation of resources, when the per capita expenditure on healthcare is only USD4, coupled with there being no evidence that this education has had any impact on behaviour. Even, if everyone educated, trained and counseled came out understanding the issues, changed their behaviour and became active peer educators (which is highly unlikely), the amount of resources allocated to this area of healthcare is questionable. Surely, HIV/AIDS education can be provided much more cheaply.

Myanmar Anti-Narcotics Association (MANA) – USD280,00

    * 5 drop in centres (established or run was not specified) and 58 workshops were held, presumably in the drop in centres. From this 868 intravenous drug users (IDU) were reached. 1 of these was referred for drug treatment, and 96 received HIV tests.
    * 48,000 syringes were distributed, though none were 868 – an usual needle exchange program. It is not clear, who obtained the syringes, but presumably they were distributed to the 868 IDU’s reached.
    * Total of 9 reports, which included 8 evaluations, reviews and 1 survey were undertaken. None of these have been made public.

Let’s assume initially that all the monies were allocated on the 868 IDUs, who were provided with 58 workshops and 196 received HIV tests. If we assume that each test cost USD10, as was the case with the MMA, then MANA would have needed to allocate around USD1,960 of their resources. This left them with around USD278,040 for other activities. Since there program reached 868 IDUs, (of which 196 were tested for HIV and 1 referred for drug counseling), then around USD320 was spent on reaching each IDU. If every IDU, who came in contact with MANA changed their behaviour (highly unlikely), then MANA this still means that a small number of people where provided with an inordinate amount of resources, especially when this is compared with the average expenditure on healthcare of only USD4.

Of course MANA did produce 8 reports, evaluations and undertook 1 survey, about which there is absolutely no information. Of course some resources would have been necessary to produce this output, which lowers the amount of resources required to produce the above outcomes. So if half of MANA’s funds were allocated to producing their reports, then the costs of reaching each IDU would have been still ridiculously high at USD160. This would also mean that a ridiculous amount of scare health resources were allocated to the production of reports, which are not even available to the public that paid for them (or to the people of Burma, who lost precious healthcare resources for their production). Another approach to examining the efficiency of the allocation of resources by MANA with regard to the production of their reports is to use the expenditure incurred by MHAA in the production of their outreach services. MANA, unlike the MHAA did produce 9 reports, so if we assumed that MANA could actually provide workshops and outreach at the same costs, then MANA could have provided all their services to IDUs at a cost of around USD10,000. This would have left around USD235,000 for the 9 reports, with each costing on average around USD26,000. Either way the allocation of resources by MANA is extremely cost inefficient.

Myanmar Business Council AIDS (MBCA) – USD273,000

    * 13,649 condoms 30,995 pamphlets distributed
    * 110 health education sessions & 10 mass awareness sessions, though no-one was reported as being reached. Maybe there was no-one at the meetings. 31 advocacy meetings were also held, though it is not clear, who with or what for.
    * 91 peer educators trained & 14 workshops. It is not clear that these are in addition to the original training provided for the 91 peer educators, or whether it took 14 workshops to train 91 peer educators
    * 10 workshops for non-health professionals & non-peer educators
    * 2 large companies with HIV/AIDS policies
    * 38 people referred for HIV tests

Let’s assume again that the distribution of condoms and pamphlets and referrals for tests are costless or incur minimal costs. As the MBCA did not provide the number of people it reached with its education sessions and workshops, the cost per person can not be calculated. So we are left with the number of health education sessions, mass awareness session and number of peer educators trained. So the MBCA held a 120 health education sessions, 10 mass awareness sessions and trained 91 peer educators. If we assume that the cost for each of training is the same, then it cost USD1,235 to hold each education session and train each peer educators. This is about 20 percent of the expenditure of MRCS to produce education sessions, but still an absurd amount of resources, when resources are so scarce. The expenditure is also considerably higher than that of MHAA to produce health education sessions. Surely, health education sessions can be held in Burma at the fraction of the cost.

Pyi Gyi Khin – USD155,00

    * 142,599 condoms and 335,000 pamphlets were distributed
    * 33 advocacy meetings were held, though there is no information for what or with whom these were held. Another 83 multi-sectoral meetings were also held, with no more information provided.
    * 30 peer educators trained and 5 workshops for peer educators. As with the MBCA it is not clear that it took 5 workshops to train the 30 peer educators or these were in additional to the original education provided.
    * 689 education sessions held, where 462 were reached, with 29 receiving counseling. As only 462 people were reached, some of these education sessions must have been without participants.
    * 1 need assessment and 1 base-line study were conducted. Again these are not publicly available.
    * 418 people were given HIV tests
    * 400 received with AIDS received home based care
    * 1,012 with HIVS were treated for opportunistic infections

If we assume that HIV tests, home based care and treating AIDS patients for opportunistic infections cost the same at USD10 (as provided by MMA), then PGK spend around USD18,300 on caring for HIV/AIDs patients. This would have left around USD136,700 for other activities. With 30 peer educators trained and 689 education sessions held this implies that each cost around USD190, less than most of the other Burmese NGOs. However, since only 462 people were reached the cost of reaching each person would have been around USD300. Again this is a ridiculous allocation of resources when per capita expenditure on health is only USD4.

Comparison of the Costs of Outreach of Burmese NGOS

    

Average No. People at each Health Education Session
    

Cost of each Education Session
    

Cost/Head Education

MHAA
    

541
    

$178
    

$2.50

MRCS
    

3,500
    

$5,800
    

$11

MANA
    

15
    

$?
    

?$320

MBCA
    

?
    

?$1,235
    

$?

PGK
    

1.5
    

?$190
    

?$300

These estimates are obviously rough, but they do indicate that resources were allocated in an extremely cost inefficient manner. The estimates also indicate that little attention has been given to the cost effectiveness of the different organisations. The large expenditure on health education, with no indication of its impact, also indicates an extremely inefficient allocation of scarce health resources.

Hopefully, the international NGOs, the United Nations agencies and Burma’s ‘government’ made better use of monies provided by FHAM. We will examine their use of scarce health resources shortly.

http://burmaeconomicwatch.blogspot.com/2008/07/not-much-bang-for-aid-buck.html