Difference between revisions of "User talk:Sebastian"

From Timelines
Jump to: navigation, search
 
(3 intermediate revisions by the same user not shown)
Line 1: Line 1:
Jump to Navigation
+
== Feature testing ==
Home
+
{{CSS image crop
GIVING EFFECTIVELY HOW WE WORK TOP CHARITIES RESEARCH OUR MISTAKES ABOUT UPDATES
+
|Image = Southern crested caracara (Caracara plancus) head adult.JPG
DONATE
+
|bSize = 200
HOME » TOP CHARITIES » SCHISTOSOMIASIS CONTROL INITIATIVE » ALL CONTENT » 2015 REVIEW, UPDATED APRIL 2016
+
|cWidth = 100
Schistosomiasis Control Initiative (SCI) — 2015 Review, Updated April 2016
+
|cHeight = 100
Facebook
+
|oTop = 180
Twitter
+
|oLeft = 60
>Print
+
|Location = center
>Email
+
|Description = testing crop feature
We have published a more recent review of this organization. See our most recent report on SCI.
+
}}
  
The Schistosomiasis Control Initiative (SCI) is one of our top-rated charities and an organization that we feel offers donors an outstanding opportunity to accomplish good with their donations.
+
{{CSS image crop
 +
|Image = Southern crested caracara (Caracara plancus) head adult.JPG
 +
|bSize = 400
 +
|cWidth = 100
 +
|cHeight = 100
 +
|oTop = 180
 +
|oLeft = 60
 +
|Location = center
 +
|Description = testing crop feature
 +
}}
  
More information: What is our evaluation process?
+
{{CSS image crop
Published: November 2015; Updated: April 2016
+
|Image = Southern crested caracara (Caracara plancus) head adult.JPG
 +
|bSize = 800
 +
|cWidth = 100
 +
|cHeight = 100
 +
|oTop = 180
 +
|oLeft = 60
 +
|Location = center
 +
|Description = testing crop feature
 +
}}
  
DONATE
 
Summary
 
What do they do? SCI works with governments in sub-Saharan Africa to create or scale up programs that treat schistosomiasis and soil-transmitted helminths (STHs) ("deworming"). SCI's role has primarily been to identify country recipients, provide funding to governments for government-implemented programs, provide advisory support, and conduct research on the process and outcomes of the programs. Despite SCI sharing a number of spending reports with us, we do not feel we have a detailed and fully accurate picture of how SCI and the governments it supports have spent funds in the past. (More)
 
  
Does it work? We believe that there is relatively strong evidence for the positive impact of deworming. SCI has conducted studies in about half of the countries it works in (including the countries with the largest programs) to determine whether its programs have reached a large proportion of children targeted. These studies have generally found moderately positive results, but have some methodological limitations. (More)
+
== Visual data source repository ==
  
What do you get for your dollar? We estimate that children are dewormed for a total of around $1.26 per child. This figure relies on several difficult-to-estimate inputs including how to account for (a) donated drugs and (b) in-kind contributions from governments with which SCI works. Excluding drugs and government contributions, we estimate that SCI's cost per treatment is $0.53. The number of lives significantly improved is a function of a number of difficult-to-estimate factors, which we discuss in detail in a separate report. (More)
+
* [https://sensoneo.com/sensoneo-global-waste-index-2019/ Global Waste Index 2019]
 
 
Is there room for more funding? SCI aims to raise a total of $9.5 million for its April 2016 to March 2017 budget year, of which it is on pace to raise $1.7 million absent a GiveWell recommendation and up to $2 million is likely covered by another funder. It aims to raise about $13 million in each of the following two years. If SCI raises more than its goal, it may use funding to treat adults. We have some uncertainty about SCI's room for more funding given past difficulties with predicting how SCI would use additional funds. (More)
 
 
 
SCI is recommended because of its:
 
 
 
Focus on a program with a strong track record and excellent cost-effectiveness. (More)
 
Track record – SCI has repeatedly demonstrated success at starting and expanding national deworming programs.
 
Room for more funding – we believe SCI will be able to use additional funds to deliver additional treatments. (More)
 
Major unresolved issues include:
 
 
 
We have a limited understanding of SCI's work at a detailed level because we have limited and perhaps unreliable data on how SCI has spent funds in the past.
 
The evidence we have seen on SCI's track record of reaching those it has targeted for treatment is fairly limited. We have seen results from about half of the countries SCI has worked in and, for those countries from which we have seen results, we have generally seen one year of results, though SCI has worked in the country for several years. We have some questions about the results we have seen because of methodological limitations of the studies and past challenges communicating with SCI about its monitoring (see blog posts from 2013 and 2014).
 
Table of Contents
 
Summary
 
Our review process
 
What do they do?
 
Major funding sources for SCI's work
 
Large grants
 
Unrestricted funding
 
SCI’s role in mass drug administration programs
 
Côte d'Ivoire
 
Ethiopia
 
Mozambique
 
Uganda
 
Breakdown of SCI’s spending
 
Limitations to the spending information we've seen
 
Spending breakdown
 
DFID grant spending breakdown
 
Other projects
 
Does it work?
 
Independent evidence of program effectiveness
 
Prevalence and intensity studies
 
Which prevalence and intensity studies provide evidence of SCI's impact?
 
Niger, Burundi, and Malawi prevalence and intensity studies
 
Results from Yemen
 
Coverage surveys
 
Are SCI's monitoring results representative of its work overall?
 
Additional academic evidence
 
Possible negative and offsetting impact
 
What do you get for your dollar?
 
Our approach
 
Our analysis
 
Shortcomings of our analysis
 
Baseline infection status
 
Is there room for more funding?
 
Sources of uncertainty
 
Available unrestricted funding
 
Uses of additional funding
 
GiveWell's prioritization of SCI's funding gaps
 
Global need for treatment
 
SCI as an organization
 
Sources
 
Our review process
 
We began reviewing SCI in 2009. Our review has consisted of:
 
 
 
Reviewing published studies on SCI's programs.
 
Extensive communications with SCI Director Alan Fenwick and Deputy Director Wendy Harrison to discuss SCI's methods and funding needs.
 
Requesting and reviewing SCI internal financial, organizational, and monitoring and evaluation documents.
 
Visiting a national schistosomiasis control program meeting and demonstration mass drug administration in Malawi in October 2011 (notes and photos from this visit).
 
Following SCI's progress and plans for funds raised as a result of GiveWell's recommendation (see our updates on SCI's progress).
 
Meetings with eight SCI staff members at SCI's London headquarters in October 2014, including leadership, program managers, and monitoring and evaluation and finance staff.
 
All content on the Schistosomiasis Control Initiative, including past reviews, updates, blog posts and conversation notes, is available here.
 
 
 
What do they do?
 
SCI works with governments in sub-Saharan Africa to create or scale up mass drug administration programs (MDAs) for neglected tropical diseases (NTDs), particularly schistosomiasis and soil-transmitted helminths (STHs), in school-aged children and other groups determined to be at high risk.1 SCI's role has primarily been to solicit grants from large funders, identify country recipients, provide funding to governments for government-implemented programs, provide advisory support, and conduct research on the process and outcomes of the programs.
 
 
 
We feel that we have a limited understanding of SCI's work at a detailed level. SCI's model involves both (a) employing staff for program management and technical assistance (capacity building) and (b) funding governments to carry out mapping and treatment programs. We have a reasonable understanding of the role SCI's staff plays, but have struggled to get a clear and complete picture of how SCI and the governments it supports have spent funds in the past. Clear spending data would allow us to have a better understanding of SCI's model, have more confidence in its financial management (including whether funds may be diverted for uses that don't support the program), and compare future plans to past spending to understand whether future funds are likely to achieve similar outcomes.
 
 
 
Major funding sources for SCI's work
 
Large grants
 
SCI's work has been driven by a number of large grants, each with somewhat different program designs and geographic coverage. SCI's major grants:
 
 
 
Initial Gates Foundation grant: SCI was founded in 2002 through a $32 million grant from the Bill and Melinda Gates Foundation.2 This grant was used to create national treatment programs for schistosomiasis and soil-transmitted helminths (STHs) in six countries.3
 
Grants for integrated NTD control: In 2006, SCI received large grants from USAID and the Gates Foundation to support integrated NTD programs in eight countries for five years to treat lymphatic filariasis, onchocerciasis, and trachoma, in addition to schistosomiasis and STHs.4 It received a grant in 2007 to expand its work to Rwanda and Burundi.5 All of these grants were completed in 2011.6
 
DFID grants: In 2010, SCI received £10.5 million7 (plus separate funding for drugs) from the UK's Department for International Development (DFID) for treating schistosomiasis and soil-transmitted helminths8 in eight countries over five years.9 Other NTDs are not covered by the grant, though DFID also provided funding to the Filarial Programmes Support Unit (FPSU; formerly the Centre for Neglected Tropical Diseases) to integrate treatment for lymphatic filariasis with SCI-funded schistosomiasis and STH programs in six countries.10 In 2014, DFID awarded SCI an additional £16.6 million over four and a half years (June 2014 to December 2018) to extend the program and expand it to an additional two countries.11
 
Unrestricted funding
 
Prior to 2011, unrestricted funds accounted for a very small portion of SCI's total funding.12 SCI told us that this funding was primarily used to fund treatments in regions of Côte d'Ivoire and Mozambique.13
 
 
 
In part due to GiveWell's recommendation, since November 2011, SCI has received significantly more unrestricted funds: GiveWell has tracked about $11.4 million in donations to SCI due to our research.14 We estimate that at least 70% of the unrestricted funds that SCI raised in April 2013 to March 2015 (the period for which we've seen data) were due to GiveWell's recommendation.15
 
 
 
Unrestricted funds now make up a large portion of SCI's revenue. For April 2013 to March 2015, SCI reports that 61% of its revenue was unrestricted (though we believe that some non-DFID restricted funds are classified as unrestricted in this report, see below).16
 
 
 
SCI’s role in mass drug administration programs
 
SCI's role in mass drug administrations (MDAs) in general is to:17
 
 
 
Advocate for the benefits of mass drug distributions to government officials.
 
Assist with planning and budgeting.
 
Deliver funding and drugs to governments.
 
Provide financial management and technical support.
 
Develop procedures for monitoring and evaluation, analyze data, and write reports.
 
In 2014, we spoke with four of SCI's program managers to better understand SCI's role in four countries. These conversations were mostly consistent with our general understanding of SCI's work. We selected Côte d'Ivoire, Ethiopia, and Mozambique because SCI has spent significant unrestricted funds, the type of funding GiveWell has recommended, in these countries (more below). SCI selected Uganda as the fourth case study. Summaries follow.
 
 
 
In addition, in 2015, SCI shared some details about its work in Sudan, where it recently began supporting a program, and in Nigeria, where it is considering supporting a program in the future. We have not yet written up summaries of this work (documents SCI shared listed in footnote).18
 
 
 
Côte d'Ivoire
 
History: There was no large-scale schistosomiasis treatment program in the country prior to SCI's involvement.19 SCI began working in Côte d'Ivoire in 2010, but mapping and treatments did not begin until 2012 due to political turmoil. It has funded mapping and treatment for all qualifying districts.20 SCI expected to fund about 4.9 million treatments during its 2014-15 budget year;21 it reported delivering 3.1 million treatments in that period.22 SCI previously told us that it spent a small amount of unrestricted funding in the country prior to 2011; we do not know what this funding was for.23
 
Current plans (as of October 2015): SCI plans to deliver 0.4 million treatments in its 2015-16 budget year, 2.7 million treatments in its 2016-17 budget year, and 2.3 million treatments in each of the following two years. These plans may be dependent on SCI's fundraising.24
 
Funding: SCI has used DFID funding, a grant from the company Vitol, and unrestricted funding in Côte d'Ivoire.25 In the 2014-15 budget year, SCI spent about $250,000 in restricted funding and $820,000 in unrestricted funding in the country, a significant increase over 2013-14, when SCI spent $580,000 in restricted funding and $20,000 in unrestricted funding.26
 
Impact of unrestricted funds: SCI believes that availability of unrestricted funds has allowed the program to scale up faster than it otherwise could have.27
 
Partners: In the early stages of the program, SCI primarily worked with the Ministry of Health and, for mapping, the Université Félix Houphouët-Boigny. Starting in 2014, SCI began working with other partners, including Sightsavers, the END Fund, and Helen Keller International on a more integrated NTD program.28
 
Role of SCI staff: SCI's Program Manager for Côte d'Ivoire described her role (as of October 2014) as providing technical expertise, helping create budgets, monitoring training and MDAs, responding to problems (for example, surveying health workers to understand low coverage of non-enrolled children, and advising on how to proceed with the program during a security threat), refining the protocol for the coverage survey, and coordinating with partners for integrated MDAs.29 From November 2013 through November 2014, she traveled to Côte d'Ivoire five times and spent almost four months there in total to assist with and monitor mapping, sentinel site and coverage survey data collection, three MDAs, and budgeting.30 She also noted the role of other SCI staff in the program: the finance team checks receipts against expense reports,31 and the biostatistician analyzes monitoring data and advises government staff on data issues.32
 
Ethiopia
 
History: There was no sustained, large-scale schistosomiasis treatment program in the country prior to SCI's involvement, only a one-off, sub-national treatment.33 SCI began conversations with the government of Ethiopia about starting a national schistosomiasis treatment program in 2012. In 2013, it funded planning for the program. It funded mapping between November 2013 and April 2014, and a first round of treatment in 2013.34 In April 2015, Ethiopia treated a reported 2.9 million children for schistosomiasis and STH. SCI planned to conduct a coverage survey following this round of treatment and collected baseline data on infection rates from 22 sentinel schools prior to the treatment round (more on these data collection methods below).35 We have not yet seen results.
 
Current plans (as of October 2015): SCI reports that it had delivered 2.9 million treatments between April and September 201536 and plans to deliver a total of 10.6 million treatments in its 2015-16 budget year. It also plans to deliver 14.4 million treatments in its 2016-17 budget year and about 17 million treatments in each of the following two years. These plans may be dependent on SCI's fundraising.37
 
Funding: SCI used about $1.7 million in unrestricted funds for planning, mapping, and the first two rounds of treatment.38 The END Fund and DFID have now allocated restricted funds for treatment in the country.39
 
Impact of unrestricted funds: SCI believes that availability of mapping data significantly improved the chances of securing funding from the END Fund and DFID, and that, at a minimum, the availability of unrestricted funds accelerated the program by a few years.40 Ethiopia's Neglected Tropical Diseases Program Manager told us that the government had not dedicated funding to schistosomiasis and that without SCI's involvement the program would likely not have gone forward. According to him, Ethiopia was not aware of any other potential partners for this work when it began working with SCI.41
 
Partners: SCI is funding the Federal Ministry of Health (FMOH) to carry out treatments. It partnered with the Ethiopian Public Health Institute, the technical arm of the FMOH, and the Partnership for Child Development (PCD) on mapping. PCD funded mapping in part of the country and delivered treatments in a pilot program. Deworm the World Initiative is providing technical assistance to the government.42
 
Role of SCI staff: SCI's Program Manager for Ethiopia described his role (as of October 2014) as assisting with initial program set up and funder agreements, drafting detailed plans for the next 12 months and general plans for the next 5 years, and providing technical assistance for mapping and some guidance on MDA implementation. He lives in the country43 and works in the same office as the government NTD team.44
 
Mozambique
 
History: There was no large-scale schistosomiasis treatment program in the country prior to SCI/FPSU's involvement. SCI had previously funded a small-scale treatment program in Mozambique with unrestricted funding.45 SCI's grant from DFID, which started in 2010, includes funding for a sub-grant to FPSU to run schistosomiasis treatment programs in three countries, one of which is Mozambique.46 SCI and FPSU decided on this arrangement because SCI believed it would increase the chances of DFID awarding the grant.47 SCI reports that it delivered 14.3 million treatments between April 2011 and March 2015.
 
Current plans (as of October 2015): SCI reports that it had delivered 4.4 million treatments between April and September 201548 and plans to deliver a total of 5.4 million treatments in its 2015-16 budget year. It also plans to deliver 9.6 million treatments in its 2016-17 budget year and 16.8 million treatments in each of the following two years. These plans may be dependent on SCI's fundraising.49
 
Funding: SCI has spent both restricted funding from DFID and unrestricted funding in Mozambique. The data we have seen on SCI's spending in Mozambique is incomplete (see below), but our impression is that Mozambique has been one of SCI's largest programs.50 It has funding from the SCORE project for research on "gaining and sustaining control of schistosomiasis" in one province.51
 
Impact of unrestricted funds: SCI told us that unrestricted funds have helped the program scale, but perhaps not as much as it might appear, since SCI has shifted DFID funding to other countries with the expectation of using unrestricted funds in Mozamique.52
 
Partners: The Ministry of Health implements the program. SCI/FPSU also collaborate with RTI International, which is working on trachoma in Mozambique and has staff in-country. RTI helps to keep SCI/FPSU informed about program progress.53
 
Role of SCI staff: FPSU manages the program day-to-day. SCI has provided some assistance with monitoring and evaluation. We do not have more detail on SCI/FPSU role in the country.54
 
Uganda
 
History: SCI has worked in Uganda since 2003 and has received funding for this work from each of its large grants: Gates Foundation, USAID, and DFID. There was a gap in SCI's work between the USAID and DFID funding.55 SCI reported that it delivered 23,000 treatments in April 2014 to March 2015.56 It had planned to fund 400,000 treatments this period;57 we haven't discussed with SCI why significantly fewer treatments were delivered than planned.
 
Current plans (as of October 2015): SCI plans to deliver 1 million treatments in its 2015-16 budget year, 1.8 million treatments in its 2016-17 budget year, 1.4 million in 2017-18 and 1.8 million in 2018-19. These plans may be dependent on SCI's fundraising.58
 
Funding: In recent years, SCI has primarily used DFID funding in Uganda (about $176,000 in the fiscal year covering 2013-14 and $30,000 in 2014-15).59 SCI allocated a small amount of unrestricted funding to Uganda to increase the number of sentinel sites for operational research purposes.60 SCI also spent about $30,000 in unrestricted funds in Uganda in 2014-15;61 we don't know what these funds were used for.
 
Partners: RTI International, funded by USAID, works in Uganda on an integrated mass NTD treatment program. SCI provides schistosomiasis treatment in those areas where RTI does not.62
 
Role of SCI staff: SCI told us that, due to strong in-country capacity, SCI provides only limited technical assistance for MDA. Instead, SCI's technical assistance in the country is focused on operational research on how best to move toward elimination of schistosomiasis.63 SCI's program manager for Uganda describes her role as (as of October 2014) assisting with budgets and plans, talking to the government regularly (largely through email) leading up to an MDA to make sure all the components are prepared, traveling to the country to oversee collection of prevalence and intensity data prior to an MDA, following up with the government after an MDA to get the data needed for reporting to DFID, and participating in activity reviews after each stage of the program (for example, after an MDA and after monitoring and evaluation activities).64
 
Breakdown of SCI’s spending
 
We have seen fairly limited information on how SCI has spent funds in the past. SCI has only recently compiled a comprehensive report of all of its expenditures. We know of several limitations to this report and are not confident that the report is complete and fully accurate.
 
 
 
Limitations to the spending information we've seen
 
There are several structural reasons to be cautious about interpreting the data:
 
 
 
SCI is housed within Imperial College London. Because it is not a standalone charity, it does not publish annual financial statements or undergo annual organization-wide audits (as U.S. charities are required to do).
 
It uses an accounting system created by Imperial College which seems ill-fitted to its needs and which breaks up its finances into many different "accounts" rather than giving an overall view of the organization's financial position.65
 
SCI has noted that its financial systems need improvement and has been working to expand its accounting team and improve its systems.66 SCI's current Finance and Operations Manager joined the organization in mid-2015 after the previous Finance and Operations Manager left the organization after about six months.67
 
Based on our experiences with SCI's financial reports, we believe that they are prone to containing errors. We detail the errors we have learned about in this footnote.68
 
In addition, the information we have seen is limited in scope:
 
 
 
We have only seen comprehensive data for two budget years: April 2013-March 2014 ("2013/14") and April 2014-March 2015 ("2014/15").
 
We have not seen recent information on how much funding SCI currently holds or on what portion of the funds that it holds are committed and uncommitted.
 
We have not seen complete or recent information on how SCI has spent funds within country programs. The data we have seen is only broken down by country.
 
The data we have seen only tracks funds to the point of being transferred to accounts within the countries that SCI works in. Therefore, some of the recorded expenditures do not represent actual spending. We don't know how long funds typically sit in in-country accounts before being spent.
 
Spending breakdown
 
Notes about this data:
 
 
 
For Zambia, Liberia, and Mozambique, SCI transfers funds to FPSU, rather than directly to the country. SCI told us that it has not yet been able to break out some of that funding and so some of those payments are included in "central programme expenditure" rather than the total for the country program, particularly in the 2013/14 data. SCI believes this is the main reason why it appears that central expenditure falls significantly in 2014/15.69
 
Some restricted grants other than funds from DFID (such as funds from the END Fund or large donors who have restricted their funds to a specific country) are included in the unrestricted revenue total, while the expenditures from these grants may be included in the totals for restricted expenditure.70 We do not know how much funding these grants account for.
 
For full details see SCI financial statement 2013/14 and 2014/15 (revised October 2015).
 
 
 
2013/14 expenditures by country (in millions USD)
 
 
 
Restricted Unrestricted Total % of total
 
Central expenditure $1.27 $0.65 $1.92 28.5%
 
Ethiopia - $1.15 $1.15 17.1%
 
Burundi $0.49 $0.20 $0.69 10.3%
 
Tanzania $0.65 - $0.65 9.7%
 
Côte d'Ivoire $0.58 $0.02 $0.60 8.9%
 
Uganda $0.28 $0.10 $0.38 5.6%
 
Malawi $0.27 $0.10 $0.37 5.5%
 
Niger $0.30 - $0.30 4.5%
 
Zanzibar $0.27 $0.01 $0.28 4.2%
 
Yemen $0.13 - $0.13 1.9%
 
Rwanda $0.08 - $0.08 1.2%
 
Zimbabwe - $0.07 $0.07 1.0%
 
Mauritania - $0.06 $0.06 0.9%
 
Mozambique $0.05 - $0.05 0.7%
 
Total $4.37 $2.36 $6.73 100.0%
 
2014/15 expenditures by country (in millions USD)
 
 
 
Restricted Unrestricted Total % of total
 
Central expenditure $0.78 $0.28 $1.06 15.1%
 
Mozambique $0.22 $1.03 $1.25 17.8%
 
Ethiopia $0.64 $0.58 $1.22 17.3%
 
Côte d'Ivoire $0.25 $0.82 $1.07 15.2%
 
Malawi $0.52 - $0.52 7.4%
 
Sudan - $0.37 $0.37 5.3%
 
Rwanda $0.34 - $0.34 4.8%
 
Democratic Republic of the Congo $0.34 - $0.34 4.8%
 
Zanzibar $0.19 $0.06 $0.25 3.6%
 
Burundi $0.15 $0.05 $0.20 2.8%
 
Yemen $0.12 $0.01 $0.13 1.8%
 
Uganda $0.05 $0.05 $0.10 1.4%
 
Madagascar $0.09 - $0.09 1.3%
 
Liberia - $0.07 $0.07 1.0%
 
Tanzania - $0.02 $0.02 0.3%
 
Niger $0.01 - $0.01 0.1%
 
Total $3.70 $3.34 $7.04 100.0%
 
DFID grant spending breakdown
 
We have not seen comprehensive data on how funds have been spent within country programs, but we have seen some data from seven countries: Niger, Tanzania, Uganda, Malawi, Mozambique, Zambia, and Liberia over two to three years, mid-2011 to 2013 or 2014 (actual periods vary by country).71 SCI told us that these are the only countries for which it has completed this type of spending analysis.72 We do not know why these particular countries were selected, but note that they are countries that are either funded by DFID and have received little or no unrestricted funds73 or country programs that are managed by FPSU.74 SCI has a long history of involvement in Niger, Tanzania, and Uganda, so we would guess that these programs are at a more advanced stage of development than many of the countries where SCI has begun working more recently.
 
 
 
In-country SCI spending in seven countries75
 
 
 
Budget item % of spending Description Range across countries
 
Mass drug administration 40% Largely unspecified transfers to districts, individuals, or other organizations; per diem payments for health workers and supervisors; production of dose poles and treatment registers. 9% to 76%
 
Mapping (surveys to determine disease prevalence in each district) 19% Per diem payments; fuel and transport; equipment; utilities 0% to 50%
 
Monitoring and evaluation (baseline and follow up surveys of prevalence and intensity; coverage surveys) 9% Per diem payments for attendance at review meeting; per diem payments for surveyors; fuel and transport; equipment 1% to 22%
 
Training 8% Limited details 0% to 29%
 
Central management (country-level) 8% Salaries for government staff; fuel and transport; utilities; office supplies and equipment; taxes; bank charges 0% to 47%
 
Advocacy and social mobilization (building community awareness of the program) 8% Printing; radio and TV broadcasts; unspecified per diem payments 1% to 30%
 
Supervision 3% Per diem payments; fuel and transport 0% to 7%
 
Strategic planning 2% Per diem payments; meeting venue and refreshments; fuel and transport; supplies 0% to 8%
 
Drug supply chain 2% Custom fees; transport; per diem payments; lab materials; storage 0% to 4%
 
Other projects
 
In addition, SCI has received some smaller grants for a variety of projects, including:
 
 
 
Research. SCI has received a number of smaller grants to carry out research related to NTD control.76
 
Other NTD-related activities. SCI has also used funding from individuals for surgeries for hydrocele (a symptom of lymphatic filariasis) in Niger, and health education and water and sanitation programs in Burundi.77
 
Does it work?
 
SCI's mass drug administration programs are focused on delivering treatments that have been independently studied in rigorous trials and found to be effective.
 
 
 
SCI has conducted studies in some of the countries it has worked in to determine whether its programs have reached a large proportion of children targeted. We have now seen some recent monitoring results from about half of the countries in which SCI works, including the five countries where SCI has delivered the largest number of treatments. For those countries from which we have seen monitoring results, we have generally seen one year of results, though SCI has worked in the country for several years. We have some questions about the results we have seen because of methodological limitations of the studies.
 
 
 
To determine SCI's track record at executing programs, we have considered:
 
 
 
Studies of changes in prevalence and intensity of infection over time in three of the countries SCI has worked in. The studies show substantial improvements following SCI treatment programs. These studies have a number of limitations and represent a small portion of SCI's past work.
 
Treatment coverage surveys from six of the countries SCI has worked in, including many of the countries where SCI's work has been focused in the past five years. These studies track what percentage of individuals who were targeted for treatment actually received treatment. Overall, the studies found moderate rates of coverage. We note some limitations of these studies below.
 
Other published papers that might reflect the treatment coverage achieved by SCI's programs by directly measuring deworming drug uptake or by measuring worm prevalence in countries where SCI has worked. The evidence is mixed, but makes a weak case for low coverage in the areas studied.
 
Details follow.
 
 
 
Independent evidence of program effectiveness
 
SCI's primary program is mass combination deworming, which we discuss extensively on another page. There is a very strong case that mass deworming is effective in reducing infections. The evidence on the connection to positive quality-of-life impacts is less clear, but there is a fairly strong possibility that deworming is highly beneficial.
 
 
 
Prevalence and intensity studies
 
SCI has conducted studies to track changes in schistosomiasis and STH prevalence and intensity rates following SCI-supported treatment programs. In each of these studies, SCI tracked infection rates at the same schools ("sentinel sites") each year. In general, prevalence and intensity of the parasites decreased over time in each of the countries studied. We note several methodological limitations of these studies below.
 
 
 
Which prevalence and intensity studies provide evidence of SCI's impact?
 
Below, we discuss results from studies of schistosomiasis and STH prevalence and intensity from three countries: Niger (2004-2006),78 Burundi (2007-2010),79 and Malawi (2012-2015).80
 
 
 
SCI also shared studies from Uganda81 and Burkina Faso,82 and we included results from these studies in our previous reviews of SCI. We learned in 2013 (and in follow up work in 2014) that participants in the studies in Uganda and Burkina Faso received separate, more intensive treatment than other children in those countries (discussed in blog posts in 2013 and 2014). Therefore, we believe that the results from Uganda and Burkina Faso do not reflect the quality of the national programs which were supported by SCI.
 
 
 
It is our understanding that, in the Niger, Burundi, and Malawi studies, study participants received treatment in the same manner as other children in the country, and thus that those studies reflect the performance of the national MDAs. However, we are not highly confident in this conclusion because we had difficulties communicating clearly with SCI about the methodology of these studies (discussed in more detail in the blog posts linked above).
 
 
 
SCI told us that it is currently conducting similar studies in Mozambique, Ethiopia, DRC, Liberia, Côte d'Ivoire, Zambia, Tanzania, Uganda, and Niger (for more recent years). We have not yet seen results from any of these studies.83
 
 
 
Niger, Burundi, and Malawi prevalence and intensity studies
 
As discussed above, SCI has conducted studies to track changes in schistosomiasis and STH prevalence and intensity rates following SCI-supported treatment programs in Niger, Burundi, and Malawi. In each study, with one exception, the same individuals were examined before the initial round of treatment and before each subsequent round of treatment.84 In the 2015 Malawi study, SCI switched to a cross-sectional sample, where random children from the same schools were surveyed, rather than the same individuals.85 To partially account for this change, the data from Malawi presented below is for the 6-8 year old age group only. There is no control group for these studies.86
 
 
 
In general, prevalence and intensity for the two main types of schistosomiasis, S. haemotobium and S. mansoni, and for hookworm (more on the other two STHs below), decreased over time in each of the countries studied. Though it is possible that other factors besides the treatment program caused these changes (such as improved sanitation infrastructure), the pattern of decline in a short period following treatment strongly suggests that treatment caused or contributed to the declines.
 
 
 
Changes in worm prevalence and intensity87
 
 
 
Schistosoma haematobium Schistosoma mansoni Hookworm
 
Country Changes in prevalence Changes in intensity Changes in prevalence Changes in intensity Changes in prevalence Changes in intensity
 
Niger 75.4% at baseline to 38% at one year88 21.8% prevalence of heavy-intensity infections at baseline to 4.6% at one year89 Very low prevalence at baseline90 N/A Low prevalence at baseline91 N/A
 
Burundi (pilot) Not reported (SCI reports very low baseline prevalence92) N/A 12.7% at baseline to 1.7% at four years93 20 epg94 at baseline to 1 epg at three years95 17.8% at baseline to 2.7% at four years96 16 epg at baseline to 24 epg at three years97
 
Burundi (other schools) Not reported (SCI reports very low baseline prevalence98) N/A 6.2% at baseline to 0.7% at three years99 15 epg at baseline to 8 epg at one year100 15.1% at baseline to 5.4% at three years101 15 epg at baseline to 8 epg at one year102
 
Malawi 9% at baseline, 6% at one year, 4% at two years103 Low rates of heavy infection (note: different results given in first follow up report)104 Low rates at baseline and follow up105 Very low rates of heavy infection at baseline and follow up106 No hookworm found at baseline, 1% at one year, 2% at two years107 No participants heavily infected at baseline or follow up108
 
For the other two prominent soil-transmitted helminths, ascaris and trichuris, prevalence was low in the Niger and Malawi studies.109 In Burundi, prevalence of ascaris and trichuris decreased somewhat (though in a few cases there were temporary increases). Data from Burundi are given in the footnote.110
 
 
 
Some of the studies also report results for other indicators of disease such as anemia. We omit discussion of these other indicators because they are more likely to be influenced by external factors than are prevalence and intensity (see our previous review of SCI for discussion of these indicators).
 
 
 
Limitations of the prevalence and intensity study data include:
 
 
 
Monitoring of selected locations. It appears that, in the Niger and Burundi pilot studies, locations included in the study were selectively chosen rather than selected to be a representative sample of treated areas.111
 
Low follow-up rates. Follow up rates were low in two of the three countries (at the first year follow-up, 89% in Niger, 33%-50% in the pilot survey and 53%-80% in the other schools survey in Burundi, and 52% in the first follow up in Malawi).112 To be included in follow up surveys, children must be present in school when the surveys are done.113 If those who are present in school are less likely to be infected than those who are not present, this could lead to overstating the impact of the program. The connection between infection status and absenteeism could be a direct relationship (infection could cause absenteeism) or an indirect one (a third factor, such as poverty, could cause both higher levels of infection – perhaps through poor sanitation infrastructure – and absenteeism). Because the second follow up in Malawi was done as a cross-sectional study among children present in school on the day of the study and we present results for children ages 6-8 only, absenteeism and dropout are less likely to bias the results than in the panel studies.
 
Results from Yemen
 
In addition, we have two types of results from Yemen:
 
 
 
Partial sentinel site data: A report SCI shared with us mentions that initial analysis of sentinel site data from July 2014 in Yemen found that prevalence of schistosomiasis decreased substantially.114 These results only include 2,000 of the 8,000 individuals who were surveyed at baseline and the result noted that "a full round of impact evaluation" would be completed in September 2014.115 We have not seen more details about these initial results or any results from the full round.
 
Remapping survey: This study compared the number of districts at high-risk, moderate-risk, low-risk for, and uninfected with schistosomiasis at "baseline" (data collected between 2004 and 2010) and in 2014.116 It found large improvements after 2-3 rounds of treatment.117 It is not clear to us whether baseline and follow up results are directly comparable. Baseline data was collected over several years and details of the methodology used at baseline are not given.
 
Coverage surveys
 
SCI has conducted, or worked with partners to conduct, surveys in Côte d'Ivoire (in 2014), Malawi (2012 and 2014), Uganda (2014), Mozambique (2015), Zanzibar (2015), and Zambia (2015) to determine what proportion of people targeted for mass drug administration received treatment.
 
 
 
In each of the surveys, surveyors visited a sample of households and asked children, or their parents on their behalf, whether they received treatment in the most recent MDA. Other survey questions, such as age, gender, where the respondent received the treatment, and why they did not take the drug(s), were often included as well. The methodology used differed somewhat across countries. We have summarized the details of the methodologies used in the studies in this spreadsheet ("Methods" sheet). Key differences in the methods used across countries include:118
 
 
 
Selection of geographic target area: All of the surveys were limited to specific geographic areas (such as districts). In Uganda, Zambia, and Côte d'Ivoire, these were selected randomly or nearly randomly. In Malawi (both 2012 and 2014 surveys), the districts were purposefully selected and not intended to be nationally representative. The selection procedure for Zanzibar was not given in the survey report.119 We are aware of one case, in Mozambique, where a selected village was replaced by another because the surveyors could not locate it; this issue was not mentioned in the survey report.120
 
Independence from the government: In Mozambique, the survey was carried out by government health staff, who may have had an incentive to bias the results. SCI told us, "[M]ost of the interviews in one district were done by the other district officers with no connection with the district."121 The reports on the Malawi 2014 and Côte d'Ivoire studies note that the surveyors were independent of the government. SCI told us that university students or staff conducted the other surveys (Malawi 2012, Uganda, Zanzibar, and Zambia).122
 
Length of time between MDA and survey: This varied between one and six months. Intuitively speaking, the more time that passes, the less likely children are to remember accurately and the more likely they are to confuse past MDAs (we discuss one case below, from Mozambique, where there may have been confusion been MDAs). Mozambique had the shortest interval at 1-2 months and Zambia had the longest at 5-6 months. Other surveys were generally carried out 2-4 months after the MDA.
 
Whether parents or children were interviewed: In Mozambique, parents were interviewed about whether their children took the drugs. In both Côte d'Ivoire and Malawi (both 2012 and 2014 surveys), if children in a household were not available then their parents were interviewed about whether the children had received deworming drugs. SCI made different choices about whether to include these responses in the results, which slightly inflated the results overall.123 SCI told us that parents were not asked to answer on behalf of their children in Uganda or Zanzibar.124 The report on the Zambia survey does not mention adults answering for children.
 
In addition to limitations in specific surveys, we note some cross-cutting limitations to the methods used:
 
 
 
Use of verification methods: In Côte d'Ivoire, Malawi (2014), Uganda, Zanzibar, and Zambia participants were asked a number of questions about the treatment program, such as whether they recognized pills or dose poles presented by the interviewers, what they thought of the pills (praziquantel is very large and tastes bitter) and how many pills they took.125 Answers to these questions were not recorded in Côte d'Ivoire,126 and we do not know if they were recorded in the other surveys. We believe that the answers to questions such as these would provide additional evidence about the quality of the coverage results and it is unclear to us why SCI has not recorded, or if it has recorded it, why it has not shared this information. In Mozambique, respondents were asked whether they recognized the dose pole used in schistosomiasis MDAs. However, in that survey parents were surveyed on their children's behalf127 and most children (79%) received drugs at school,128 presumably when parents were not present. We don't know how to interpret the result that a very high percentage of parents (median 90%, ranging 61-94% across provinces) reported recognizing the dose pole.129 SCI hypothesized that parents may either recognize the dose pole from publicity efforts prior to the MDA or remember a similar dose pole from previous MDAs for lymphatic filariasis.130 In either case, this may indicate that the coverage survey is not measuring actual delivery of drugs to children in the most recent MDA.
 
Accuracy of responses from young children: SCI told us, "It can be difficult to get clear, accurate answers from young children (5-6 years old). Children, especially younger ones, may be influenced by others who are around during the survey. This is especially so because surveyors often interview older children first, in front of younger siblings."131
 
Supervision and auditing of surveys: There is no mention in the coverage survey results of any re-surveying of households to check the accuracy of the data collected and, although "supervisors" are mentioned in several reports, it is not clear what role they played.132 SCI described to us the supervision used in the Mozambique survey (details in footnote); audits were not mentioned.133
 
The fact that the surveys identified low coverage in several cases increases our confidence in their reliability. Given the smaller sample size, government involvement in the survey, and question about parents recognizing the dose pole noted above, we are more skeptical about the results from Mozambique than those from other countries.
 
 
 
Mass drug administration coverage among school-aged children
 
 
 
Survey Median PZQ coverage and range Median ALB coverage and range
 
Malawi (2012) 77% (64%-90%) 59% (33%-85%)
 
Malawi (2014) 69% (55%-77%) 44% (25%-77%)
 
Côte d'Ivoire (2014) 82% (67%-88%) 82% (68%-89%)
 
Uganda (2014) 47% (24%-86%) N/A
 
Mozambique (2015) 81% (73%-89%) N/A
 
Zambia (2015) 93% (89%-94%) N/A
 
Zanzibar (2015) 80% (75%-85%) 87% (84%-89%)
 
Notes about these results:
 
 
 
Results broken down by district (or similar geographic region) are in this spreadsheet.
 
PZQ is the drug used to treat schistosomiasis. ALB is the drug used to treat STH.
 
Ranges and medians are calculated from district-level results.
 
The results are for school-aged children only. Some of the surveys also measured coverage rates in adults, who are targeted for treatment in some SCI-supported programs.
 
The results exclude districts in Uganda that were included in the survey but which do not receive support from SCI.134
 
For context, the World Health Organization recommends that treatment programs aim for coverage rates above 75%.135
 
Are SCI's monitoring results representative of its work overall?
 
We have now seen recent monitoring results from about half of the countries in which SCI works, including the five countries were SCI has delivered the largest number of treatments. For those countries from which we have seen monitoring results, we have generally seen one year of results, though SCI has worked in the country for several years.
 
 
 
Details in this spreadsheet.
 
 
 
For countries or years for which we have not seen monitoring data, it is generally unclear to us whether SCI collected additional monitoring data that it has not shared with us or whether it did not collect monitoring data. SCI told us that it is sometimes unable to share results because third parties (e.g. governments, WHO, funders) often need to give permission before data can be shared and because it can take some time for data, once collected, to reach SCI because in some countries it is cleaned and analyzed by country program staff before being shared with SCI.136
 
 
 
For Mozambique, one of SCI's largest recipients of both restricted and unrestricted funds, SCI shared a report from a consultant who visited the country in May 2015 to assist with data cleaning and analysis for prevalence data from 2012, 2013, and 2014. The report notes major problems with this data and refusal of the government to allow SCI to have access to the data outside of Mozambique.137
 
 
 
Additional academic evidence
 
To provide additional information on SCI's track record, in 2014, we conducted a search for published papers on treatment coverage rates and schistosomiasis and STH prevalence and intensity in countries where SCI has worked.
 
 
 
We focused on papers by Melissa Parker and Tim Allen, who were funded by the Bill and Melinda Gates Foundation to provide an anthropological perspective on SCI's work,138 papers cited in papers by Melissa Parker and Tim Allen, and other papers we identified on this topic from a Google Scholar search.139
 
 
 
The papers we identified were all from Tanzania, Uganda, and Zanzibar, perhaps because these are places that Melissa Parker and Tim Allen's work has focused on.
 
 
 
This spreadsheet summarizes the papers we considered. We have not fully vetted these studies. The studies were generally designed for purposes other than to evaluate SCI's programs so in many cases there is uncertainty about SCI's role in the areas studied. In the discussion below, we have excluded studies that we do not believe were conducted soon after treatment programs in areas targeted by SCI-funded programs.
 
 
 
SCI staff and/or SCI funding were involved in many of the studies.140
 
 
 
It is difficult to draw any conclusions from these studies because of small sample sizes and lack of clarity on SCI's role in each location. In summary:141
 
 
 
Tanzania (excluding Zanzibar): Two studies at the district level, Stothard et al. 2013 and Chaula and Tarimo 2014, show relatively low schistosomiasis prevalence after treatment (4% and 15% respectively; the latter claims that prevalence pre-MDA was 30%). SCI was working in Tanzania at the time of both studies, but it is not clear if SCI was working in the specific areas studied. Chaula and Tarimo 2014 also shows low treatment coverage (around 40%), though the researchers asked in 2013 about treatment provided in 2011 and 2012.
 
Zanzibar: All studies were conducted on Unguja, "the largest and most populated island of Zanzibar."142 Two studies, Stothard et al. 2009 and Rudge et al. 2008, each in a single school, found high prevalence of schistosomiasis in areas that had received treatment (50% and 65% prevalence, respectively). The sample sizes were small, it was not clear why these particular schools were selected, and, while SCI was working in Zanzibar at the time of both studies, but it is not clear if SCI was working in the specific areas studied. Another study, Knopp et al. 2009, in two schools in an area that had received treatment (but perhaps not from SCI) found moderate prevalence of various STHs (21% prevalence for hookworm and low infection intensity). The schools were selected because they had been surveyed in 1994, when prevalence was found to be much higher.
 
Uganda: We reviewed three studies of schistosomiasis prevalence: Muhumuza et al. 2013 was conducted in part of a district that seems to have received several rounds of treatment (but perhaps not from SCI). Brooker et al. 2005 was conducted across a district that had its first MDA the year before, which was funded by SCI. Standley et al. 2009 was conducted across six districts, one district that had never been treated (but from which only one school, which had low prevalence, was included in the analysis), two districts that seem to have been treated only once, and three districts that seem to have been treated for many years by SCI. Standley et al. 2009 explicitly aimed to "ambush" schools that might not have been reached by treatment programs.143 Each study found moderate to high prevalence of schistosomiasis (Muhumuza et al. 2013: 35%, Brooker et al. 2005: 28%, and Standley et al. 2009: 42%). Muhumuza et al. 2013 and Standley et al. 2009 found fairly high infection intensity (116 and 634 average eggs per gram respectively among positive cases; the WHO threshold for "high intensity" is 400 eggs per gram144). Muhumuza et al. 2013 found 28% coverage in the most recent round of treatment. The coverage survey in Muhumuza et al. 2013 seems to have been six months after treatment.
 
Possible negative and offsetting impact
 
Concerns over whether treatment was sustained: We believe it is important that deworming programs are sustained over time, as re-infection is rapid and a one-time treatment may have little long-term effect.145 It is not clear to us the extent to which SCI-funded programs have succeeded in treating the same children multiple times, as opposed to simply treating the same areas multiple times (and thus treating different children once each).
 
 
 
We remain unsure about how many treatments are needed to impact health. SCI told us that its views on what groups should be treated and how often "is largely based [on] WHO recommendations, but also on government wishes and on intuition and common sense, though it usually works and SCI collects sufficient data to know when it isn't working. In general, in high endemicity areas re-infection is a major issue; in lower endemicity areas, a single treatment can be sufficient."146 One example of the variation in treatment patterns is what SCI told us about its program in Yemen:
 
 
 
SCI distinguishes between high, medium and low prevalence areas. In high prevalence areas, SCI treats the whole population once, and children for five years. In medium prevalence areas, SCI treats the whole population once, and children every other year. In low prevalence areas, SCI treats children every other year.147
 
Displacement of government funding for deworming: This could be a concern if, due to SCI's spending on national deworming programs, government funds that otherwise would have been spent on deworming are spent on other, less worthwhile budget items. In the past, SCI has largely supported programs that did not exist before its support.148 We have not seen data on government spending on NTDs before and after receiving SCI support. Ethiopia's Neglected Tropical Diseases Program Manager told us that the government had not dedicated funding to schistosomiasis and that without SCI's involvement the program would likely not have gone forward.149
 
Diversion of skilled labor: Drug distribution occurs only once or twice per year and appears to be conducted by teachers, community drug distributors (who receive minimal training to fulfill this role), and health center staff.150 Given the limited time and skill demands of mass drug distribution, we are not highly concerned about distorted incentives for skilled professionals.
 
Popular discontent: We have heard a couple accounts of discontent in response to SCI's mass drug administration campaigns, including one case that led to riots.151 SCI notes that following episodes of popular discontent, it has worked with governments to improve public education about the programs.152
 
What do you get for your dollar?
 
We estimate that on average the total cost of a schistosomiasis treatment delivered in SCI's programs is $1.26. Excluding the cost of drugs (which are often donated) and in-kind government contributions to the programs, we estimate that SCI's cost per treatment is $0.53. There is significant uncertainty around this number. As discussed above, the information we have seen on SCI’s expenses is limited and possibly unreliable. Similarly, we are not confident in the accuracy of the data we have seen on number treatments delivered. Given this, we make a number of assumptions and judgments in interpreting the data that we have seen and this could introduce errors (which could potentially overstate or understate the actual cost, though we generally try to conservatively err on the side of counting fewer treatments and more costs).
 
 
 
In 2014, SCI estimated its cost per treatment at $0.80.153 As of October 2015, SCI estimates that after April 2016 its cost per treatment excluding costs for drugs and costs paid for by governments will generally be about $0.30.154 We are unsure how it calculated these estimates. In September 2015, SCI told us that it expects to improve its cost-effectiveness analysis by the end of the year.155
 
 
 
We discuss how the cost per treatment figure relates to how much it costs to improve a child's health and development at our report on mass treatment programs for schistosomiasis and STHs.
 
 
 
The remainder of this section discusses our cost per treatment analysis. Our calculations and sources are shown in more detail in this spreadsheet. We also discuss the prevalence and intensity of disease in the places SCI works, which we use to adjust our estimate of SCI's cost-effectiveness compared to the cost-effectiveness of the best-studied deworming programs.
 
 
 
In 2014, we estimated that on average SCI delivers a schistosomiasis treatment for $1.23. The main changes since last year's estimate:
 
 
 
DFID began providing funding for programs in Ethiopia and DRC in the 2014-15 budget year and we have added the costs of these programs (starting in April 2013156) and number of treatments delivered to our estimate.
 
We added treatments and costs for the 2014-15 budget year.
 
As discussed below, we discounted the number of reported treatments by 10%.
 
We added an estimate of costs paid by Imperial College (e.g., office space and some legal and administrative expenses). We increased SCI's costs by 10% to account for this.
 
Our approach
 
Our general approach to calculating the cost per treatment is to identify comparable cost and treatment data and take the ratio. We prefer to have a broadly representative selection of treatments in order to mitigate possible distortions, such as using data from a new program, which may incur costs from advocacy, mapping, etc. before it has delivered any treatments.
 
 
 
It is our understanding that SCI intends to treat for STHs in all places where it treats for schistosomiasis, so the treatments SCI reports can generally be interpreted as combination schistosomiasis and STH treatments,157 though we are aware of several cases in which schistosomiasis-only treatments were delivered either by design or due to problems with implementation.158
 
 
 
To get the total cost, we attempt to include all partners (not just SCI) such that our cost per treatment represents everything required to deliver the treatments.159 In particular, we include these categories:
 
 
 
SCI’s funding to country programs (e.g. to fund drug delivery).
 
SCI’s headquarter costs (e.g., for management and technical salaries), including an estimate of costs paid by Imperial College (e.g., office space and some legal and administrative expenses).
 
Cost of drugs. We include the full market cost of all praziquantel that is needed to deliver the treatments, regardless of whether SCI purchased it or used donated drugs.
 
Costs incurred by the government implementing the program (e.g. for staff salaries when working on treatment programs).
 
SCI notes that cost per treatment calculations should include sensitivity analysis160 – analysis on the degree to which the cost per treatment varies when various assumptions vary. We have not yet completed such an analysis.
 
 
 
Our analysis
 
We analyzed several sources of data, which cover different time periods, and developed a several estimates, two of which we summarize here. Full details in this spreadsheet.
 
 
 
SCI’s recent programs: $1.26 per treatment. The ICOSA program (the name for SCI's DFID-funded work) covers eleven country programs and has been funded primarily by SCI’s first DFID grant, with additional funding from both restricted and unrestricted sources. We have seen data that we think provide a relatively comprehensive picture of spending and treatments over the first five years of the program (October 2010 - March 2015). We have not included the cost of research programs (which are funded separately) in these countries. It is possible that some of this research funding contributed to the treatments we are counting. A more conservative estimate, including about $1.8 million of research expenses in the same countries, is $1.32 per treatment.
 
SCI’s early programs: $1.46 per treatment. SCI previously shared with us a summary of the treatments it delivered and the costs it incurred during its early programs (from about 2003-2009). There is some ambiguity in the treatment data that we have interpreted conservatively (in other words, this estimate may be too high, as described in the footnote).161
 
Note that these estimates are in nominal dollars – we have not adjusted for inflation.
 
 
 
Shortcomings of our analysis
 
While we believe the estimate described above is reasonable, we want to highlight specific reasons to interpret it with caution.
 
 
 
We rely on reported treatment data. Our understanding is that these data can overstate treatments, so we have discounted this data (by 10%) based on the differences between reported treatment rates and treatment rates found in the coverage surveys discussed above (see footnote for why this is an imperfect comparison).162
 
 
 
We rely on an estimate that 30% of overall program costs are attributable to the government. We derived this from an analysis of a single program in Niger (this footnote elaborates on the details and concerns).163
 
 
 
We do not have data that indicate what proportion of drugs are wasted. We expect that in some cases drugs are purchased or donated but expire before use. We do not know how common this is. In our analysis, we have assumed that 10% of drugs are wasted, which increases the cost per treatment by about $0.05.
 
 
 
We do not have data on Imperial College's expenses that support SCI. Based on a conversation with SCI, we have roughly estimated this support as 10% of SCI's expenses (excluding drugs and government contributions).164
 
 
 
We simply estimate an average cost across programs and do not account for variations in different contexts. SCI told us that costs can vary significantly, for example, due to increased transportation costs in some contexts.165
 
 
 
Baseline infection status
 
SCI's current and future programs may be less cost-effective than past programs or than programs discussed in our report on deworming because of lower baseline infection rates in current and future programs.
 
 
 
In 2014, SCI shared baseline data from countries it had recently started work in. We compare these rates to rates observed in the best evidence for the effectiveness of deworming, in order to understand how similar SCI's impact is likely to be to that observed in the studies. Schistosomiasis and STH prevalence and intensity in these countries was generally fairly low compared to the studies providing the best evidence for the benefits of deworming (Bleakley 2007, Croke 2014, and Miguel and Kremer 2004) and compared to some of SCI’s earlier prevalence and intensity studies (from Burkina Faso, Uganda, Niger, and Burundi).166
 
 
 
For the most part, baseline data was collected in schools that had been selected for prevalence and intensity studies. The baseline reports use methodologies that seem similar to the other SCI panel studies discussed above. With the exception of the study discussed above from Malawi, we have not examined how representative these schools are of the full treatment area or fully vetted the methodology used. For Ethiopia, we have used data collected during disease mapping and made several assumptions to make that data comparable to the data from other countries.167
 
 
 
In Malawi an error in data collection may have resulted in prevalence being underestimated.168 In Zanzibar, treatment has been ongoing,169 so the study does not reflect pre-treatment conditions.
 
 
 
Detailed results and sources in this spreadsheet.
 
 
 
 
 
Is there room for more funding?
 
In short:
 
 
 
Estimated needs: SCI estimates that it would use the following amounts of unrestricted funding in each of the next three years (in millions of US dollars):
 
April 2016- March 2017: $9.5
 
April 2017- March 2018: $13.6
 
April 2018- March 2019: $13.3
 
Cash on hand: SCI currently has relatively limited unrestricted funding available. Our best guess is that, absent a renewed GiveWell recommendation in November 2015, SCI would hold approximately $1.5 million as of April 2016 (the start of its next fiscal year) that it could allocate to program implementation. In addition, it will hold approximately $662,000 and 2 years worth of staff salaries in reserve. December 2015 update: Good Ventures made a $1 million grant to SCI.
 
Other sources of funds: Our impression is that GiveWell-influenced donors contribute most of SCI’s unrestricted funds. As a result of the funding it has received due to GiveWell's recommendation, SCI has not prioritized fundraising. After SCI set its fundraising targets, a funder committed $2 million per year for three years for the Ethiopia deworming program.
 
Past spending: SCI has allocated all of the unrestricted funds it had received by April 2015 (including $6.4 million from GiveWell-influenced donors in 2014170), and expects to have spent these funds by April 2016. We have relatively limited information about whether it has continued to spend additional funds in a cost-effective manner, particularly in the past year when it received a significant increase in unrestricted funds.
 
Additional considerations: We have continued to face communications challenges with SCI that reduce our confidence that we have a complete, accurate understanding of SCI’s financial situation. Also, the plans SCI shared with us at the end of 2014 did not match its eventual use of the funds it received and it is unclear to us what caused the shift (more in our August 2015 update).
 
Details follow.
 
 
 
Sources of uncertainty
 
Overall, there is significant uncertainty in our expectations of how much additional funding SCI could use and how additional funding would be spent. We believe this is primarily due to SCI having limited ability to predict the opportunities it will have in the future because of quickly changing circumstances in many countries and our relative lack of understanding about how SCI has spent funds in the past and why its plans have changed. Country programs may face many different non-funding related constraints, and these seem hard to predict. For example, political unrest delayed the program in Côte d'Ivoire for 18 months,171 in 2014 SCI was not yet ready to allocate additional funds to Mozambique because of lack of confidence in the program's ability to scale further at that time,172 and the Ebola outbreak has delayed work in Liberia.173 Factors that can shift SCI’s planned uses of unrestricted funding include political unrest, expiring drug supplies, additional donated drugs becoming available, delays and budget changes due to coordination with other actors, results of disease mapping, and grants from other donors.174
 
 
 
Challenges communicating with SCI on its room for more funding may also be contributing to our uncertainty. In the past, our understanding was developed largely through conversations with SCI's leadership, supplemented with details from many other sources. In retrospect, our understanding of how SCI planned to use funds often did not match how SCI decided to allocate funds the next time it set program budgets.175
 
 
 
This year, we have relied primarily on information from SCI about how much total unrestricted funding it aims to raise and how it would spend this funding. We have limited information on:
 
 
 
How SCI will prioritize opportunities if it receives less than its fundraising goal. We have not attempted to learn more about this because our understanding is that SCI's budgeting process is based on meetings with program managers in the first few months of the year and it is difficult for SCI to predict how programs will be prioritized ahead of time.
 
How SCI would use funding that exceeded its fundraising goal.
 
How likely it is that each country program might face non-funding constraints to scaling up or, in the case of Nigeria, to starting a new program. SCI has taken initial steps toward starting a program in Nigeria. SCI attended a meeting in June 2015 in Nigeria to discuss the possibility of SCI working in the country.176 SCI told us that, more recently, leaders from several Nigerian states have approached SCI to request its assistance starting deworming programs.177
 
Other potential funders. SCI has applied for a $500,000 grant for its work in Malawi and a £100,000 (about $150,000) grant for its work in Madagascar.178 Beyond these grants, it is our understanding that SCI does not have major funding prospects for the next year, but that existing funders could make gifts that would reduce its room for more funding.179 In the past, SCI has not actively fundraised from corporate or individual donors, but it is considering doing more by hiring someone to work on this or working with Imperial College's fundraising team.180 GiveWell-influenced donors contribute most of SCI’s unrestricted funds.181 In 2014, we tracked about $6.4 million donated to SCI as a result of our recommendation.182 SCI's draft financial statements for April 2014 - March 2015, state that SCI raised about $7.2 million in unrestricted funds in that period.183 As we discuss above, we have concerns about the accuracy of these financial statements.
 
In addition, as of October 2015, SCI reported a $2.22 million funding gap in Ethiopia for the budget year 2016-2017, a $2.96 million gap for 2017-2018, and a $3.06 million gap for 2018-2019.184 Since then, a funder committed $2 million per year for three years to the Ethiopia deworming program. The grant is not to SCI and SCI notes that the uses of this funding may not fully overlap with its planned activities.185 SCI also notes that it still aims to reach its previously stated fundraising goal because this would give it some more flexibility. For example, it is interested in spending $1 million to purchase schistosomiasis drugs for adults in Ethiopia.186
 
 
 
Available unrestricted funding
 
We do not know how much unallocated, unrestricted funding SCI currently holds. We guess that, in the absence of a renewed GiveWell recommendation in late 2015, it would have very roughly $2.2 million to allocate for programs and funding reserves in its April 2016 to March 2017 budget year. This guess is based on (a) SCI telling us in early 2015 that it expected to spend all unrestricted funds it held as of that time by the end of the budget year,187 and (b) a rough estimate that SCI will receive around $2.2 million in unrestricted funds between April 2015 and March 2016, absent a renewed GiveWell recommendation in November 2015.188
 
 
 
Starting in April 2016, SCI may begin holding some unrestricted funds in reserve to help sustain programs in the event that its revenue decreases. It expects to allocate one month's worth of expenses (based on 2014-2015 spending), about $662,000, to this reserve,189 leaving very roughly $1.5 million available to allocate to programs (~$2.2 million minus ~$0.7 million).
 
 
 
Uses of additional funding
 
SCI aims to raise $9.5 million for its April 2016 to March 2017 budget year (this figure does not appear to take into account unallocated funds that SCI currently holds; more above). SCI has $7.4 million in restricted funds committed for that budget year from DFID, the END Fund, and other donors, so in total it aims to have a budget of $16.9 million. SCI projects that it would deliver 55.1 million treatments with a budget of this size. This is an increase over the 42.8 million treatments SCI expects to deliver in April 2015 to March 2016.190 Accounting for a $1 million grant from Good Ventures awarded in December 2015 (discussed below), we estimate that SCI has about ~$4.9 million (~$9.5 million minus ~$1.5 million minus $2 million minus $1 million) in room for more funding for its next budget year.191
 
 
 
Additional funding would support programs in 15 countries. SCI has previously supported programs in 14 of these countries. It has not yet worked in Nigeria. Five country programs would receive 81% of the additional funding: Ethiopia (24%), Mozambique (21%), Sudan (14%), Nigeria (13%), and Malawi (9%).192
 
 
 
Full breakdown of how SCI would spend additional funding by country is in this spreadsheet.
 
 
 
SCI told us that if it does not fully close its funding gap the first thing it would cut would be support for a new program in Nigeria. If it were to raise more than $9.5 million, it might have opportunities to support new programs in additional countries, for example in Chad.193
 
 
 
SCI aims to continue to expand its programs between April 2017 and December 2019 and has a goal of delivering at least 300 million treatments between 2010 and 2019. To do so, it estimates that it will need to raise an additional $13.6 million to support programs for April 2017 to March 2018 and $13.3 million for April 2018 to December 2019.194
 
 
 
GiveWell's prioritization of SCI's funding gaps
 
We have broken down our top charities' funding gaps and ranked them based on:
 
 
 
Capacity relevance: how important the funding is for the charity's development and future success.
 
Execution relevance: how likely it is that the charity's activities will be constrained if it does not receive the funding.
 
We believe that "capacity-relevant" gaps are the most important to fill, and "execution"-related gaps vary in importance. More explanation of this model is in this blog post.
 
 
 
We do not classify any of SCI's estimated need for unrestricted funding over the next three years as "capacity-relevant." We cannot determine whether funds received would be important for SCI's development and future success since our overall understanding of what the funds would be used for is limited. We consider the funding gaps for SCI's planned programs to be "execution" gaps and assign them a level (1, 2, or 3) that corresponds to how likely we believe it is that SCI would be constrained by funding if it is unable to fill the funding gap. Level 1 is a 50% chance of funding being the constraint over the next year, level 2 is a 20% chance, and level 3 is a 5% chance. These judgments are rough, and largely based on our intuitions.
 
 
 
We estimate that SCI has $4.9 million in remaining room for more funding for April 2016-March 2017, which we classify as its Execution Level 1 funding gap.195
 
 
 
We have also very roughly estimated that SCI's Execution Level 2 is its funding gap for its 2017-2018 budget year (an additional ~$11.6 million). Even more roughly, we estimate its Execution Level 3 funding gap by topping up the sum of its Execution Level 1 and Level 2 funding gaps by 50% (an additional ~$8.8 million) – details in footnote.196
 
 
 
Global need for treatment
 
There appears to be a substantial unmet need for schistosomiasis treatment globally.
 
 
 
SCI shared an unpublished World Health Organization (WHO) estimate of the number of school-aged children and adults who require treatment for schistosomiasis and those who received treatment in 2014. We do not have permission to publish this data, but we note that only a small portion of those who WHO believes need treatment received it in 2014.197
 
 
 
SCI as an organization
 
Track record: SCI has consistently gotten national deworming programs to go through, as discussed above. We know fairly little about how effectively these programs have delivered treatments.
 
Self-evaluation: SCI’s self-evaluation is strong compared to the vast majority of organizations we have considered. That said, this evidence is incomplete and has quite a few limitations. In addition, we have a significantly different perspective than SCI on the strength of the evidential case for deworming (see our 2012 post on deworming and the comments that follow it).
 
Transparency: SCI has consistently been strong in its commitment to transparency. It has generally provided the information we’ve asked for and has never hesitated to share it publicly (unless it had what we felt was a good reason). It has allowed a lot of public dialogue that other charities may have been uncomfortable with.
 
Communication: We don’t feel that SCI has ever purposefully been indirect with us, but we have often struggled to communicate effectively with SCI representatives, and we still lack important and in some cases basic information about SCI's finances. We find this problematic given the amount of effort we have put into understanding SCI's finances.
 
More on how we think about evaluating organizations at our 2012 blog post.
 
 
 
Sources
 
Document Source
 
Alan Fenwick, email to GiveWell, November 3, 2015 Unpublished
 
Alan Fenwick, email to GiveWell, October 15, 2015 Unpublished
 
Alan Fenwick, email to GiveWell, October 1, 2015 Unpublished
 
Alan Fenwick, email to GiveWell, October 29, 2015 Unpublished
 
Alan Fenwick, email to GiveWell, September 28, 2015 Source
 
Alan Fenwick, SCI Director, conversation with GiveWell, October 14, 2014 Unpublished
 
Alan Fenwick, SCI Director, conversation with GiveWell, October 15, 2014 Unpublished
 
Alan Fenwick, SCI Director, email to GiveWell, November 24, 2014 Unpublished
 
Alan Fenwick, SCI Director, phone conversation with GiveWell, February 16, 2011 Unpublished
 
Alan Fenwick, SCI Director, phone conversation with GiveWell, September 15, 2011 Unpublished
 
Alderman et al. 2006 Source (archive)
 
Allen and Parker 2011 Source (archive)
 
Allen and Parker 2012 Source (archive)
 
Anna Phillips, SCI Country Program Manager for Burkina Faso and Niger, email to GiveWell, October 13, 2011 Source
 
Benjamin Styles, SCI Senior Biostatistician, phone conversation with GiveWell, August 12, 2011 Unpublished
 
Bleakley 2007 Source (archive)
 
Brooker et al. 2005 Source (archive)
 
Chaula and Tarimo 2014 Source (archive)
 
Croke 2014 Source (archive)
 
Crown Agents Total ICOSA Procurement Spend (updated 2014) Unpublished
 
DFID glossary Source (archive)
 
Fenwick et al. 2009 Source (archive)
 
Fiona Fleming, conversation with GiveWell, November 5, 2015 Unpublished
 
Fiona Fleming, email to GiveWell, November 5, 2015 Unpublished
 
Fiona Fleming, email to GiveWell, September 18, 2015 Unpublished
 
Fiona Fleming, SCI Senior Monitoring & Evaluation Manager, conversation with GiveWell, October 14, 2014 Unpublished
 
Fiona Fleming, SCI Senior Monitoring & Evaluation Manager, conversation with GiveWell, September 21, 2015 Unpublished
 
Fiona Fleming, SCI Senior Monitoring & Evaluation Manager, email to GiveWell, November 9, 2014 Unpublished
 
Gates Foundation, Imperial College London (June 2002) Source
 
GiveWell analysis of SCI spending under DFID grant Source
 
GiveWell analysis of SCI spending under DFID grant (updated 2015) Source
 
GiveWell summary of SCI finances (October 2014) Source
 
GiveWell's non-verbatim summary of a conversation with Alan Fenwick and Najwa Al Abdallah, September 14, 2015 Source
 
GiveWell's non-verbatim summary of a conversation with Alan Fenwick, SCI Director, July 31, 2015 Source
 
GiveWell's non-verbatim summary of a conversation with Alan Fenwick, SCI Director, June 17, 2010 Source
 
GiveWell's non-verbatim summary of a conversation with Alan Fenwick, SCI Director, October 2, 2015 Source
 
GiveWell's non-verbatim summary of a conversation with Giuseppina Ortu on June 20, 2014 Source
 
GiveWell's non-verbatim summary of a conversation with Lynsey Blair, October 16, 2014 Source
 
GiveWell's non-verbatim summary of a conversation with Michael French, October 15, 2014 Source
 
GiveWell's non-verbatim summary of a conversation with Oumer Shafi, November 4, 2014 Source
 
GiveWell's non-verbatim summary of a conversation with Sarah Nogaro, October 16, 2014 Source
 
GiveWell's non-verbatim summary of a conversation with Wendy Harrison and Najwa Al Abdallah on September 8, 2015 Source
 
GiveWell's non-verbatim summary of a conversation with Yolisa Nalule, October 14, 2014 Source
 
GiveWell's notes from visit to Malawi on October 17-19, 2011 Source
 
GiveWell’s non-verbatim summary of conversations with Grace Hollister on September 21 and October 1, 2015 Unpublished
 
Kabatereine et al. 2001 Source (archive)
 
Kabatereine et al. 2006 Source (archive)
 
Kabatereine et al. 2007 Source (archive)
 
Kieran Bird, SCI Finance Manager, conversation with GiveWell, October 16, 2014 Unpublished
 
Kieran Bird, SCI Finance Manager, email to GiveWell, November 4, 2014 Unpublished
 
Knopp et al. 2009 Source (archive)
 
Knopp et al. 2013 Source (archive)
 
Koukounari 2011 Source
 
Koukounari et al. 2007 Source (archive)
 
Leslie et al. 2011 Source (archive)
 
LSTM Mozambique trip report (May 2015) Unpublished
 
Mapping of Schistosomiasis and Soil-transmitted helminths in Yemen Source
 
Mazigo et al. 2012 Source (archive)
 
Michelle Clements, SCI Senior Biostatistician, conversation with GiveWell, October 15, 2014 Unpublished
 
Miguel and Kremer 2004 Source (archive)
 
Muhumuza et al. 2009 Source (archive)
 
Muhumuza et al. 2013 Source (archive)
 
Muhumuza et al. 2014 Source (archive)
 
Najwa Al Abdallah, conversation with GiveWell, October 22, 2015 Unpublished
 
Nigeria NTD stakeholders meeting and potential SCI involvement scoping document Source
 
Nigeria stakeholders meeting summary (May 2015) Source
 
Nigeria trip report (June 2015) Source
 
Parker and Allen 2011 Source (archive)
 
Parker and Allen 2014 Source (archive)
 
Parker, Allen, and Hastings 2007 Source (archive)
 
Pinot de Moira et al. 2010 Source (archive)
 
Rudge et al. 2008 Source (archive)
 
Scheich et al., 2012 Source (archive)
 
SCI Account summary (May 2011) Source
 
SCI advisory board financial report (June 2013) Source
 
SCI advisory board financial report (June 2014) Source
 
SCI board financial details (June 2014) Unpublished
 
SCI Board management accounts (April 2010) Source
 
SCI Burundi June 2014 Open Day poster Source
 
SCI Burundi: Impact Source (archive)
 
SCI CNTD spending data (FY2-FY4) Source
 
SCI contribution to the global effort to control and eliminate schistosomiasis Source
 
SCI Côte d'Ivoire coverage survey 2014 Unpublished
 
SCI Côte d'Ivoire panel study baseline report Unpublished
 
SCI draft budget 2015-2016 Source
 
SCI draft financial statements for 2013/14 and 2014/15 Source
 
SCI draft reserves policy (September 2015) Source
 
SCI Ethiopia mapping of SCH and STH 2014 Source
 
SCI Ethiopia treatment campaign summary report (April 2015) Source
 
SCI financial statement 2013/14 and 2014/15 (revised October 2015) Source
 
SCI fundraising targets (November 2014) Source
 
SCI Gates Foundation final report (January 2011) Source
 
SCI IC Trust summary (September 2011) Source
 
SCI ICOSA Mid-Year Report 2014 Source
 
SCI impact and coverage survey plans Source
 
SCI Imperial initiative to protect children from tropical disease awarded ₤25m government backing Source (archive)
 
SCI Liberia panel study baseline report Unpublished
 
SCI M&E timeline (May 2015) Source
 
SCI Malawi coverage survey 2012 Source
 
SCI Malawi coverage survey 2014 Source
 
SCI Malawi impact study – second follow up Source
 
SCI Malawi panel study Source
 
SCI Malawi spending data (November 2011 to August 2013) Source
 
SCI Mozambique coverage survey 2015 Source
 
SCI Neglected tropical diseases in Mozambique Unpublished
 
SCI Niger panel study 2011 Unpublished
 
SCI Niger spending data (October 2011 to May 2013) Source
 
SCI planned SCH treatment numbers by country by year (October 2015) Source
 
SCI Proposal by SCI, Imperial College to manage the Program for Integrated Control of Neglected Tropical Diseases in Côte d'Ivoire Unpublished
 
SCI report to DFID (October 2013) Source
 
SCI report to DFID (September 2015) Source
 
SCI report to GiveWell (September 2013) Unpublished
 
SCI report to GiveWell (September 2014) Source
 
SCI responses to GiveWell questions on financial statements (October 2015) Source
 
SCI Rwanda June 2014 Open Day Poster Source
 
SCI Rwanda: Strategy Source (archive)
 
SCI Summary sheet of treatments instigated and overseen by SCI Source
 
SCI supporting documents matrix (September 2015) Source
 
SCI Tanzania spending data (March 2011 to July 2013) Source
 
SCI treatment numbers (October 2014) Source
 
SCI Uganda coverage survey 2014 Source
 
SCI Uganda panel study baseline report Unpublished
 
SCI Uganda spending data (September 2011 to August 2013) Source
 
SCI Zambia coverage survey 2015 Source
 
SCI Zambia panel study baseline report Unpublished
 
SCI Zanzibar coverage survey 2015 Source
 
Standley et al. 2009 Source (archive)
 
Standley et al. 2010 Source (archive)
 
Stothard et al. 2009 Source (archive)
 
Stothard et al. 2013 Source (archive)
 
Styles 2011 Source
 
Sudan annual workplan (April 2015 to March 2016) Source
 
Sudan annual workplan for WHO (2015) Source
 
Sudan campaign photos Source
 
Sudan cash book Source
 
Sudan joint request for selected PC medicines Source
 
Sudan NTD concept paper (2015-2018) Source
 
Sudan PZQ and ALB treatments by locality (2015) Source
 
Summary Technical Report: Schistosomiasis Control in Yemen (July 2014) Source
 
Tohon et al. 2008 Source (archive)
 
Top 20 countries, estimated schistosomiasis infections Source
 
Touré et al. 2008 Source (archive)
 
Utroska et al. 1989 Source (archive)
 
Wendy Harrison and Sarah Knowles, SCI Managing Director and Biostatistician, conversations with GiveWell, April 9 and 14, 2014 Source
 
Wendy Harrison, SCI Managing Director, email to GiveWell, March 4, 2014 Unpublished
 
WHO schistosomiasis treatment gap data Unpublished
 
Wikipedia entry for Unguja Source (archive)
 
EXPAND FOOTNOTES
 
Related Content
 
All content on SCI
 
All blog posts on SCI
 
Malawi site visit
 
HOME CONTACT STAY UPDATED FAQ FOR CHARITIES SITE MAP PRIVACY POLICY
 
FOLLOW US:
 
Facebook
 
 
Twitter
 
 
RSS
 
SUBSCRIBE TO EMAIL UPDATES:
 
EMAIL ADDRESS
 
GiveWell, aka The Clear Fund (a tax-exempt 501(c)(3) public charity), was founded in 2007. We serve donors across the Globe; GiveWell's donors are based primarily in the United States, United Kingdom, Australia, Germany, and Canada. This work is licensed under a Creative Commons Attribution-Noncommercial-Share alike 3.0 United States License
 

Latest revision as of 12:01, 12 September 2020

Feature testing

testing crop feature
testing crop feature
testing crop feature
testing crop feature
testing crop feature
testing crop feature


Visual data source repository