Why we have over-rated Cool Earthpost by Sanjay · 2018-11-26T02:29:41.731Z · EA · GW · 30 comments
0. Exec summary 1. How Cool Earth works -- intro 2. Why is it considered a high-impact choice of charity to donate to? 3. Reasons to doubt Cool Earth’s impact 4 Impactfulness varies by country 5 Other options for donors to consider Conclusions Appendices Appendix 1: More background info about loggers and logging Appendix 2: Summary of GWWC’s $1 per tonne of CO2 analysis Appendix 3: Displacement risk -- more details Appendix 4: More on Cool Earth’s opinion on Papua New Guinea Appendix 5: Polarisation -- why Cool Earth is likely highly cost-effective or not at all Appendix 6: What is the marginal impact of the next pound donated? Appendix 7: Providing contraception None 30 comments
0. Exec summary
Cool Earth is a non-profit that tackles climate change; it has been enthusiastically supported by many in the Effective Altruism community. In this post, I will say the following:
1. How Cool Earth works
2. According to earlier analysis by Giving What We Can (GWWC), Cool Earth is a high-impact charity
3. However this analysis omits certain aspects/risks
4. Once you take these into account, Cool Earth’s work is either fairly low impact, or there is (currently) not enough evidence available to say; this depends on which project (i.e. country) you’re looking at
5. Some other options that climate-interested donors may wish to consider
6. At this stage the information gathered doesn’t allow me to give a strong opinion on Cool Earth, but I would recommend that we now consider the earlier GWWC analysis out of date
As for the omitted aspects/risks (item 3), these include the displacement effect -- i.e. when loggers are unable to log the trees protected by one community, they will simply go elsewhere. A secondary concern is risks post-exit. (read on for more explanation of these)
Many thanks to Cool Earth for their input, and note that Cool Earth do not agree with all that has been said in this piece. I will try to represent their views as well.
I would have preferred to investigate these issues more fully before sharing this piece, given that some of the information gaps are quite material, however that has not been possible, largely because of time constraints on the part of Cool Earth. I would also like to thank Cool Earth for the time they have given me for this analysis.
As for whether to donate to Cool Earth or not, I would suggest not to based on the current evidence, although as more evidence arises, this opinion could change.
Donors interested in climate change may be interested in the recent analysis conducted by Founders Pledge. I will also list below some other options that donors interested in supporting climate change might consider, i.e. providing contraception and female education.
1. How Cool Earth works -- intro
Cool Earth is a non-profit that tackles climate change by working with rainforest communities to avoid deforestation. They do this by supporting the livelihoods of the people in the rainforest.
The logic is that local communities in the rainforest will be less likely to permit logging if they are not in need of money, since they fundamentally want to preserve the rainforest as well.
This results in rainforest being preserved, which is positive from a climate change perspective.
2. Why is it considered a high-impact choice of charity to donate to?
According to analysis by Giving What We Can (an Effective Altruist organisation which used to do research into charities) Cool Earth was considered a high-impact choice.
The rationale is that the Cost per tonne of CO2 is low compared to other opportunities, i.e.
- Cost per tonne of CO2 for Cool Earth is ~ $1
- Cost per tonne of CO2 for other opportunities is typically more expensive, e.g. ~ $10 (esp direct work, not campaigning)
See Appendix 2 for more about GWWC’s analysis.
My main claim is that GWWC’s claim of c. $1 per tonne of CO2 claim is optimistic. Indeed Cool Earth’s overall impact from a climate change perspective may be negligible, although this depends on various factors.
The GWWC analysis was last updated in 2016, and GWWC no longer conducts analysis of charities, so GWWC were not available to comment on this piece.
3. Reasons to doubt Cool Earth’s impact
- 3a Reason to doubt 1: Displacement risk
- 3b Reason to doubt 2: Risks post-exit
- 3c Polarisation: how calcs lead to all-or-nothing views
3a Reasons to doubt 1: Displacement risk
- Local communities (e.g. the Ashaninka in Peru) will be less likely to permit logging if they are not desperate for money
- Cool Earth helps them to be not desperate for money
- So the loggers will be unlikely to perform logging in *those communities*
- However they would probably just go somewhere else and log there instead.
If we were to model this as saying that loggers would *definitely* go and log somewhere else, then the model should assume that Cool Earth has achieved nothing from a climate change perspective. While it's unlikely that 100% of loggers would "be displaced" (i.e. just log elsewhere) I suspect it applies to most loggers.
In the appendices, I have considered factors which I thought might mitigate this concern, which I call:
- exogenous-logger-gives-up scenario
- long-term-broad-coverage scenario and the
- endogenous-logger-would-be-material scenario.
Having looked into each of those scenarios (see appendices for more), it seems that their effect is small in Peru and Papua New Guinea, so the displacement effect is likely to dominate. I.e. the displacement effect is likely to mean that Cool Earth is having a small amount of impact on climate change. (See appendix A3(ii) for more on this)
This might not be the case for all the countries where Cool Earth operates -- more on this in the next section (section 4).
I have been in touch with Cool Earth, and they have been kind enough to give me some time to discuss these concerns. They disagree with this assessment, especially around the endogenous-logger-is-material scenario. More info in the appendices, especially appendix A3(iii), including why I'm not convinced by their arguments.
3b Reasons to doubt 2: Risks post-exit
Cool Earth’s model involves supporting a community for some years and then leaving and moving on. Once that’s happened, will the support that Cool Earth provides still persist?
By including a probability of logging of only 0.5% with Cool Earth’s intervention (see the summary of GWWC’s analysis), we are essentially placing a high probability that the intervention will work *not only now but in the future*. If in fact the effects are not permanent, then the cost-effectiveness model should be different from the one used by GWWC. Instead it should take into account the expectation that logging will restart, but model it as happening at some point in the future, and discount that back to the present day. Alternatively we could adjust the model by increasing the costs to reflect the fact that Cool Earth will be required to work with those indefinitely, rather than just for a limited period of time as originally assumed.
There are actually not very many partnerships that Cool Earth has exited from (only 4) and even those are not good examples because they were imposed by external circumstances. Although the little data we have about those seem positive, we don’t have enough evidence to know whether the preventative effects will persist after Cool Earth’s work has finished.
Quote from Cool Earth on this topic:
“At the very beginning of Cool Earth's existence ten year's (sic) ago we worked in Ecuador and Brazil briefly but had to end these early due to political issues beyond our control. For one of the Brazilian communities we worked with for two years, back in 2007, we have satellite images that show the forest to still be intact. This is a good sign that the communities remained resilient but of course we need many more data points from current and future partnerships to be able to provide really strong evidence for the long-term effectiveness. This is something we are putting together now by building a portfolio of partnerships globally which will test how much our support creates resilient communities and how this needs tweaking under different circumstances and slightly different methods (eg direct funding to households instead of to community associations) to improve its effectiveness. ”
In order to get to a clearer picture on this, I would need to better understand the assumptions behind the costings. I have requested this Cool Earth; they have not been able to provide this information.
3c Polarisation: how the calcs lead to all-or-nothing views
One of the quirks of the way the calculations work is that we are likely to believe either that Cool Earth’s work is comfortably more cost-effective than tree-planting or that it has minimal impact. There is only a very small range of input assumptions which lead to conclusions in between. This point is slightly technical, so I have expanded on it in an appendix (see appendix 5).
4 Impactfulness varies by country
The details differ between the different countries where Cool Earth operates. Cool Earth have been kind enough to discuss this with me, although we have not had time to cover everything.
Displacement risk is a key driver of whether Cool Earth has some impact.
(More notes on the “marginal impact of the next pound” in Appendix 6.)
Papua New Guinea: I think it’s unlikely that a palm oil company would be put off from doing their work just because an area can no longer be logged, so it seems the displacement risk is high. Note that Cool Earth argue that there is a potential systemic impact, however I don’t find this convincing. More on this in Appendix 4
Mozambique/Cameroon: These are new countries that Cool Earth is expanding into. Cool Earth has informed me that the logging that they are tackling is mostly endogenous -- if true, this would suggest that the cost-effectiveness is good. I would have liked to have investigated this further, however our conversations ran out of time and I was unable to dig into this deeper, which is unfortunate.
Peru: Cool Earth has suggested that the logging would be endogenous to a material degree. This seems unlikely to be the case, in my view. See Appendix 3(iii) for more detail.
Democratic Republic of Congo: Cool Earth has informed me that the logging that they are tackling is mostly endogenous -- if true, this would suggest that the cost-effectiveness is good. I would have liked to have investigated this further, however our conversations ran out of time and I was unable to dig into this deeper, which is unfortunate.
5 Other options for donors to consider
The purpose of this piece is not to provide the full answer to the question of how to donate if you are keen on tackling climate change. However here are a few (not necessarily thoroughly researched) ideas:
- Founders Pledge has given this question plenty of thought and has some recommendations (link)
- Providing contraception for free to people who otherwise don’t have access to it could be competitive with Cool Earth’s cost-effectiveness, even under GWWC’s assessment of Cool Earth. More on this in Appendix 7.
- Funding education for girls has been proposed by drawdown.org as a high-impact choice. Some of the arguments made in Appendix 7 would apply here too. I don’t know how this would stack up in terms of cost-effectiveness, although I’d guess it would likely be on the same order of magnitude as providing contraception, with similar non-climate-change benefits.
The main purpose of mentioning these is to indicate that there *are* other potentially high-impact opportunities for climate-interested donors (i.e. it’s not that we are looking at Cool Earth because there are no other promising donation candidates)
Cool Earth’s work is either fairly low impact, or there is (currently) not enough evidence available to say; this depends on which project (i.e. country) you’re looking at.
For the countries for which we don’t have enough evidence, it may be that more evidence would show Cool Earth to be more cost-effective. While Cool Earth staff have been very kind in giving me their time, there has been a limit to how time they have been able to provide -- these conclusions may evolve with more information.
Based on the information as it stands, I would be more inclined to recommend that donors not donate until more analysis is done. I also note that there are several other promising interventions that could be at least as cost-effective as Cool Earth’s work.
There now follow some appendices.
Appendix 1: More background info about loggers and logging
Cool Earth operates in 3 locations:
2 partnerships in Peru
2 partnerships in Papua New Guinea
1 partnership in Democratic Republic of Congo
They plan to expand to Mozambique and Cameroon
In Peru, the loggers are mostly small-scale loggers, possibly one-man-bands, not necessarily operating legally.
(Side note: if loggers are willing to operate outside of the law, it’s possible that they would be willing to log even though the local community doesn’t permit it. I have not investigated this in any detail, however I understand that Cool Earth does careful monitoring of the rainforest in the areas where they work, which gives some cause for reassurance about this risk.)
In Papua New Guinea, the loggers are more likely to be large multinational companies.
(For DRC I have not checked -- it is unfortunate that I've not been able to get more time with Cool Earth)
Appendix 2: Summary of GWWC’s $1 per tonne of CO2 analysis
The below table summarises the analysis done by Giving What We Can (GWWC). You can see the full sources for Giving What We Can’s (GWWC’s) analysis here: https://www.givingwhatwecan.org/report/cool-earth/
Appendix 3: Displacement risk -- more details
Displacement risk is the risk that someone intending to log in a given area is rebuffed from that area because of Cool Earth’s work, but then the logger simply goes elsewhere.
At first glance, I think that the most likely scenario is that most loggers are not put off (i.e. they are simply displaced and perform logging elsewhere), largely because they have already travelled a long distance to get there, so to expect them to be put off by the thought of travelling just a little further seems surprising.
However I consider here some possible mitigating factors:
- A3(i) Exogenous-logger-gives-up scenario: The logger may have attempted logging somewhat opportunistically, and once the nearest/easiest logging opportunity has been rebuffed, they simply choose to do something else instead of logging.
- A3(ii) Long-term broad coverage scenario: The immediate impact of logging may be that the logger just goes elsewhere. However if Cool Earth were to continue to expand enough, eventually there would be so few areas left, that the logger may be forced to give up on logging. If that occurs, it seems fair to give some credit now to that future possibility.
- A3(iii) Endogenous-logger-would-be-material scenario: By “endogenous loggers” I mean people from within the community that Cool Earth supports. If Cool Earth has materially reduced the number of endogenous loggers, it would suggest that Cool Earth’s intervention is impactful, because those people are less likely to go elsewhere.
A3(i) Logger-gives-up scenario
For our analysis, we need an estimate of the probability of the logger-gives-up scenario. I have tried discussing this with Cool Earth, however I’m conscious that they are experts on the communities that they support, and not experts on the communities who do the logging. So they are not best placed to answer this question.
However I’m conscious that the logger has travelled some distance to perform the logging. For example, in Peru the loggers are typically small-scale loggers who may have travelled from the Andean region -- i.e. potentially a long distance across the width of the country -- in order to reach the forests. In Papua New Guinea, the loggers are large, serious logging companies. Neither of these groups are likely to be casual/opportunistic about the intent to log.
I would be inclined to assume that this sort of giving up doesn’t happen to any material extent, although if there was evidence that this was an (at least) somewhat common occurrence, it might change the overall analysis. Similarly if we believed that loggers who have travelled a long distance, are forced to travel a bit further, and are therefore disinclined to travel again and more inclined to seek income from elsewhere, this would also change the analysis.
A3(ii) Long-term broad coverage scenario
If Cool Earth were to continue to expand enough, eventually there would be so few areas left where logging was possible that the logger may be forced to give up on logging. How far away are we from that?
The below bullet points indicate why I think that we are very far away from that. Hence considers the long-term broad coverage scenario to be far away enough that it gives it negligible credit.
- The loggers are typically coming from the Andes and travelling several hundred km to get to there
- So the amount of extra travel needed would perhaps need to be several hundred more kilometres before it deters a logger
- One community will typically occupy around 100 acres of land
- This might require around 600m to traverse (or possibly a bit less)
- So they would need to add another several hundred communities
- They are currently covering 2 communities in Peru
- Papua New Guinea
- The loggers are typically large multinational logging companies
- So if it became harder to access forest in one location, the company would simply look elsewhere
- Democratic Republic of Congo
- Cool Earth has one partnership in DRC. I have not considered DRC closely in this analysis
A3(iii) Endogenous logger would be material scenario
Currently there is very little “endogenous” logging (i.e. logging by people from within the community which Cool Earth supports) -- it accounts for around 3% of the logging which occurs, according to Cool Earth.
The key question is whether endogenous logging would be material if Cool Earth weren’t working in those communities -- if so, then the displacement risk is less of a concern.
From my conversations with Cool Earth I understand the following:
- They believe that endogenous logging would be material without them
- They were unable to provide data about the extent to which endogenous logging occurs in other nearby communities
- They were able to give an anecdote about another community who logged 25% of the land around them
This has not been enough to convince me that endogenous loggers are material. In particular:
- Cool Earth works with communities who are particularly prone to value the rainforest, this suggests that endogenous logging is likely to be low there. Indeed they have argued that the community’s desire to preserve the rainforest is part of the reason for their success.
- If the communities would have logged 25% of the rainforest around them without Cool Earth’s work, this would be more than enough to push Cool Earth into cost-effective territory; however I don’t believe the community that did this was typical.
- In my conversations with Cool Earth, they have referenced some more data about the deforestation that’s been occurring both within and outside of Cool-Earth-supported communities, and this might help to shed some light on this. At the time of writing, Cool Earth have not had time to share this with me.
Appendix 4: More on Cool Earth’s opinion on Papua New Guinea
Cool Earth have suggested that the work in Papua New Guinea is still valuable, even though displacement risk is high for the loggers. They have noted that a major palm oil company -- New Britain -- has decided that they will stop logging and only create palm oil on existing sites rather than perpetrate further deforestation. They believe that Cool Earth has a valuable role to play in achieving outcomes like these.
I am not so sympathetic to this claim. Cool Earth is not a campaigning organisation, and does not allocate a material amount of its resource to influencing governments or corporates. I suspect that New Britain would have made the same decision had Cool Earth never existed, and that Cool Earth’s continued existence is unlikely to influence other such palm oil companies.
Appendix 5: Polarisation -- why Cool Earth is likely highly cost-effective or not at all
Appendix 2 sets out the model used by Giving What We Can (GWWC) when they did their analysis. This post has focused on how much reduction in overall probability of logging is actually generated by Cool Earth’s work, in other words:
- if the probability of logging without Cool Earth is 28% (line (B) from Appendix 2)
- how much does that probability reduce to with Cool Earth? (line (C) from Appendix 2)
The below chart shows how the cost per tonne of CO2 averted varies with assumption (C) from Appendix 2, while keeping all other assumptions fixed.
GWWC’s assumption was that the probability of logging *with* Cool Earth doing their work was 0.5%. This means that almost any reduction in the scale of Cool Earth’s work (e.g. instead of bringing the probability of logging down to 0.5%, let’s say you thought it was actually 10%) would still leave Cool Earth more cost-effective than a straightforward tree-planting scheme. In other words, for most of the chart, the green line is lower than (i.e. more cost-effective than) the blue line.
It’s only if Cool Earth’s work is having very little impact that we get a cost-effectiveness which is much worse than tree-planting.
There is still a zone where Cool Earth’s cost-effectiveness happens to be close to that of tree-planting, however this is a narrow window.
Note that I’ve included here a comparison with tree-planting mostly because this is an easy comparison to make -- it’s not because I expect tree-planting to be the best climate change intervention outside of Cool Earth.
Appendix 6: What is the marginal impact of the next pound donated?
At the margin, the next pound donated to a charity is typically likely to support the following types of work:
-- those projects which the charity is seeking to grow or get off the ground (i.e. which are somehow “expanding” beyond what they are currently doing)
-- those projects which are less “core” to the charity (and therefore more likely to be at risk if money were short)
These are subtle things, and often not easy to determine. In the case of Cool Earth, because they are expanding in certain areas (Papua New Guinea, Mozambique and Cameroon) these are likely to be good candidates for the marginal impact of the next pound.
If we happened to know that the impact of the next pound was to fund work in Papua New Guinea, say, then the impact analysis on (say) Peru would be almost irrelevant. It would not be entirely irrelevant because it says something about the approach of the organisation being funded. And there is always some chance that the funds could end up supporting (say) Peru, or indeed anything else Cool Earth does.
Appendix 7: Providing contraception
This is a very shallow review, and needs more work before it could be considered definitive. However here are some rough and ready calcs to indicate that the cost per tonne of CO2eq averted is probably only around $1:
[EDIT: Some commenters have argued that the post would be better without this section. They are probably right. The spirit of this section is meant to be: "if we keep on thinking creatively about climate change we will probably keep finding creative solutions -- I don't know if contraception is really a good idea, but I'm just throwing an idea out to illustrate this." I'm keeping this section in anyway.]
Cost per pregnancy averted?
- Cost to fully meet the need for modern contraceptives: $9.4 billion [A]
- Number of unwanted pregnancies per year: 74 million [B]
- Cost per pregnancy averted ~ $100 (= (1) / (2) with some rounding)
Average CO2 footprint per person?
- Total CO2 emissions = c 53 billion tonnes of CO2eq per annum [C]
- Total world population = c 7 billion
- Average CO2 footprint = c 7.5 tonnes per person p.a. (= (4)/(5) with some rounding)
- Average CO2 footprint for someone in the developed world = 10 tonnes per person p.a. [D]
- Average CO2 footprint for someone in the developing world ~ 5 tonnes per person p.a. (rough estimate, see [E])
Cost per tonne of CO2eq averted?
- Lifetime CO2eq footprint ~ 100 tonnes per person (roughly calculated, see [F])
- Cost per tonne of CO2eq averted ~ $1 per tonne of CO2eq (= (3)/(9))
[A] source: Guttmacher Institute, p5 of https://www.guttmacher.org/sites/default/files/report_pdf/addingitup2014.pdf . The source did not say that this was a per annum cost, but I’m guessing it probably is.
[B] source: Pathfinder International: “In developing countries, more than half of all women of reproductive age want to avoid pregnancy. 214 million women are not using an effective method of contraception. And millions of women lack the services they need, leading to 74 million unintended pregnancies in developing countries each year.” https://www.pathfinder.org/focus-areas/contraception-family-planning/
[C] World bank gives a figure of c 53,526,300,000 tonnes as at 2012 https://data.worldbank.org/indicator/EN.ATM.GHGT.KT.CE
[D] Online source says: “The average person's carbon footprint is about 10 tonnes” https://www.carboncalculator.co.uk/averages.php
[E] I didn’t find a source for the average carbon footprint for someone who is living in a context where contraception is not readily available and needs to be provided by a charity. However as a rough guess I just chose 5 tonnes because the average of 10 tonnes and 5 tonnes comes to the desired 7.5 tonnes -- this does not weight correctly for population sizes, but I think this will probably mean I’m understating the carbon footprint.
[F] The simplistic way to calculate the lifetime carbon footprint is to multiply the annual carbon footprint by the life expectancy in years, which would be c. 50 years for countries with the shortest life expectancies. However we should discount the future years because there is uncertainty about the future (e.g. maybe we will have solved climate change in 30 years’ time; or maybe we will have caused a climate catastrophe and wiped out humanity in 30 years’ time). So to account for this I’ve used a discount rate of 5%, and to get the right discounted present value would involve dividing by 5% (i.e. multiplying by 20) assuming people live in perpetuity, which would give a similar number to assuming that people live for 50 years.
- This is based on the cost to fully meet the contraceptive need -- this may include more expensive “last mile” contraceptive needs and the best use of funds at the margin might be more cost-effective (although I haven’t checked how the $9.4 bn figure was calculated, so I don’t know if that’s correct)
- Calculations of this sort often ignore an important factor: each person adds a demand on the world’s resources, but also brings a small (maybe only 0.00001% or whatever) chance that the person will create a phenomenal breakthrough which will solve the problem. As an example, many have worried over the years about having enough food to feed a growing population. However if we had limited population growth as a result of this, we might have never had Norman Borlaug, and never had the Green Revolution. That said, for people born in the developing world, it’s likely that the best way to unlock their potential to help the world is probably to have fewer of them and make sure they are well educated rather than having more of them in countries with still-developing education infrastructures.
- A separate study has found that if you met the unmet need for contraceptives (no coercion whatsoever) then you could achieve large scale abatement at less than a dollar a ton. Source: https://www.vox.com/energy-and-environment/2018/11/16/18096352/climate-change-clean-energy-policies-guide. Note that I had seen this quote before I put together my numbers, so I probably sub-consciously tweaked my choice of discount rate to agree to this. (I.e. if you think that two independent assessments have come to the same figure, it’s probably better to say that they are not entirely independent)
- Note that this gives no credit to the many other benefits of the intervention, notably reducing maternal mortality, enabling female empowerment, and other sustainability benefits.
Many thanks to John Halstead and Johannes Ackva who reviewed earlier drafts of this post.
Comments sorted by top scores.