Posts

Growth and the case against randomista development 2020-01-16T10:11:51.136Z · score: 268 (118 votes)
Dataset of Trillion Dollar figures 2020-01-13T13:33:25.067Z · score: 37 (19 votes)
Let’s Fund: annual review / fundraising / hiring / AMA 2019-12-31T14:54:35.968Z · score: 37 (12 votes)
[updated] Global development interventions are generally more effective than Climate change interventions 2019-10-02T08:36:27.444Z · score: 100 (48 votes)
New popular science book on x-risks: "End Times" 2019-10-01T07:18:10.789Z · score: 16 (9 votes)
Corporate Global Catastrophic Risks (C-GCRs) 2019-06-30T16:53:31.350Z · score: 55 (25 votes)
Crowdfunding for Effective Climate Policy 2019-05-25T18:17:05.070Z · score: 68 (32 votes)
Nick Bostrom on Sam Harris' podcast 2019-03-19T11:21:09.483Z · score: 17 (9 votes)
[EAGx Talk] Considerations for Fundraising in Effective Altruism 2019-01-15T11:20:46.237Z · score: 14 (5 votes)
EA orgs are trying to fundraise ~$10m - $16m 2019-01-06T13:51:03.483Z · score: 50 (22 votes)
New web app for calibration training funded by the Open Philanthropy Project 2018-12-15T15:18:54.905Z · score: 17 (8 votes)
Impact investing is only a good idea in specific circumstances 2018-12-06T12:13:46.544Z · score: 76 (38 votes)
Effective Altruism in non-high-income countries 2018-11-15T17:18:42.761Z · score: 37 (21 votes)
“The Vulnerable World Hypothesis” (Nick Bostrom’s new paper) 2018-11-09T11:20:42.330Z · score: 22 (10 votes)
Why donate to meta-research? 2018-11-08T09:29:58.740Z · score: 17 (5 votes)
[link] Why donate to (scientific) research? 2018-10-29T11:13:25.026Z · score: 7 (2 votes)
Announcing: "Lets-Fund.org: High-Impact Crowdfunding campaigns" & "Let's Fund #1: A (small) scientific Revolution" 2018-10-25T21:22:14.605Z · score: 39 (31 votes)
A generalized strategy of ‘mission hedging’: investing in 'evil' to do more good 2018-02-18T17:41:31.873Z · score: 23 (15 votes)
69 things that might be pretty effective to fund 2018-01-21T22:47:32.094Z · score: 34 (30 votes)
Some objections and counter arguments against global poverty/health interventions 2015-08-05T09:44:11.863Z · score: 9 (5 votes)
Giving What We Can's response to recent deworming studies 2015-07-23T18:19:59.535Z · score: 9 (9 votes)
Long-lasting insecticide treated nets: $3,340 per life saved, $100 per DALY averted. How is this calculated? 2015-07-13T16:08:20.169Z · score: 9 (5 votes)
An update on Project Healthy Children 2015-06-08T13:36:16.414Z · score: 7 (3 votes)
Room for more funding: Why doesn’t the Gates foundation just close the funding gap of AMF and SCI? 2015-06-03T14:48:07.317Z · score: 4 (4 votes)
Feedback and $2k in funding needed for EA essay competition 2015-05-13T15:13:29.362Z · score: 15 (11 votes)

Comments

Comment by haukehillebrandt on Are there good EA projects for helping with COVID-19? · 2020-03-04T08:47:49.249Z · score: 10 (4 votes) · EA · GW

Found this paper: "Optimizing respiratory management in resource-limited settings"
"Mechanical ventilation is an expensive intervention associated with considerable mortality and a high rate of iatrogenic complications in many LMICs. Recent case series report crude mortality rates for ventilated patients of between 36 and 72%. Measures to avert the need for invasive mechanical ventilation in LMICs are showing promise: bubble continuous positive airway pressure has been demonstrated to decrease mortality in children with acute respiratory failure and trials suggest that noninvasive ventilation can be conducted safely in settings where resources are low." ... "One of the most significant developments in acute care research in LMICs in recent years has been the publication of three trials demonstrating that continuous positive airway pressure (CPAP) can reduce mortality in children under 5 years of age, compared with oxygen delivered via standard low-flow nasal cannula [35▪,36,37▪]. CPAP can also decrease the need for invasive mechanical ventilation [38▪▪]. There are three main ways to generate CPAP: first, by using a pressure driver or a ventilator; second, using high flow nasal-cannula oxygen therapy (HFNC); or third, by submerging the expiratory limb of a breathing circuit in water to create so-called bubble CPAP. Traditionally bubble CPAP circuits also contain a driver, although some newer iterations only use the oxygen/air flow from an oxygen concentrator to generate CPAP [39].

All three trials used bubble CPAP as the intervention and together showed a risk ratio of survival of 0.58 [95% confidence interval (CI) 0.41–0.82] [38▪▪]. One study had an additional intervention arm using HFNC, but no conclusions were drawn regarding its efficacy as the study was terminated early due to increased mortality in the control group.

Nasal cannulae, used as the patient interface in all three trials, are an attractive option for understaffed environments because they generally require lower levels of nursing supervision to use safely [39]. The basic circuits and simplified care protocols meant that the equipment required few adjustments, especially when compared with invasive mechanical ventilation.

There are elements of each of these studies that epitomize context-appropriate innovation and research. The bubble CPAP circuit deployed in the Bangladesh study was fashioned out of readily available, cheap equipment (standard nasal cannula, a shampoo bottle and intravenous fluid tubing) so the cost of the circuit was approximately $3 per patient [35▪]. They used an oxygen concentrator and no driver in the circuit with additional cost savings." https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6319564/

here’s a build diagram for a bubble cpap

https://www.edjones.org/articles/bubble-cpap-in-resource-poor-settings/?fbclid=IwAR05oxQ2tPg1LDD6o73cBWOGKukaYBq8APcFpNmB1y900nPovTwV0yFBWBQ

are studies indicating it’s helpful for adults (despite its primary use in infants)

https://intjem.biomedcentral.com/articles/10.1186/s12245-019-0224-0

Comment by haukehillebrandt on Growth and the case against randomista development · 2020-02-17T10:01:20.766Z · score: 5 (3 votes) · EA · GW

Great comment - strong upvote! :)

Comment by haukehillebrandt on Growth and the case against randomista development · 2020-01-28T10:03:47.439Z · score: 2 (1 votes) · EA · GW

Yes, I list a couple more quotes from this article, but also many quotes from their Duflo and Banerjee's book in the Appendix. I think these quotes really get to the crux of the disagreement between growth and randomista development.

From their Foreign affairs article:

“There are many ways to improve allocation, from the moves away from collectivized agriculture that China made under Deng to the efforts India made in the 1990s to speed the resolution of debt disputes and thus make credit markets more efficient. But the flip side to this is that at a certain point, the gains start to diminish. Many developing economies are now reaching this point.”
“Quality of life means more than just consumption. Although better lives are indeed partly about being able to consume more, most human beings, even the very poor, care about more than that. They want to feel worthy and respected, keep their parents healthy, educate their children, have their voices heard, and follow their dreams. A higher GDP may help the poor achieve many of those things, but it is only one way of doing so, and it is not always the best one.”

Quotes from Duflo and Banerjee

From “Good Economics for Hard Times”

“How would anyone know whether pre-1991 growth would have continued had there been no crisis and the trade barriers not been brought down in 1991? To complicate matters, trade was being liberalized gradually starting in the 1980s; 1991 just sped that up (a lot). Was the big bang necessary? We will never know unless we are allowed to rewind history and let it go down the other path. Unsurprisingly, however, economists find it very hard to let go of this sort of question.”
CHASING THE GROWTH MIRAGE
“Unfortunately, just as we don’t know much about how to make growth happen, we know very little about why some countries get stuck but others don’t—why South Korea kept growing but Mexico did not—or how one gets out."
“Perhaps the reason why some countries, like China, can grow so fast for so long is that they start with a lot of poorly used talent and resources that can then be harnessed”
“Solow’s was what economists call an exogenous growth model, where the word “exogenous,” meaning driven by outside effects or forces, acknowledges our inability to do anything about the long-run growth rate. Growth, in short, is beyond our control.”
“The growth tide does raise all boats, but it doesn’t lift all boats to the same level—many economists worry that there may be such a thing as the middle-income trap, an intermediate-level GDP where countries get stuck or nearly stuck. According to the World Bank, of 101 middle-income economies in 1960, only 13 had become high income by 2008.122 Malaysia, Thailand, Egypt, Mexico, and Peru all seem to have trouble moving up. “
“In Indian manufacturing there was a sharp acceleration in technology upgrading at the plant level, and some reallocation toward the best firms within each industry after 2002. This appears to be unrelated to any economic policy, and is described as “India’s mysterious manufacturing miracle.”121 But it is no miracle. At its root, it is a modest improvement from a dismal starting point, and one can imagine various reasons it happened. Perhaps a generational shift, as control passed from the parents to their children, often educated abroad, more ambitious, and savvier about technology and world markets. Or the effect of the accumulation of modest profits that eventually made it possible to pay for the shift to bigger and better plants.”
“One very real danger is that in trying to hold on to fast growth, India (and other countries facing sharply slowing growth) will veer toward policies that hurt the poor now in the name of future growth. The need to be “business friendly” to preserve growth may be interpreted, as it was in the US and UK in the Reagan-Thatcher era, as open season for all kinds of anti-poor, pro-rich policies (such as bailouts for over indebted corporations and wealthy individuals) that enrich the top earners at the cost of everyone else, and do nothing for growth. If the US and UK experience is any guide, asking the poor to tighten their belts, in the hope that giveaways to the rich will eventually trickle down, does nothing for growth and even less for the poor. If anything, the explosion of inequality in an economy no longer growing has the risk of being very bad news for growth, because the political backlash leads to the election of populist leaders touting miracle solutions that rarely work and often lead to Venezuela-style disasters. Interestingly, even the IMF, so long the bastion of growth-first orthodoxy, now recognizes that sacrificing the poor to promote growth was bad policy. It now requires its country teams to include inequality in factors to take into consideration when providing policy guidance to countries and outlining conditions under which they can receive IMF assistance.123
“The bottom line is that despite the best efforts of generations of economists, the deep mechanisms of persistent economic growth remain elusive. No one knows if growth will pick up again in rich countries, or what to do to make it more likely. The good news is that we do have things to do in the meantime; there is a lot that both poor and rich countries could do to get rid of the most egregious sources of waste in their economies. While these things may not propel countries to permanently faster growth, they could dramatically improve the welfare of their citizens. Moreover, while we do not know when the growth locomotive will start, if and when it does, the poor will be more likely to hop onto that train if they are in decent health, can read and write, and can think beyond their immediate circumstances. It may not be an accident that many of the winners of globalization were ex-communist countries that had invested heavily in the human capital of their populations in the communist years (China, Vietnam) or countries threatened with communism that had pursued similar policies for that reason (Taiwan, South Korea). The best bet, therefore, for a country like India is to attempt to do things that can make the quality of life better for its citizens with the resources it already has: improving education, health, and the functioning of the courts and the banks, and building better infrastructure (better roads and more livable cities, for example).”
On climate: “Mitigation through better technologies may not do the trick; people’s consumption will need to fall. We may have to be content not only with cleaner cars but also with smaller cars, or no cars at all.”
Comment by haukehillebrandt on Growth and the case against randomista development · 2020-01-26T10:59:21.068Z · score: 2 (1 votes) · EA · GW

Excellent comment - strongly upvoted for engaging with the data.

how did you calculate the median figure for Vietnam that you reference in section 4 ($6,914 GDP per capita)?

The sheet where we calculated the median growth episode within the spreadsheet is here:

https://docs.google.com/spreadsheets/d/1VcQ2r5zuCztd1_2vRscK8UOEAiQqhvvhkJVfagCzpqQ/edit#gid=1331750623&range=D26

Source: Pritchett, Labor p23

Vietnam was just the median of these selected growth episodes- because Pritchett in his example uses quite a big growth episode. Pritchett calculates the NPV gain from growth acceleration per person from this median case as $6,914. This is for illustrative purposes, picking Vietnam has no special significance here. "To be affected by a think tank" also has no special significance, we didn't check whether this growth episode was likely affected by a think tank.

These are selected by Pritchett:

"These are a list selected the largest episodes of growth acceleration. Source: Selected episodes. Author’s estimates from estimates in Pritchett, Sen, Kar, and Raihan 2016"

so... re your question:

When I look at the those figures in Appendix A, though, it seems like the median growth episode calculated using PRM (without reference to dollar size) is somewhere around Ecuador's negative growth in 1978, which doesn't seem like it would line up even with the conversion to $PPP.

Yes, this is likely largely due to Vietnam having a roughly ~10x higher population and being 10x poorer back then.

I think it is okay to use, as Pritchett does, these selected growth episodes, because if one wants to maximize effectiveness using policy one can strategically only look at big poor countries. One could further look at only those countries where growth is sluggish and perhaps where economic policy is particularly bad.

I write about this in the appendix:

Because effective altruism often tries to focus on the poorest countries, where a dollar goes 100x further than in rich countries, there is perhaps most hope for growth diagnostics.
So perhaps Duflo is right in that “Growth is likely to slow, at least in China and India, and there may be very little that anyone can do about it.” And this is actually born out in China’s and India’s performance on the World Bank’s Doing Business indicators, where they score 63th and 31st out of 189 countries, though being relatively poor. Thus, there seem no low hanging fruit to improve their economic policy.
However, below I show a table where I multiply population size of every country by their poverty multiplier (i.e. $1 is worth x times more going to this country than to the richest country in the sample. See appendix 2 of this doc for more info). This can then be ordered by the utility created by increasing GDP per capita by $1. India comes out on top because of its large population (1.3bn) and relatively low GDP per capita ($6,574). China comes 3rd, because though it has a large population, it is already relatively rich ($15,531). Recall that the problem is that we might not know how to increase growth in India and China.
However, there are many smaller very poor countries in the top 10 sample such as DRC and Ethiopia - very poor countries with 100 million population. This can then also multiplied further by neglectedness/tractability criteria. For instance, in a country’s ranking on the WB Doing Business ranking divided by GDP. There one can see that, relative to its GDP per capita, China already does quite well on the Doing Business ranking. However, the DRC and Ethiopia do poorly on the doing business ranking, even relative to their GDP. These countries could be most cost-effective for economic policy assistance.
Comment by haukehillebrandt on Growth and the case against randomista development · 2020-01-17T17:25:44.328Z · score: 2 (1 votes) · EA · GW

I think this view would probably be endorsed by many prominent development economists. But I concede that there are also development economists who believe that health and education is very important.

When I first read about Rodrik's theory of development, I updated in the direction that health and education are not that important for growth at least for very poor countries, even though it's quite unintuitive.

From the appendix doc:

Historically, almost all non-poor countries have grown their economies in three steps:
Rural to urban migration: Unskilled (subsistence) farmers migrate to cities and start working in factories. Over night, this increases their productivity many times over.
Manufacturing absorbs vast amounts of unskilled labor: These workers need very little human capital: they do not need to be educated because work in factories is very simple. Population health does not prevent growth either, because there are enough to replace sick workers.
Manufacturing exports niche products to the world market: The factories find their niche product (e.g. initially often garments) and export to the world market, which can absorb large amounts of the same good (e.g. billions of shoes)

Again quoting Weil's review of "Health and growth" (emphasis mine):

As is often the case in economics, the observation that income and health are correlated, is only the beginning of the discussion. Such a correlation can be induced by causation running in either direction, as well as by the effects of some third factor. A priori, there are good reasons to think that all of these are possibilities. People who are healthier can work harder and learn more in school; and where people live longer they will be incentivized to invest more in education.Thus, we would expect better health to cause economic growth. On the other hand, higher income allows individuals or governments to make investments that yield better health. Finally, differences in the quality of institutions (looking across countries), in human capital (looking across individuals), or in the level of technology (looking over time) can induce correlated movements in health and income."

re: Nunn: I'm not ruling out that invariant geographical factors influence economic development by way of health. But it's a different question on whether we can do anything about that by ramping up health spending and ameliorate these differences and whether that's important for growth.

Comment by haukehillebrandt on Growth and the case against randomista development · 2020-01-17T17:05:04.833Z · score: 2 (1 votes) · EA · GW

In my opinion, randomistas do not focus on growth at all, be it level effects or growth effects.

Though to be fair there's this short passage in Duflo's new book on this:

while we do not know when the growth locomotive will start, if and when it does, the poor will be more likely to hop onto that train if they are in decent health, can read and write, and can think beyond their immediate circumstances. It may not be an accident that many of the winners of globalization were ex-communist countries that had invested heavily in the human capital of their populations in the communist years (China, Vietnam) or countries threatened with communism that had pursued similar policies for that reason (Taiwan, South Korea).

Also we do say that "we do not think that the things assessed by RD do not increase economic growth at all: indeed some RD health interventions increase earnings and consumption later in life, and thus do increase growth to an extent. However, evaluating whether the effect size is trivial or not should be a top priority for proponents of RD."

Comment by haukehillebrandt on Growth and the case against randomista development · 2020-01-17T15:50:39.379Z · score: 24 (9 votes) · EA · GW

Yes, interesting take.

Aside from risk aversion, in the appendix, I list some more cognitive biases that might be at play for why people prefer RCTs.

Relatedly, perhaps people sympathetic to long-termism might believe that speeding up growth might speed up GCRs from emerging technologies. And while it is unclear when growth will speed up x-risk at all (see for instance), I think that when it comes to differential technological development, not all growth is equal.

What speeds up risks from emerging technologies is mostly growth in highly technical sectors in high-income countries. Growth in low-income countries will not increase world growth much and is less likely to cause risks from emerging technologies.

Put simply: Burundi’s catch-up growth won’t speed up global growth by much, is unlikely to speed up risks from AI or bio any time soon. Growth has been argued to lead to “Greater opportunity, tolerance of diversity, social mobility, commitment to fairness, and dedication to democracy.” Perhaps growth in poor countries will actually increase stability and thus be good from a differential technological development point.

Lower skilled labor also competes with AI R&D and so increasing trade and migration decrease AI R&D (see “Why Are [Silicon Valley] Geniuses Destroying Jobs in Uganda?”.

But even if growth in poor countries will slightly increase x-risks, then it might still be optimal to support it and offset the x-risk increase through targeted interventions to decrease x-risks. This is because multiobjective optimization for both x-risk reduction and global poverty is likely harder than single objective optimization for the most effective interventions in each category separately.

Comment by haukehillebrandt on Growth and the case against randomista development · 2020-01-17T15:37:12.239Z · score: 7 (5 votes) · EA · GW

The paper we cited is a comprehensive recent meta-analysis on the topic of health and growth that synthesizes the literature on this topic.

The paper concludes:

“If improving health leads to growth, this would be a reason, beyond the welfare gain from better health itself, that governments might want to make such investments. However, the evidence for such an effect of health on growth is relatively weak. Cross-country empirical analyses that find large effects for this causal channel tend to have serious identification problems. The few studies that use better identification find small or even negative effects. Theoretical and empirical analyses of the individual causal channels by which health should raise growth find positive effects, but again these tend to be fairly small. Putting the different channels together into a simulation model shows that potential growth effects of better health are only modest, and arrive with a significant delay.”

We did however acknowledge that this claim is controversial:

Moreover, and more controversially, we do not believe that health interventions (whether directly funded or implemented by the state) are the best way to increase growth in the poorest countries.[15] Here, we want to start a discussion on what the most effective causes of growth are, given its huge importance.

This is a topic of ongoing debate in the literature - future research could look into this topic more and a starting point could be the citation trail from the study above.

Comment by haukehillebrandt on Growth and the case against randomista development · 2020-01-17T15:11:03.395Z · score: 26 (13 votes) · EA · GW

Thanks (strongly upvoted for trying to falsify a central claim). All opinions are mine.

1. While the interesting paper you cite shows that policies bad for growth are at historic lows and argues that much progress has been made, 20% of all countries still have bad policies, and 25% of SSA countries. Given the potential very high effectiveness of growth policy, that we tried to demonstrate in the piece, the value of information of looking into this further is high.

2. I do cite Rodrik in the Appendix who argues that these days, “standard prescriptions” (i.e. Washington Consensus) might not work any longer and we should be skeptical of top-down, comprehensive, universal solutions (though perhaps there are some more generalizable policy prescriptions to be discovered with further research - Rodrik for instance expands the Washington consensus with an additional 10 policy prescriptions).

However, technical assistance by more specialized agencies (e.g. DFID, USAID, GIZ as well as the World Bank’s country offices), and also NGOs such as the International Growth Center, the Copenhagen Consensus, etc. might be able to do “growth diagnostics” to find out where growth is bottlenecked and then help with tailor-made policies on a country-by-country basis.

They might also help with implementation issues, and even indifference issues.

Comment by haukehillebrandt on Growth and the case against randomista development · 2020-01-17T14:24:30.003Z · score: 2 (1 votes) · EA · GW
I'm not sure the 'extreme scepticism' (perhaps we could just call it scepticism?) argument is given a fair shake. Note that answering the question of what causes a country to grow is basically the big question of development economics, and as such it has received considerable attention from economists. In the Duflo and Banarjee piece, they argue that economists did find good low hanging fruit, notably misallocation of resources, but they argue this is reaching a point of diminishing returns. Economists are now struggling to find great opportunities in growth economics, and so there is a good case for looking at different approaches to development. This argument feels plausible to me, and it means you do not have to make the apparently crazy claim that economists never had significant influence on past effective growth policies.

Yes, I steelman this view in the Appendix (my view not necessarily John's):

"Growth is not as neglected as RD, its low-hanging fruit have been picked, and the marginal dollar is not as effective"
“The evidence that macroeconomic policies, price distortions, financial policies, and trade openness have predictable, robust, and systematic effects on national growth rates is quite weak—except possibly in the extremes. Humongous fiscal deficits or autarkic trade policies can stifle economic growth, but moderate amounts of each are associated with widely varying economic outcomes.”
For instance, take the debate over trade liberalization. Recall that there was exceptionally weak global trade growth over recent years. Relatedly to the previous point, some argue that the “low-hanging fruit” of economic liberalization has already been picked. For instance, Weyl argues that in “Radical markets”:
“There is a consensus that the economic gains from further opening international trade in goods is minimal. Studies by the World Bank and prominent trade economists find that eliminating all remaining barriers to international trade in goods would increase global output by only a small amount, 0.3–4.1%. For global investment, the most optimistic estimate in the literature finds a 1.7% increase in global income from the elimination of barriers to capital mobility. Many believe that liberalization of international capital markets has gone too far. Three top IMF economists recently argued that even liberalization that has already taken place has brought limited gains to economies while generating inequality and instability.”

However, there is a debate about this and counterarguments:

Others argue that trade policy is still very relevant. Complete rich-country liberalization would, after a 15- year adjustment, increase income in developing countries by $100 billion per year, which is approximately twice current aid flows.
Also, guarding against protectionism and not losing the growth from trade might be very important: one study suggest that an “increase in tariffs to average bound rates of 44.7 percent in highly protectionist countries such as India, Bangladesh, Pakistan and Sri Lanka would translate into a decline in real income in South Asia by 4.2 percent or welfare losses of close to US$125 billion relative to the baseline by 2020”.

Pritchett too seems much more optimistic about growth diagnostics and believes that while we might not know everything, we generally have a reasonable understanding of what causes growth and can even influence it.

Pritchett has edited a whole volume on growth diagnostics, including on the causes of growth in India.

Generally, my take is that growth diagnostics might get harder the richer a country becomes, by virtue of there being less and less data from other countries on how they developed. Thus, for the poorest countries, growth diagnostics might be easiest because we can draw lessons from all other countries on they developed.

Because effective altruism often tries to focus on the poorest countries, where a dollar goes 100x further than in rich countries, there is perhaps most hope for growth diagnostics.

So perhaps Duflo is right in that “Growth is likely to slow, at least in China and India, and there may be very little that anyone can do about it.” And this is actually born out in China’s and India’s performance on the World Bank’s Doing Business indicators, where they score 63th and 31st out of 189 countries, though being relatively poor. Thus, there seem no low hanging fruit to improve their economic policy.

But in the Appendix I have an analysis where I multiply population size of every country by their poverty multiplier (i.e. $1 is worth x times more going to this country than to the richest country in the sample. See appendix 2 of this doc for more info). This can then be ordered by the utility created by increasing GDP per capita by $1. India comes out on top because of its large population (1.3bn) and relatively low GDP per capita ($6,574). China comes 3rd, because though it has a large population, it is already relatively rich ($15,531). Recall that the problem is that we might not know how to increase growth in India and China.

However, there are many very poor countries in the top 10 sample such as DRC, Bangladesh and Ethiopia - very poor countries with +100 million population. This can then also multiplied further by neglectedness/tractability criteria. For instance, in a country’s ranking on the WB Doing Business ranking divided by GDP. There one can see that, relative to its GDP per capita, China already does quite well on the Doing Business ranking. However, the DRC and Ethiopia do poorly on the doing business ranking, even relative to their GDP. These countries could be most cost-effective for economic policy assistance.

The Copenhagen Consensus Center is actually doing something along the lines of assisting countries / highlighting the need to improve their economic policies. For instance they are helping Bangladesh to improve its economy and prioritize which policies would have the highest social, economic and environmental benefits for every dollar spent. On top of their list is e-procurement across government and land records digitization - related to criteria used to rank countries on the WB Doing Business index.

Comment by haukehillebrandt on Growth and the case against randomista development · 2020-01-17T14:08:41.280Z · score: 8 (2 votes) · EA · GW
Randomista is clearly not a neutral term, and I think constitutes a kind of name calling (e.g. Corbynista in the UK). Do proponents of RCT development use this term for themselves?

We did not use it in a name calling way but rather as a neutral term to describe the intellectual movement. The term is used by mainstream economists who are critical in a respectful way, but also by randomistas themselves (note for instance that Duflo or Blattman have used the term).

However, it is true that

"the term was first used by another Nobel laureate and a long time and fierce critic of RCTs, Angus Deaton. According to Andrew Leigh, the author of a recent book titled, Randomistas: How Radical Researchers Are Changing Our World (2018), the term was meant mostly as an abuse which Leigh turned into a compliment. Leigh defined a Randomista as ‘‘someone who believes we can find answers to important questions by tossing a coin and putting people into a treatment and control group, comparing the outcome, and then using the randomization to get a true causal effect.” (Social Science Space, 2018)."
source
Comment by haukehillebrandt on Dataset of Trillion Dollar figures · 2020-01-17T11:59:42.354Z · score: 4 (2 votes) · EA · GW

Nice, you found another blunder in the literature!

"First, and classically, rating agencies’ fees tend to be high. The revenues of rating agencies come from new ratings and from the reexamination of former ones, as it is very difficult for a company, once it has been rated, to withdraw its rating from the market. It means the operational risk of rating agencies is quite low, just as the volatility of their revenues. We don’t know much about the prices of ratings and the profits of agencies. Nevertheless, in 2011, the operational profit of Standard and Poor’s and Moody’s was about 40 %; and Fitch’s was 31 %. For the first nine months of 2011, the revenue of Standard and Poor’s reached US$ 1.3 trillion for about 1,400 analysts. The figures for Moody’s were US$ 1.2 trillion for 1,300 analysts. These figures make for an annual revenue per analyst higher than US$ 1 million, which is quite high."

from this paper on reforming rating agencies: https://sci-hub.tw/https://link.springer.com/chapter/10.1007/978-3-319-44287-7_12

So this should be billions, not trillions.

I had actually interpreted the figure differently and thought that rating agencies analysts rate trillions in value or something.

Have deleted these from the dataset.

Comment by haukehillebrandt on Growth and the case against randomista development · 2020-01-16T15:33:25.602Z · score: 6 (4 votes) · EA · GW

[37] Pritchett, ‘Randomizing Development: Method or Madness?’ (2019), p. 23-24. see:

https://d101vc9winf8ln.cloudfront.net/documents/32264/original/RCTs_and_the_big_questions_10000words_june30.pdf#page=23

Comment by haukehillebrandt on Growth and the case against randomista development · 2020-01-16T10:26:56.639Z · score: 4 (2 votes) · EA · GW

Thanks - yes, John Halstead and I co-authored this post.

(We'll use the forum's new co-authoring function, this is why I accidentally omitted the authors when first posting, but it will take a little time reflect it, so I've fixed this provisionally).

Comment by haukehillebrandt on Dataset of Trillion Dollar figures · 2020-01-15T17:11:20.004Z · score: 2 (1 votes) · EA · GW

done! thanks for the suggestion

Comment by haukehillebrandt on Dataset of Trillion Dollar figures · 2020-01-15T10:55:44.073Z · score: 6 (1 votes) · EA · GW

Thanks - I fixed the global debt in the non-financial sector figure!

And yes, you're right that notionals need to be interpreted carefully - I initially had a paragraph in my post that notionals should be interpreted carefully, but then I cut it out. Your example is a good one and shows that, in theory, a world with a high notional value of derivatives trading can be one with a stable financial system.

However, I disagree that it is a "totally irrelevant number" and that the in practise notional total volume might be (a not entirely very bad) proxy measure for economic stability.

See Wikipedia:

"To give an idea of the size of the derivative market, The Economist has reported that as of June 2011, the over-the-counter (OTC) derivatives market amounted to approximately $700 trillion, and the size of the market traded on exchanges totaled an additional $83 trillion.[9] For the fourth quarter 2017 the European Securities Market Authority estimated the size of European derivatives market at a size of €660 trillion with 74 million outstanding contracts.[10]

However, these are "notional" values, and some economists say that these aggregated values greatly exaggerate the market value and the true credit risk faced by the parties involved. For example, in 2010, while the aggregate of OTC derivatives exceeded $600 trillion, the value of the market was estimated to be much lower, at $21 trillion. The credit-risk equivalent of the derivative contracts was estimated at $3.3 trillion.[11]

Still, even these scaled-down figures represent huge amounts of money. For perspective, the budget for total expenditure of the United States government during 2012 was $3.5 trillion,[12] and the total current value of the U.S. stock market is an estimated $23 trillion.[13] Meanwhile, the world annual Gross Domestic Product is about $65 trillion.[14]

At least for one type of derivative, Credit Default Swaps (CDS), for which the inherent risk is considered high[by whom?], the higher, nominal value remains relevant. It was this type of derivative that investment magnate Warren Buffett referred to in his famous 2002 speech in which he warned against "financial weapons of mass destruction".[15] CDS notional value in early 2012 amounted to $25.5 trillion, down from $55 trillion in 2008.[16]"

Comment by haukehillebrandt on Let’s Fund: annual review / fundraising / hiring / AMA · 2020-01-10T08:13:10.006Z · score: 11 (3 votes) · EA · GW

Yes, excellent question.

This is a really hard analysis to do because it's very hard to assess what the money would have been spent on counterfactually- see my comment above to Khorton.

My subjective impression is that the $75k for the Better Science campaign was heavily skewed towards EA donors and would have gone to EA causes anyway. However, assuming returns to research this might have still improved the quality of donation within the EA community, which counterintuitively can sometimes be more effective than growing the pie.

However, the $200k raised for the climate change campaign was heavily skewed towards non-EA donor and perhaps the counterfactual here were less effective charities or even conspicuous consumption.

Comment by haukehillebrandt on Let’s Fund: annual review / fundraising / hiring / AMA · 2020-01-10T08:01:58.934Z · score: 2 (1 votes) · EA · GW
In government, we'd consider the money you crowdfunded a cost to society as well.
I'd argue mine is more appropriate for a fundraising charity, but at least I understand the difference now!

Yes, I agree that for all non-profits or public benefit companies a net present value analysis from a societal perspective would be optimal and what we ultimately care about. However, I feel like my analysis is good approximation: of course, the money I crowdfund is a cost to society given the opportunity costs, but the implicit assumption here is that the donor's money would counterfactually be spent on conspicuous consumption or perhaps ineffective charities. If this is the case, then because the value of increasing consumption in advanced economies is comparatively small and the cost-effectiveness analysis is relatively insensitive to whether we count this, then the business analysis is a good approximation for the societal value and it's ok to leave it out for simplicity's sake.

Fundraising charities routinely use "fundraising ratios", which are benefit-cost ratios and similar to net values, so this seems standard practise.

In the future, I might look more into the true counterfactual societal net value and see whether some of the money donated would have gone to similarly effective charities in the future.

I think EAF did a good job at this where they estimated the money they raised that would not have been donated otherwise.


Comment by haukehillebrandt on Let’s Fund: annual review / fundraising / hiring / AMA · 2020-01-02T20:54:22.339Z · score: 2 (1 votes) · EA · GW

Yes, absolutely.

One year ago I've gotten $40k from the EA community, these are the cost to society, because I've spent this money on Let's Fund operations.

Then, on average roughly 1 year later, I've crowdfunded $300k for my campaigns (these are the benefits).

The benefits minus the costs are the also called the net value, which are then simply $260k. But if you discount the costs at 5% with the formula:

=NPV(0.05,(40*-1),300)

then you get a net present value of 234.

Does that make sense?

Comment by haukehillebrandt on Let’s Fund: annual review / fundraising / hiring / AMA · 2020-01-02T19:48:26.870Z · score: 11 (3 votes) · EA · GW

Really interesting questions - thank you!

How confident in your analysis and conclusion do you have to be in order to publish a recommendation? For example, do you believe "better wrong than vague"?

I'm very confident in the conclusions of the research for campaigns and the bar for publication is substantially higher than for what I post on the EA forum. I usually also ask many people to review my research for campaigns (see acknowledgment sections in the reports).

On the EA forum, I sometimes don’t excessively hedge my claims for clarity’s sake. And I have sometimes epistemic status disclaimers which you're referring (better wrong than vague’, “say wrong things”, “Big, if true”, “Strong stances’)

Do you try to caveat to show your degree of confidence?

Yes, I use sensitivity analysis and careful language throughout.

For instance, in my cost-effectiveness analysis I caveat:

"Below we present a very rough, simple, back-on-the-envelope cost-effectiveness analysis ("Fermi estimate”). This model is crude and should not be taken literally. Rather than leaving our assumptions unarticulated and fuzzy, we think it is better to be wrong than vague. By stating assumptions explicitly that can be questioned and falsified (as the common aphorisms in statistics go: "Truth will sooner come out of error than from confusion" and "All models are wrong, but some are useful"). It also helps us think through relevant considerations and formalizes our intuitions. If you disagree with any of the inputs to our model, then you can create a copy of our spreadsheet and plug in your own parameters."

I also use the word "might" about 90 times in the Clean Energy campaign.

But there are some statements even in the report where I'm intentionally wrong for clarity's sake. For instance, when I write:

"The focus of advanced economies like EU countries to prioritize reducing their own domestic emissions is a natural impulse ('clean up your own backyard first'). But 75% of all emissions will come from emerging economies such as China and India by 2040. Only if advanced economies' climate policies reduce emissions in all countries, will we prevent dangerous climate change. We call this the cool rule: only if all countries reduce their emissions will the planet stay cool."

The bolded sentence is clearly wrong on some level, because we can perhaps use geoengineering to cool the planet or maybe emerging economies such as China will solve the issue. However, I feel this is less important to emphasize because it's somewhat unlikely and writing all that out would distract from the central message. By making strong statements such as "Only if" you're making your writing and central claims really clear so that they can be more easily falsified. But some people might disagree and like to hedge more.

How easy would it be to find a demonstrably incorrect statement or paragraph in your work?

I'm quite careful I think but given the length of the report I cannot rule out that there are errors in somewhere. But I'd be somewhat surprised if they were easy to find. So I'll pay a bug bounty of $20 for any statement that is demonstrably incorrect.

However, the central claims I'm very confident in, because I'm trying to triangulate with multiple lines of evidence, so that the conclusions do not depend on a single piece of data (https://blog.givewell.org/2014/06/10/sequence-thinking-vs-cluster-thinking/ ).

Comment by haukehillebrandt on Let’s Fund: annual review / fundraising / hiring / AMA · 2020-01-02T18:58:15.471Z · score: 12 (4 votes) · EA · GW

Excellent comment!

If the recent Bill Gates documentary on Netflix is to be believed, then Gates first became seriously aware of the problem of diarrhea in the developing world thanks to a 1998 column by Nicholas Kristof. It's hard to assess the counterfactual here (would Gates have encountered the issue in a different context? Would he have taken the steps he ultimately did after reading the Kristof piece?) but it seems plausible that Kristof's article constitutes a cost-effective intervention in its own right (if a not particularly targeted one).

To clarify, the win here was not to influence Gates, because he is already very much on board with clean energy innovation agenda (though perhaps if he really read the article then it might be that it might have ever so slightly shifted his views towards the importance of government vs. private R&D which I feel he doesn't focus on enough).

Rather that he has 50mn followers on Twitter and is considered a public intellectual / authority on climate change (publishing a book on climate change in 2020).

But yes your general point is good, because for instance Reid Hoffman and others retweeted the Gates tweet - so perhaps there might be a very small chance of a "Kristof's effect".

I'm curious if you consider the wide propagation of your research in the news media a "risky and very effective" project, and if your research products have been intentionally structured toward this end.

Yes, the research is intentionally structured for wide-ish dissemination. This manifests in several ways:

I feel there are relatively few and only modest downside risks to this project. For instance, there's little information hazard, however there are some moral hazards. See the "Risks, reservations, drawbacks section of the report" where we write:

"Overall, what these quotes have in common are concerns about the moral hazard of spreading a meme like 'breakthrough technology by itself will solve climate change'. Authors repeatedly caution that additional policies—especially carbon taxes—are needed. We think these concerns are warranted, but do not believe that this suggests that clean energy R&D should not be increased. We believe there is consensus amongst even these climate policy scholars that R&D levels must be increased substantially. Crucially, while the moral hazard aspect of clean energy R&D increases might drag out emission reduction, another aspect pushes more strongly in the other direction and make carbon taxes more likely. For instance, one economic model suggests that "if a carbon tax imposes a dollar of cost on the economy, induced innovation will end up reducing that cost to around 70 cents".[82] Given that political acceptability is mainly a function of cost, making clean energy cheaper might make carbon taxes more likely."

The crowdfunding aspect of the campaign means that the campaign and its topic are at least a little bit optimized for being more readily understood by the wider public. This means that there are other harder to explain topics that are perhaps more neglected and thus might be more effective. For instance, in the report in the Section on Climate change is relatively non-neglected, we write:

"Climate change is a high-profile topic that many people work on. It is funded by both governments and big private foundations. Thus, even though clean energy innovation in particular has been relatively underfunded within the climate policy space, it is conceivable that in the future ITIF might receive grants for their clean energy innovation program from other funders, which lowers the counterfactual impact of donating to this project. In other words, comparatively, climate change is not very neglected. For instance, the risks and expected losses of pandemics are of a similar magnitude than those of climate change, yet the area is more neglected by other funders."

Also, there the page is intentionally structured hierarchically going from less to more in-depth, with summaries at the top and then for people who really want to read all the details there's heavily footnoted analysis further down on the page.

If you have some takeaways from your big success so far, it could be very helpful to post them here- widely taken-up tweaks to make research propagate more effectively through the media are marginal improvements with potentially very high value.

Generally with questions about success, there's of course a lot of survivorship bias and a lot of it was perhaps just luck. Similarly, the first step to make research propagate widely is that you need to spend a lot of time and effort researching and editing until it's the research not only really good but also very readable, which requires a lot of resources/privilege.

So perhaps take the following with a grain of salt- your mileage will vary.

If you want your research be covered by the media you of course need a good pitch and get in touch with a lot of relevant journalists (numbers game). You can have it peer-reviewed by people and say so and so has reviewed it and says it's really good/interesting/novel research.

Then to push the coverage of your research you can find influencers for whom your content is highly relevant. I used social proof and had a few select academics and policy wonks I was connected to retweet/endorse the article because it was very much in their field of expertise even if they didn't have very many followers. Then I used this to contact relevant influencers who in the past had tweeted about climate change and had also in the past retweeted Vox articles (aligned political leaning). You can tell them that so and so has already retweeted it as social proof, and ask if they could perhaps also retweet because it's relevant to their audience.

Then there's a technique called power mapping that I used, where you get in touch with people that are connected to even more influential people. You're connected to many people through only very degrees of separation (small world phenomenon), so you perhaps know someone who knows someone who knows an "influencer". You can see who for instance, Obama follows on twitter and then if you get to those people to to get say Obama to retweet the coverage of your research (because it's on reputable site such as Vox).

Sorry if this was a bit rambly, but I hope you get the general idea.

Comment by haukehillebrandt on [updated] Global development interventions are generally more effective than Climate change interventions · 2019-11-23T17:05:59.844Z · score: 2 (1 votes) · EA · GW

Thanks this is a great idea!

I'm really sorry that I don't have more time for this.

Comment by haukehillebrandt on [updated] Global development interventions are generally more effective than Climate change interventions · 2019-11-23T12:31:57.752Z · score: 3 (2 votes) · EA · GW

Thank you so much for this thoughtful comment.

Unfortunately, I won't have the time to give it the time it deserves to engage with it properly (e.g. rework the analysis etc.).

I think you raise interesting points and people looking to extend this work further should take your comment into account.

Comment by haukehillebrandt on [updated] Global development interventions are generally more effective than Climate change interventions · 2019-11-16T17:23:00.492Z · score: 6 (1 votes) · EA · GW

Thanks for this comment and apologies for the delay in replying.

As mentioned I had emailed the authors and they didn't get back to me, and also the author of this paper this paper, "A social cost of carbon for (almost) every country" had apparently trouble replicating their analysis.

But both these papers do cite individual costs of carbon for each country - for instance here:

https://country-level-scc.github.io/explorer/

There you can for instance see that India's SCC is $85.4 . I think this is based average income till the end of the century taking into account growth ("socioeconomic projections" (perhaps $20k/capita ).) and how much they're hurt by climate damages given how rich they are. This figure is then income adjusted so that it is comparable with present day Americans (it might actually downward adjust the damages of rich countries which will be richer than present day US by the end of the century). Because then it can be interpreted as "this is what the Indian government should spend right now in USD to avert per tonne", which seems to make most sense.

These costs are then all added together to arrive at the global social cost of carbon and what we should spend collectively to avert a tonne of CO2. This is why I think we can compare this with Givedirectly and the analysis is roughly correct.

I think my current model is suboptimal in that it mixes up different income adjustment factor eta = 1.5 and combining it with a different eta they use in the paper. It might be good to standardize this in one analysis. This might change the results.


I'm sorry that I didn't understand quite the point you're making, but you might legitimately be onto something. I think to get to the bottom of this it might be best to create a new model with open and transparent code to figure this out. I saw that you did very good quantitative analysis in a different post, so perhaps you could take this on. But maybe this would be more something for a slightly bigger org with more research capacity to take on. Not sure how productive this is or whether there's really demand for such an analysis though.

Comment by haukehillebrandt on Update on CEA's EA Grants Program · 2019-11-16T16:52:50.850Z · score: 28 (7 votes) · EA · GW

Would shutting down EA grants significantly reduce the overall quantity of meta funding in the community or would the freed up resources be routed into the Meta-fund?

Comment by haukehillebrandt on Be the Match: a volunteer list for bone marrow donation · 2019-10-30T23:23:56.392Z · score: 12 (4 votes) · EA · GW

This paper here might have some useful info for a cost-effectiveness analysis (https://link.springer.com/article/10.1007/s11166-015-9222-7) see Table 11 "Benefits and Costs to World Population of 1,000 Additional Adult Registrants"

Lives Saved

Caucasian 0.00669

Black 0.00668

Asian 0.01940

U.S. Hispanic 0.00538

You'd need ~50k to 100k people to spend 10 mins each to save one life.

Comment by haukehillebrandt on Be the Match: a volunteer list for bone marrow donation · 2019-10-28T09:58:26.220Z · score: 10 (3 votes) · EA · GW

As I said that in my previous reply, I do agree with you that sometimes it can be very effective to improve a generally not very cost-effective intervention such as this. I also agree that one's marginal time might not be very valuable (yet not zero). So I do agree that in theory this could be a cost-effective intervention.

However, I think the marginal cost per death averted figure above of spreading this meme were overstated for two reasons:

1. Crucially, on the marginal costs:

Assume that including cognitive overhead that it takes roughly an hour for an EA to do the swap (e.g. using the swap, mailing it etc.) and the marginal value of my time on Sunday afternoon when I would otherwise just do something pointless is $10 (this is conservative see http://globalprioritiesproject.org/wp-content/uploads/2015/03/NeutralHours.pdf ).

This buys us a 1 in 800, or 0.125% chance of being a match.

This means we need to convince 800 EAs to spend $10 to find a match. The cost is $8000. In other words, your own probability of saving a life is 1 in 800 but you have to pay $10 for it.

Technically, but less crucially, we also need to add that one EA will actually have to go through the donation process for this to have an effect. 20 hours * $10 = $200. If you're unwilling to accept a 1 in 800 probability of going through the process, there'll be no benefit. For one person that means that we need to add 1 in 800 probability of paying $200 (or 25 cents). So the total for one person comes to $10.25.

We're also subsidizing expensive health care, spreading the meme that this is good, and perhaps in the process displacing more effective treatment, which can do net harm (see https://www.nice.org.uk/news/blog/carrying-nice-over-the-threshold ).

2. Also less crucially on the benefits side: As shown in the papers above the QALYs gained from might not add up to a "whole life saved", which can be seen by their measure "per additional treatment success' ("For patients at standard risk for disease, the treatment success rate was 80.3% for BMT recipients" [...] For patients with high-risk disease, the treatment success rate was 23.5% for BMT recipients"). Then, even after successful treatment, cancers often come back, which reduces the cost-effectiveness ("only 62% of patients survived the first year post-BMT, 98.5% of patients alive after 6 years survived at least another year. Almost 1/3 (31%) of the deaths in long-term survivors resulted from causes unrelated to transplantation or relapse. " see https://www.ncbi.nlm.nih.gov/pubmed/16545726 ).

Comment by haukehillebrandt on Be the Match: a volunteer list for bone marrow donation · 2019-10-27T18:25:35.602Z · score: 3 (2 votes) · EA · GW

Cost-effectiveness analysis looking at bone marrow transplantation often include all these costs (e.g. "The median cost of the first 6 months of care including donor identification, marrow collection, patient hospitalization for transplantation and all outpatient medications and readmissions through 6 months postmarrow infusion was $178,500" https://ashpublications.org/blood/article/92/11/4047/133908/The-Costs-and-Cost-Effectiveness-of-Unrelated ).

Here's another more recent related study that similarly suggests that bone marrow transplantation is quite expensive to treat even for advanced economy standards:

https://www.ncbi.nlm.nih.gov/pubmed/20348004

Note that I haven't looked into this very deeply, and this this just a hunch.

Also, again, I agree that sometimes, it can be effective to improve a generally not very effective intervention.

Comment by haukehillebrandt on Be the Match: a volunteer list for bone marrow donation · 2019-10-27T17:56:04.852Z · score: 2 (1 votes) · EA · GW

I meant that, based on Sanjay's info, that only 1 out of 800 people are a match for anyone, and they will spend around 1 hour to fill in forms, do the swap (which in itself creates costs etc.), this should be included in a full costing analysis of the cost per death averted figure and make it no longer competitive with other priority interventions.

Comment by haukehillebrandt on Be the Match: a volunteer list for bone marrow donation · 2019-10-27T12:17:28.350Z · score: 7 (4 votes) · EA · GW

We should probably also add the some 799 hours of those not selected * $25 per hour opportunity cost, to the cost per life saved calculation above.

Comment by haukehillebrandt on Review of Climate Cost-Effectiveness Analyses · 2019-10-24T18:17:10.439Z · score: 2 (1 votes) · EA · GW

Thanks for engaging with this critically!

Generally, my agenda was probably a bit simpler than people might have supposed. This was not intended to be the last word on whether climate change or development interventions are always better. Rather it's a starting point and “choose your own adventure” model to help prioritizing between a concrete climate and a concrete development charities. Different situations call for the model to be adapted.

Note that there are four parameters that drive the results of this analysis (the SCC, the income adjustment eta, the cost to avert CO2, and the effectiveness of global dev/health vs. cash). For the first two, there really is a lot more uncertainty, but for the latter two, it’s more clear. This makes the model actually valuable and with action guiding potential.

For instance, if you’re a small donor and can’t decide between GiveDirectly and the Coalition for Rainforest Nations, then, if you believe that CfRN really has a cost-effectiveness of $0.02 / tCO2e averted, in many scenarios, especially the realistic one around which there is most consensus, it will often beat unconditional cash-transfers, even if you believe that social cost of carbon is quite low.

However, CfRN does lobbying, not a scalable intervention that one could invest a lot of money in. So, in contrast, if you’re a billionaire and are looking to decide between global development and climate change as a cause area for your foundation, then perhaps global development might be a better bet.

You write that there are very large flaws in my methodology, but because you then adopted the methodology, I think you actually have quarrels with the empirical estimates that I’ve plugged in, correct?

Some comments on the parameter estimates that you use in your model:

Your less $1/tCO2e averted figure for Cool Earth seems fair for small donors (deforestation prevention can probably absorb a few hundreds of millions - see “Redd+ agreement” and social impact bonds between Norway and Brazil).

However, for large foundations/governments it doesn’t seem quite as scalable in terms of absorbing very large amounts of money as many global development interventions. That’s why I used an intervention such as ocean alkalinity as an example because it might be a way to absorb large amounts of carbon up to 100 billion tonnes / year) for as little as $10 per tonne of CO₂ averted. https://iopscience.iop.org/article/10.1088/1748-9326/aabf9f/pdf

I thought this intervention was representative / similar order of magnitude (10s of $ / ton averted) as some of the bigger ones in the McKinsey report. The next order of magnitude, 100s of $ is for direct air capture, which as far as I understand could absorb most of the CO at scale, but is too expensive. I think this is why getting direct air capture costs down by one more order of magnitude is seen as a climate holy grail, where you can just pump money into and then it solves the whole problem.

But your model seems to be geared towards small donors deciding between to different charities, and then is inconsistent, because you used the median Givewell charity effectiveness (7.95) and not the most effective 17, comparing the best in class low risk offsetting with the median development charity. Even using your model parameters suggests small donors should donate to Deworming over Cool Earth.

Finally, can you say a bit more why you prefer the eta, marginal utility of consumption, to be equal to 1? I felt you you did not provide sufficient empirical justification for this.

See:

“To arrive at estimates of social discount rates consistent with these growth rates, it is necessary to obtain estimates of the elasticity of marginal utility [...] the survey by Groom and Maddison (2019) suggests estimates between 0.5 and 2.0.

The most substantial cross-country analysis available (Evans, 2005; with a focus on advanced economies) arrives at an estimate of 1.4, which we adopt. This estimate is consistent with a review of some 200 experts who have published on social discount rates, which returns a mean value of 1.35 (Drupp et al., 2018). This estimate—drawing on experts who have published on discount rates in highly ranked journals is not necessarily confined to advanced economies, though the authors acknowledge that expertise from developing countries might be underrepresented.”

“We present a quantitative survey of estimates of the elasticity of intertemporal substitution in what we believe is the largest metaanalysis conducted in economics. We collect 2735 estimates from 169 published studies and find that the mean elasticity is 0.5, but that the estimates vary greatly across countries and methods.” https://www.sciencedirect.com/science/article/abs/pii/S002219961500032X

“estimates of η are derived from the so-called Euler-equation although in the macroeconomics literature this information is normally presented in terms of the elasticity of intertemporal substitution (EIS) which is equal to 1/η.” https://link.springer.com/article/10.1007/s10640-018-0242-z

So the last study suggest the eta is equal 2.

This is why I plugged in 1.5 for the realistic case. But I haven’t looked into this in detail and I’d love to have more people look into it.

I hope this did not come across as too critical - I generally really enjoyed reading your treatment and synthesis of the issue.

Comment by haukehillebrandt on [updated] Global development interventions are generally more effective than Climate change interventions · 2019-10-24T14:51:40.254Z · score: 4 (2 votes) · EA · GW

You're absolutely right - generally, my agenda was probably a bit simpler than people might have supposed. This was not intended to be the last word on whether climate change or development interventions are always better. Rather it's a starting point and “choose your own adventure” model to help prioritizing between a concrete climate and a concrete development charity.

Note that there are four parameters that drive the results of this analysis (the SCC, the income adjustment eta, the cost to avert CO2, and the effectiveness of global dev/health vs. cash). For the first two, there really is a lot more uncertainty, but for the latter two, it’s more clear. This makes the model actually valuable and with action guiding potential.

For instance, if you’re a small donor and can’t decide between GiveDirectly and the Coalition for Rainforest Nations (or perhaps the Let's Fund Clean Energy campaign that you cite), then, if you believe that CfRN really has a cost-effectiveness of $0.02 / tCO2e averted, in many scenarios, especially the realistic one around which there is most consensus, it will often beat unconditional cash-transfers, even if you believe that social cost of carbon is quite low.

However, CfRN does lobbying, not a scalable intervention that one could invest a lot of money in. So, in contrast, if you’re a billionaire and are looking to decide between global development and climate change as a cause area for your foundation, then perhaps global development might be a better bet.

Comment by haukehillebrandt on [updated] Global development interventions are generally more effective than Climate change interventions · 2019-10-24T14:47:41.696Z · score: 4 (2 votes) · EA · GW

Thanks Rob for taking the time to comment and my sincere apologies for the delay in replying.

  1. There really is a lot of uncertainty here. Note that all parameters estimates are based on or grounded in empirical and published estimates. Even my adjustment for the social cost of carbon being over- or underestimate by 10x corresponds to values with similar orders of magnitude you can find in the literature - see cell comments of the spreadsheet model. For instance, one recent paper by a renowned climate economist finds that under different model specifications the SCC ranges from $3.38/tCO2e to $21,889/tCO2e. Ditto with what the eta parameter for the income adjustment.
  2. The “realistic estimate” model scenario is what I perceive to use parameters estimates around which there more consensus, but that’s just my opinion and one can reasonably disagree with these choices.
  3. I used the extreme scenarios to highlight the uncertainty and to make statements such as “Even if you believe the true social cost of carbon is higher than most models suggest (i.e. $20k per tonne, the most extreme value in the literature), then that still often is not enough to beat global development interventions”.

Generally, my agenda was probably a bit simpler than people might have supposed. This was not intended to be the last word on whether climate change or development interventions are always better. Rather it's a starting point and “choose your own adventure” model to help prioritizing between a concrete climate and a concrete development charity.

Note that there are four parameters that drive the results of this analysis (the SCC, the income adjustment eta, the cost to avert CO2, and the effectiveness of global dev/health vs. cash). For the first two, there really is a lot more uncertainty, but for the latter two, it’s more clear. This makes the model actually valuable and with action guiding potential.

For instance, if you’re a small donor and can’t decide between GiveDirectly and the Coalition for Rainforest Nations, then, if you believe that CfRN really has a cost-effectiveness of $0.02 / tCO2e averted, in many scenarios, especially the realistic one around which there is most consensus, it will often beat unconditional cash-transfers, even if you believe that social cost of carbon is quite low.

However, CfRN does lobbying, not a scalable intervention that one could invest a lot of money in. So, in contrast, if you’re a billionaire and are looking to decide between global development and climate change as a cause area for your foundation, then perhaps global development might be a better bet.

Re: Monte Carlos - I think some of the parameter inputs rely on Monte Carlos already. My hunch is that there’s no free lunch here that would reduce the uncertainty much over and above the point estimate / realistic scenario, but this is definitely that I’d like to see other people explore in future research.

Comment by haukehillebrandt on Long-Term Future Fund: August 2019 grant recommendations · 2019-10-24T08:36:32.233Z · score: 7 (2 votes) · EA · GW

Thanks for the heads up - I've cleaned up the formatting now to make it more readable.

Comment by haukehillebrandt on JP's Shortform · 2019-10-07T16:56:30.839Z · score: 1 (3 votes) · EA · GW

Mandatory field 200 characters summarizing the blogpost.

Mandatory keywords box.

Better Google Docs integration.

Comment by haukehillebrandt on Long-Term Future Fund: August 2019 grant recommendations · 2019-10-07T14:28:59.722Z · score: 30 (9 votes) · EA · GW

Thank you for the detailed write-ups.

I will focus on where I disagree with the the Chris Chambers / Registered Reports grant (note: this is Let’s Fund’s grantee, the organization I co-founded).

1. What if all clinical trials became Registered Reports?

You write:

“Chambers has the explicit goal of making all clinical trials require the use of registered reports. That outcome seems potentially quite harmful, and possibly worse than the current state of clinical science.”

I think, if all clinical trials became Registered Reports, then there’d be net benefits.

In essence, if you agree that all clinical trials should be preregistered, then Registered reports is merely preregistration taken to its logical conclusion by being more stringent (i.e. peer-reviewed, less vague etc.).

Relevant quote from the Let’s Fund report (Lets-Fund.org/Better-Science):

“The principal differences between pre-registration and Registered Reports are:

  • In pre-registration, trial outcomes or dependent variables and the way of analyzing them are not described as precisely as could be done in a paper
  • Pre-registration is not peer-reviewed
  • Pre-registration also often does not describe the theory that is being tested.

For the reason, simple pre-registration might not be as good as Registered Reports. For instance, in cancer trials, the descriptions of what will be measured are often of low quality i.e. vague, leading to ‘outcome switching’ (i.e. switching between planned and published outcomes) [180], [181]. Moreover, data processing can often involve very many seemingly reasonable options for excluding or transforming data[182], which can then be used for data dredging pre-registered trials (“With 20 binary choices, 220 = 1,048,576 different ways exist to analyze the same data.” [183]). Theoretically, preregistration could be more exhaustive and precise, but in practice, it rarely is, because it is not peer-reviewed.”

Also, note that exploratory analysis can still be used in Registered Reports, if it’s clearly labelled as exploratory.

----

2. Value of information and bandwidth constraints

You write:

“Ultimately, from a value of information perspective, it is totally possible for a study to only be interesting if it finds a positive result, and to be uninteresting when analyzed pre-publication from the perspective of the editor.“

Generally, a scientist’s priors regarding the likelihood of treatment being successful should be roughly proportional to the value of information. In other words, if the likelihood that a treatment is successful is trivially low, then it is likely too expensive to be worth running or will increase the false positive rate.

On bandwidth constraints: this seems now largely a historical artifact from pre-internet days, when journals only had limited space and no good search functionality. Back then, it was good that you had a journal like Nature that was very selective and focused on positive results. These days, we can publish as many high-quality null-result papers online in Nature as we want to without sacrifice, because people don’t read a dead tree copy of Nature front to back. Scientists now solve the bandwidth constraint differently (e.g. internet keyword searches, how often a paper is cited, and whether their colleagues on social media share it).

In your example, you can combine all 100 potential treatments into one paper and then just report whether it worked or not. The cost of reporting that a study was carried out are trivial compared to others. If the scientist doesn’t believe any results are worth reporting they can just not report them, and we will still have the record of what was attempted (similar to it being good that we can see unpublished preregistrations on trials.gov that never went anywhere as data on the size of publication bias).

3. Implications of major journals implementing Registered reports

You write:

“Because of dynamics like this, I think it is very unlikely that any major journals will ever switch towards only publishing registered report-based studies, even within clinical trials, since no journal would want to pass up on the opportunity to publish a study that has the opportunity to revolutionize the field.”

This is traded-off by top journals publishing biased results (which follows directly from auction theory where the highest bidder is more likely to pay more than the true price; similarly, people who publish in Nature will be more likely to overstate their results. This is borne out empirically. See https://journals.plos.org/plosmedicine/article?id=10.1371/journal.pmed.0050201)

Registered Reports are simply more trustworthy and this might change the dynamics so that there’ll be pressure for journals to adopt the registered Reports format or fall behind in terms of impact factor.

--

3.1 On clarity

You write:

“As a result, large parts of the paper basically have no selection applied to them for conceptual clarity,”

On clarity: Registered reports will have more clarity because they’re more theoretically motivated (see https://lets-fund.org/better-science/#h.n85wl9bxcln4) and the reviewers, instead of being impressed by results, are judging papers more on how detailed and clear the methodology is described. This might aid replication attempts and will likely also be a good proxy of the clarity of the conclusion. Scientists are still incentivized to write good conclusions, because they want their work to be cited. Also, the importance of the conclusion will be deemphasized. In the optimal case of a RR, “ a comprehensive and analytically sophisticated design, vetted down to each single line of code by the reviewers before data collection began,” https://www.nature.com/articles/s41562-019-0652-0 is what happens during the review.

What is missing from the results section is pretty much only the final numbers that are plugged in after review and data collection and the result section then “writes itself”. The conclusion section is perhaps almost unnecessary, if the introduction already motivates the implications of the research results and is already used as a more extensive speculative summary in many papers.

I think the conclusion section will be quite short and not very important section in registered reports as is increasingly the case (in Nature, there’s sometimes no “redundant” conclusion section).

---

4. Is reducing red tape more important?

You write:

>>Excessive red tape in clinical research seems like one of the main problems with medical science today

I don’t think excessive red tape is one of the main problems with medical science (say on the same level of publication bias), that there are no benefits of IRBs, nor that Registered Reports adds red tape or has much to do with the issue you cite. I think a much bigger problem is research waste as outlined in the Let’s Fund report.

Most scientists who publish Registered Reports describe the publication experience as quite pleasant with a bit of front-loaded work (see e.g. https://twitter.com/Prolific/status/1153286158983581696). In my view, the benefits far outweigh the costs.

5. On Differential technological development aspect of Registered Reports

On differential tech development and perhaps as an aside: note that more reliable science has wide-ranging consequences for many other cause areas in EA. Not only global development has had problems with replicability (e.g. https://blogs.worldbank.org/impactevaluations/pre-results-review-journal-development-economics-lessons-learned-so-far and the “worm wars”), but also areas related to GBCRs (e.g. there’s a new Registered Reports initiative for research on Influenza see https://cos.io/our-services/research/flu-lab/).

Comment by haukehillebrandt on JP's Shortform · 2019-10-07T11:43:18.713Z · score: 2 (1 votes) · EA · GW

I'd be interested in seeing views/ hits counters on every post and general data on traffic.

Also quadratic voting for upvotes.

Comment by haukehillebrandt on [updated] Global development interventions are generally more effective than Climate change interventions · 2019-10-07T09:29:03.326Z · score: 6 (1 votes) · EA · GW

Thank you for your kind words Sam.

On discounting:

Of course, philosophically, pure time discounting is wrong, but:

"Another reason to discount is that far future benefits are more speculative, and changes to the world in the meantime can disrupt your project or make it irrelevant. For example, a vaccine development project that hopes to deliver a vaccine in a few decades faces a higher risk of being defunded or the disease in question disappearing, than does a similar project that expects to deliver a vaccine in a matter of years. This is a good reason to discount future benefits and costs, but the appropriate rate will vary dramatically depending on what you are looking at, and will not necessarily be the same every year into the future."

https://www.givingwhatwecan.org/post/2013/04/was-tutankhamun-a-billion-times-more-important-than-you/


The social cost of carbon is generally highly sensitive to the pure rate of time preference.

But:

"The national social costs of carbon of faster growing economies are less sensitive to the pure rate of time preference and more sensitive to the rate of risk aversion" from Tol

and from the Ricke paper:

"CSCCs were calculated using both exogenous and endogenous9 discounting. For conventional exogenous discounting, two discount rates were used, 3 and 5%. the results under endogenous discounting were calculated using two rates of pure time preference (ρ=1, 2%) and two values of elasticity of marginal utility of consumption (μ=0.7, 1.5) for four endogenous discounting parameterizations."

So maybe it's not highly sensitive to just discounting anymore.

But both the Ricke and the Tol paper use sensitivity analyses on their SCC and different parameterizations make SCC 10s to 1000s of $ per tonne and I guess they'll use "sensible" ranges for this.

I would love others to look into this more as well and could well imagine new research uncovering facts that would dominate this analysis.

Comment by haukehillebrandt on [updated] Global development interventions are generally more effective than Climate change interventions · 2019-10-07T08:45:00.784Z · score: 6 (1 votes) · EA · GW

Excellent point

I think it's a mixture of the following:

1. African countries are relatively small

2. the social cost of carbon measures the cost to GDP - and if your GDP is not very big to start with then there's not a big cost to you.

It is quite unintuitive/ disconcerting that the official social cost of carbon for the DRC (one of the poorest countries, 80 million people, close to the equator and particularly affected by climate change), only has a social cost of carbon of 30 cents per tonne, whereas the US has one of $40 - see:

https://country-level-scc.github.io/explorer/

As I said above, there are contributors to (true) social cost of carbon not fully captured by empirical, macroeconomic damage functions, and their likely impacts on the social cost of carbon (see Table S5 in the paper’s supplementary material and Table 1 in[21]). For instance:

  • Adjustment costs (short-term costs of adaptation)
  • Non-market damages (biodiversity loss, cultural losses, etc.)
  • Tipping points in the climate system (catastrophic climate events, hysteresis etc.)
  • High inertia effects of CO2 (ocean acidification, sea level rise)
  • General equilibrium effects (spillover, trade, etc.)
  • Macro-scale adaptation (long-term restructuring of economy)
  • Political instability and violent conflicts
  • Large migration flows
  • More extreme weather and natural disasters
  • Bresler finds that explicitly accounting for climate mortality costs triples the welfare costs of climate change.[22]
  • The highest social cost of carbon estimate in the literature is on the same order of magnitude ($1687[23]), and the highest figure amongst many in a recently published paper find that for 6 degrees of warming the cost will be (which has a substantial probability) is $21889 / per tonne) [24]

That's why in the pessimistic version of my model increased the SCC by 10x (higher than most estimates).

Comment by haukehillebrandt on Should CEA buy ea.org? · 2019-10-06T22:40:04.298Z · score: 4 (2 votes) · EA · GW

We got this one, which was quite cheap:

http://ea.do

Comment by haukehillebrandt on [updated] Global development interventions are generally more effective than Climate change interventions · 2019-10-02T18:17:18.901Z · score: 11 (5 votes) · EA · GW

The changes have been very substantial, because the first version was a much more simple model.

The main difference are that the first version had an error as pointed out by AGB in the comments.

Here's a comparison doc between the two version:

https://docs.google.com/document/d/1a-6xC7WjSq2-AtSuUPKaQf1ielxYFrElxfATVDBfMA4/edit?usp=sharing

Comment by haukehillebrandt on [updated] Global development interventions are generally more effective than Climate change interventions · 2019-10-02T08:55:57.551Z · score: 2 (1 votes) · EA · GW

Thank you- I've now included this in my model:

"Some global development interventions have been estimated to be 17.5x more effective than cash-transfers (e.g. deworming).[34] We use this as the optimistic case."

Comment by haukehillebrandt on [updated] Global development interventions are generally more effective than Climate change interventions · 2019-10-02T08:54:18.974Z · score: 6 (1 votes) · EA · GW

I agree that climate modelling is very uncertain, but we should not throw out the baby with the bathwater.

Quote from my analysis above:

"one study estimated a lower bound of the global social cost of carbon at US$125 and argues that:

“Quantifying the true SCC value is complicated because of various difficult-to-quantify damage cost categories and the interaction of discounting, uncertainty, large damages and risk aversion [...] The best that can be offered is a lower bound based comes from a conservative meta-estimate that aggregates studies using high and low discount rates, it does not account for various climate change damages owing to a lack of reliable information, and it does not consider a minimax regret argument addressing damages associated with extreme climate change.”

Also, as an aside, outside of prioritization, for optimal policy (e.g. carbon pricing) the social cost of carbon should be:

  1. Set to the marginal abatement cost, which can be optimal and easier to estimate.[17] or
  2. Set to err on the side of overestimating externalities[18] (while reducing other non-Pigovian taxes)."

I now include more optimistic estimates (in the sense that the SSC won't be that high) in my sensitivity analysis.

Comment by haukehillebrandt on [updated] Global development interventions are generally more effective than Climate change interventions · 2019-10-02T08:49:12.648Z · score: 3 (2 votes) · EA · GW

Thank you for this comment. Relevant quote from my updated analysis above:

"The new paper’s social cost of carbon figure is controversial and has been criticized for being too high for various methodological reasons.[6] For instance, one very critical new paper also now estimates the social cost of carbon on a country-level, suggesting that the global social cost of carbon is only $24 (and, using various sensitivity analyses, values ranging from $3.38/tCO2e to $21,889/tCO2e).[7]

To account for the new paper overestimating or underestimating the social cost of carbon, below, we use sensitivity analysis to show how our model responds to over- or underestimating the true social cost of carbon by 10x."

Comment by haukehillebrandt on [updated] Global development interventions are generally more effective than Climate change interventions · 2019-10-02T08:47:59.149Z · score: 9 (3 votes) · EA · GW

Thanks for catching this mistake.

I've updated the analysis to reflect this.

I emailed the authors but they didn't reply.

But I think the social cost of carbon figures should generally be interpreted as current US dollars. They are then discounted for decreasing returns to consumption for future people who live in countries with higher consumption.

So we should divide the $417 figure by the 100x multiplier (or more, see my sensitivity analysis).

Comment by haukehillebrandt on AID Data · 2019-09-01T07:42:59.493Z · score: 5 (3 votes) · EA · GW

The OECD is the main source for data on Official Development Assistance- see

https://stats.oecd.org/

https://data.oecd.org/searchresults/?hf=20&b=0&q=aid&l=en&s=score

https://stats.oecd.org/qwids/

The World Bank also has some interesting data:

https://data.worldbank.org/

Comment by haukehillebrandt on Best EA use of $500,000AUD/$340,000 USD for basic science? · 2019-08-27T10:29:08.877Z · score: 39 (18 votes) · EA · GW

I'm the co-founder of Lets-Fund.org.

We do independent, in-depth research to help foundations and individuals to donate to the most effective policies to solve today’s most important global challenges (e.g. the replication crisis, climate change).

One of our two campaigns is on improving all of hypothesis-driven science by implementing a new publication format called Registered Reports.

You can find our in-depth write-up on this here:

Lets-Fund.org/Better-Science

We've already crowdfunded $75,000 for this campaign (this includes a grant recommendation that the EA Long-term Future fund is currently very strongly considering), but I believe the grantee can productively absorb more money. This would fund a teaching buyout for the grantee, Professor Chris Chambers (who happens to be a fellow Aussie!).

Chris Chambers has already hired assistants to push his Registered Reports advocacy forward and I’m exceedingly excited about his work. There was an editorial in Nature about it a few weeks ago, but in brief, he has implemented the new Registered Reports publication format at more than 200 journals now (including PLoS Biology, a top biology journal). More promisingly, he is also lobbying PNAS, generally considered to be the best journal after Nature and Science, to implement the format through an open letter signed by 250 other scientists.

If he is successful and scientists realize that they can get a publication in PNAS just by submitting a paper with exceptional methodology, but independent on whether the results are positive, then it might cascade into changing science in a fundamental way in the near future. I think Chris is very driven and can productively use additional funds.

There has also been some recent interest in global development about Registered Reports (see: https://blogs.worldbank.org/impactevaluations/pre-results-review-journal-development-economics-lessons-learned-so-far ) so I feel like this could prevent another Worm Wars situation (see https://blog.givewell.org/2017/12/07/questioning-evidence-hookworm-eradication-american-south/ ) , and so this grant opportunity would also work for someone interested in improving global development.

Let me know if you have any questions at (happy to jump on a call):

Hauke@Lets-Fund.org

Comment by haukehillebrandt on EU AI policy and OPSI Consultation · 2019-08-11T09:28:41.541Z · score: 5 (3 votes) · EA · GW

just came across this - deadline has already passed but perhaps this might still be useful:

OMB Seeking Input on AI R&D

In support of the administration’s artificial intelligence R&D strategy, the White House Office of Management and Budget is accepting public comments on ways to improve the accessibility and quality of relevant federal datasets and models. OMB notes that domain areas of particular interest include weather forecasting, manufacturing, agriculture, and national security, among others. Comments are due Aug. 9. 

https://www.federalregister.gov/documents/2019/07/10/2019-14618/identifying-priority-access-or-quality-improvements-for-federal-data-and-models-for-artificial

Comment by haukehillebrandt on Cluster Headache Frequency Follows a Long-Tail Distribution · 2019-08-03T11:12:09.707Z · score: 16 (9 votes) · EA · GW

Generally I really find this research agenda interesting. I have only skimmed this post, but I also like your analysis the way you go about it.

One nitpick:

As it turns out, LSD, psilocybin, and DMT all get rid of Cluster Headaches in a majority of sufferers. Given the safety profile of these agents, it is insane to think that there are millions of people suffering needlessly from this condition who could be nearly-instantly cured with something as simple as growing and eating some magic mushrooms.

I think this is hyperbole. I reviewed the literature a while ago, and while I do agree that there is some suggestive evidence that this is true, I do not think that it is so strong as to warrant the claims you make and there are many qualifications. Also, I think you should cite the relevant studies on this subject (https://scholar.google.com/scholar?as_ylo=2015&q=LSD+cluster+headaches&hl=en&as_sdt=0,5).

Comment by haukehillebrandt on The Unit of Caring: On "fringe" ideas · 2019-08-02T12:33:14.434Z · score: 11 (7 votes) · EA · GW

Also see Linch's excellent summary of the philosophy paper "The possibility of ongoing moral catastrophe":

https://docs.google.com/document/d/18ZzC-WkDcWK-WPlIzKvDv83j8aBwSfdOxnZRmoio-zE/edit