GiveWell money moved forecasts and implications

post by vipulnaik · 2015-12-19T20:22:21.563Z · score: 8 (8 votes) · EA · GW · Legacy · 9 comments

Contents

  Note: See UPDATES 1, 2, 3, 4, and 5 at the bottom of the post.
  TL;DR
  Why is this important?
  The amounts of money moved are now sufficiently significant that this sort of analysis is worthwhile
  An overall sense of magnitude
  Money moved estimates
  Explanation of the lower and upper end estimates
  A quick theoretical model for how GiveWell moves money
  Discovery and brand awareness
  The role played by the specifics of GiveWell's recommendations and the rhetoric they use
  Increased coordinated doubling down on GiveWell recommendations by its allies
  More on the distribution of money moved to top charities
  Growth in coming years
  Implications
None
9 comments

Note: See UPDATES 1, 2, 3, 4, and 5 at the bottom of the post.

This post assumes some familiarity with the work of charity evaluator GiveWell, but here's a quick backgrounder for those not familiar: GiveWell lists a small set of top charities to donate to, and publishes detailed reviews of each. The list for 2015 comprises Against Malaria Foundation (malaria bednet distribution), GiveDirectly (unconditional cash transfers), Deworm the World Initiative (deworming), and Schistosomiasis Control Initiative (deworming). GiveWell's recommendations have acquired the status of a gold standard for conventional, unattached donors, and GiveWell has a particularly strong reputation among many of the people who read this site and identify as effective altruists. GiveWell also works closely with Good Ventures, the private foundation of Facebook co-founder Dustin Moskovitz and his wife Cari Tuna. Good Ventures has made significant donations to GiveWell's top charities, mostly at GiveWell's suggestion, with the amount reaching $44.4 million (plus $1 million to standout charities, plus a separate three-year $25 million grant to GiveDirectly) in 2015.

In a recent post on the Effective Altruists Facebook group critiquing what I saw as claims of take-off exponential growth of the EA movement, I noted some estimates of what I believe GiveWell's money moved will be for 2015 and the years to come. In this post, I want to develop and refine those estimates more.

TL;DR

[Added at request of commenters, even though I'm skeptical that this TL;DR can accurately capture the entirety of the post].

For simplicity, this TL;DR does not describe the intervals of uncertainty. For those you'll have to read the whole post (or the appropriate section thereof).

Why is this important?

GiveWell was an early proponent of the idea of room for more funding, and of the closely related concept of marginal impact, i.e., what effect does your marginal contribution have? In order to best estimate an organization's room for more funding, GiveWell tries to estimate how much the organization will raise from non-GiveWell donors, and also takes into account the amount it is recommending that Good Ventures donate. However, GiveWell doesn't provide official guidance on how much money it expects GiveWell-influenced donors other than Good Ventures to give. I believe that carrying out this estimation is important for a few reasons:

It could still be the case that while accurate answers are helpful, wild speculation isn't. But getting to accurate answers itself requires practice at estimation and forecasting, even if initial forecasts are of low quality. This post is an attempt at initial forecasts, along with a walkthrough of the reasoning used to generate the forecasts. The forecasts might well be of low quality, but I hope people find the exercise illuminative.

In addition to the value of the final forecasts, the reasoning used in arriving at the forecasts helps shed light on other facets important to donors, recipient organizations, charity evaluators, and fundraising organizations.

The amounts of money moved are now sufficiently significant that this sort of analysis is worthwhile

As GiveWell's donor base has increased in size and its annual money moved has hit eight digits, estimation makes more and more sense. A few reasons:

An overall sense of magnitude

To give some sense of the numbers:

What about GiveWell-recommended charities? Here are some rough estimates I got with a few minutes of Googling; numbers may be far from accurate.

These broad estimates show that in terms of money moved, fraction of the philanthropic sector, fraction of the wealth in donor groups, and fraction of the resources available to recipient groups, GiveWell and its top charities are still quite small. This suggests that there is considerable upward room to grow without hitting hard limits. Barriers to growth can of course arise in a variety of ways, but even so, there is no strong evidence against the proposition that, over the next few decades, GiveWell's annual non-Good Ventures money moved can grow by 1-2 orders of magnitude, while still remaining a small fraction of the quantities involved. For instance, GiveWell cited the Money for Good study (funded to help rethink the Nonprofit Marketplace Initiative of the Hewlett Foundation) and noted that, based on interpretation, somewhere between $550 million and $4 billion in annual giving in the United States could be influenced by GiveWell-like resources. This would be somewhere between 0.2% and a little over 1% of annual giving in the United States, and represents a 40-200X increase in the non-Good Ventures money moved by GiveWell.

Money moved estimates

A few relevant pieces of information:

My estimates for the non-Good Ventures money moved by GiveWell have changed slightly since my Effective Altruists Facebook group post; here are the revised estimates.

In contrast, according to an October 2015 board meeting, Elie Hassenfeld of GiveWell was confident that it would move over 20 million dollars in 2015. I believe this confidence is misplaced, for reasons I discuss below (with that said, the 20 million+ scenario is about 40% likely, so it's not way off from my estimate). But it is also possible that they have insider information that I lack access to. That is part of the reason for my upward revision of my estimates since the Facebook post.

For the 2015 money moved estimate, here are some more details before I delve into my reasons.

Here is the distribution by percentile:

Here is the distribution for top charities (50% probability for each):

The low lower-end estimate for Deworm the World Initiative arises from the fact that it raised less money last year, and the increased endorsement of the organization happened at the same time that GiveWell doubled down on AMF as a top pick. Moreover, Good Ventures has agreed to fill its entire Level 1 funding gap, making it less attractive for individual donors who take GiveWell's analysis serious.

Here's the size distribution of donors for 2012, 2013, and 2014 (from the same GiveWell blog post).

GiveWell money moved by size bucket

 

We see that 2013 was a significant increase year for number of donors, whereas 2014 was an year when the number of donors didn't increase much (there was a lot of churn) but the average money moved per donor increased significantly. I believe that the situation for 2015 will look positive on both fronts: an increase in the number of donors, and an increase in the money moved per donor. My median estimates for the growth on both fronts is around 20%. Explicitly, my 50% interval for the number of donors in 2015 is 9,500-12,500. My 50% interval estimate for the money moved per donor is $1,500-2,000.

Explanation of the lower and upper end estimates

Explanation of the lower end: Since GiveWell has already moved $5.56 million in money in the first two quarters of 2015, and Elie was reasonably confident of the 20 million figure for the whole year, I expect that donations did not completely stop in Q3 of 2015. Therefore, I expect that just the first three quarters alone would have accounted for at least $6.5 million in donations. That leaves about $2.3 million in potential donations for the end of year, which is fairly easy to achieve with existing pledges. Therefore, I expect a probability of about 1% of missing the $8.8 million mark. That 1% covers major scandals at GiveWell and its recommended charities, or other unforeseen problems.

Explanation of the upper end: The upper end of estimates is mainly governed by factors such as wealthy donors, highly successful fundraising groups or celebrity endorsements or media publicity, and other unforeseen factors.

With the lower and upper end taken care of, I'll focus on explaining the middle. Specifically, I want to explain why my median estimate is lower than the "well over 20 million" that Elie and Holden have put out.

A quick theoretical model for how GiveWell moves money

One thing seems reasonably clear to me from many different sources of anecdotal evidence: 

Here's my description of the most typical pathway for a GiveWell donor:

Based on this model, a few key determinants of GiveWell's money moved stand out:

At the beginning of my investigation, I believed that the focus should be on discovery and conversion. This partly stems from my pro-web bias, where short-run pageviews are both the most important and fairly decent proxies for long-run pageviews. However, after I dug deeper into the data, I now believe that the main driver of GiveWell's growth in the near term is retention and upsell. My upshot:

Discovery and brand awareness

While we don't have money moved or GiveWell web traffic data from August 2015 onward, we do have other pieces of data that we can use to understand GiveWell's growth.

I combine pageview counts for the Wikipedia page about GiveWell with GiveWell's spreadsheet with data on the number of unique visitors. Due to month-to-month variation in traffic I look at year-over-year growth for a three-month total ending at the current month. So the first row is the percentage increase between the September to November 2014 period and the September to November 2015 period.

Month Views of page GiveWell Year-over-year growth (three-month total ending at the current month) Unique visitors (GA) Year-over-year growth (three month total ending at the current month)
201511 2126 -3.14%    
201510 1783 -5.75%    
201509 1735 3.40%    
201508 1840 31.09% 58,371 14.51%
201507 1929 53.40% 58,997 66.47%
201506 1966 60.89% 69,496 58.21%
201505 1695 44.61% 91,280 41.86%
201504 1893 49.77% 69,479 12.29%
201503 1722 60.99% 60,762 6.29%
201502 1660 37.24% 51,995 10.26%
201501 1884 25.99% 67,305 -5.05%
201412 2316 18.10% 145,927 -11.95%
201411 1928 41.12% 68,231 -24.74%
201410 1910 45.40% 64,799 -8.76%
201409 1989 46.04% 56,306 -9.58%
201408 1786 8.53% 76,064 -15.90%
201407 1548 -12.95% 39,238 -27.20%
201406 1041 -20.33% 47,890 -16.99%
201405 1055 -16.96% 44,894 -3.81%
201404 1356 -26.09% 52,754 -1.19%
201403 1261 -38.41% 58,512 -4.87%
201402 905 -32.33% 51,031 18.58%
201401 1105 -25.41% 59,859 49.38%
201312 2260 -17.50% 129,662 75.63%
201311 1499 -13.02% 106,909 89.65%
201310 1452 -2.23% 80,231 101.46%
201309 1178 2.79% 64,443 104.32%
201308 1280 10.59% 71,414 118.98%
201307 1187 22.54% 53,943 92.31%
201306 1564 56.99% 68,686 82.39%
201305 1435 100.64% 58,711 33.86%
201304 1334 169.36% 47,934 32.52%
201303 1653 188.64% 55,700 36.17%
201302 1778 208.71% 60,624 35.46%
201301 1880 179.39% 61,754 4.82%
201212 2652 167.06% 80,480 -8.85%
201211 1989 55.28% 56,205 -21.56%
201210 1675 45.68% 43,695 -17.82%
201209 1083 40.44% 32,758 -20.52%
201208 1241 156.33% 30,810 -13.96%
201207 1222 154.74% 29,325 -3.45%
201206 1182 82.06% 28,476 -7.37%
201205 1012 23.20% 36,494 -4.26%
201204 566 3.39% 31,160 13.80%
201203 626 17.42% 53,627 44.43%
201202 577 47.26% 39,163 110.29%
201201 637 64.83% 37,986 144.78%
201112 830 84.48% 72,603 180.97%
201111 867 167.22% 78,724 249.98%
201110 668 137.05% 46,566 277.10%
201109 1522 133.58% 43,824 308.39%
201108 555 45.10% 40,133 300.78%
201107 448 43.58% 32,925 235.28%
201106 419 66.59% 29,934 206.37%
201105 474 101.24% 34,801 231.79%
201104 623 165.27% 39,044 186.15%
201103 692 58.76% 52,837 82.94%
201102 396 42.95% 17,039 18.61%
201101 479 47.04% 20,671 24.87%
201012 513 124.13% 33,503 47.92%
201011 424 134.43% 23,167 58.11%
201010 345 119.73% 13,763 53.78%
201009 375 89.32% 11,391 50.04%
201008 438 64.15% 9,458 49.54%
201007 268 62.43% 7,771 85.33%
201006 274 58.26% 8,469 118.57%
201005 392 24.16% 12,888 68.58%
201004 244 -12.84% 12,517 69.78%
201003 253 -37.61% 12,776 136.14%
201002 148 -62.03% 12,771 300.57%
201001 586 -74.00% 23,948 435.80%
200912 237 -79.56% 23,319 442.92%
200911 140 -70.74% 14,672 592.54%
200910 195 6.90% 9,623 479.03%
200909 153 39.27% 6,267 489.83%
200908 179 36.61% 6,617 442.80%
200907 239 10.36% 6,191 264.66%
200906 179 -4.96% 4,377 198.79%
200905 157 28.09% 5,149 205.76%
200904 239 50.41% 5,972 112.69%
200903 320 98.74% 11,527 -8.44%
200902 181 292.18% 4,921 -78.52%
200901 1081   4,512  
200812 1295   5,555  
200811 1328   1,493  
200810 175   1,722  
200809 165   1,198  
200808 153   967  
200807 92   1,069  
200806 192   1,130  
200805 237   2,111  
200804 176   1,946  
200803 146   3,350  
200802 170   5,245  
200801 480   14,298  
200712 2   50,240  

If you dig into the Wikipedia pageviews more at the link, you'll see there was a sharp increase in the latter half of July 2014. In fact, the date range for the increase can be narrowed to the period between July 11 and August 4. You can also check against stats.grok.se data.

For this reason, we see that three-month periods whose end dates are after August 2014 but before August 2015 show robust year-over-year growth, whereas three-month periods September 2015 onward show anemic growth in Wikipedia views.

On these indicators, 2015 seems to be a year of flatlining. However, Google Trends data suggests robust year-on-year growth from 2014 to 2015, up to and including November and December. I'm not really sure how much relative weight to place on Wikipedia view data versus Google Trends data. Since Google Trends methodology is less transparent, I'm a little more suspicious of it. But on the other hand, Google clearly has much more money and brains behind the quality of the data. Overall I would say the evidence for growth in 2015 (over and above whatever happened in the middle of 2014) is decidedly mixed. Either way, it is unlikely to be the driver of an increase in donations. If discovery and brand awareness were the only thing that I were looking at, I wouldn't estimate more than 10% increase in annual money moved. But I actually estimate a much greater increase (over 40%, with 20% being due to more donors  and 20% being due to more money per donor). Why? Basically, conversion and upsell. Since these two are somewhat hard to separate, I'll discuss some changes in 2015 that I believe will positively affect both conversion and upsell, without separating conversion from upsell.

I should also mention here some currently fledgeling efforts being made to increase brand awareness of GiveWell and the ideas surrounding it. Gleb Tsipursky wrote a Huffington Post listicle on the 8 Secrets of Savvy Donors, and he reported in an EA Forum post that it increased awareness and also converted at least one moderately influential person. It's too early to evaluate the significance of these for the 2015 Giving Season.

The role played by the specifics of GiveWell's recommendations and the rhetoric they use

There is a reasonable argument that the actual list of top charities, and the commentary that GiveWell makes when recommending them, itself affects how much money is moved to them. The top charity list hasn't changed since the 2014 giving season. But there are a few important differences:

Indeed, as described in the 2015 staff members' personal donations blog post, most of GiveWell's staff are donating all or most of their own annual money to AMF, with Holden and Elie in particular donating all their money to AMF. In contrast, in previous years, GiveWell's own staff members were more skeptical of the ranking or relative allocation suggested by GiveWell, and opted for a different mix. GiveWell's staff also expressed more confidence in 2015 (justified or not) in the rigor of the process used to evaluate top charities and their room for more funding. For instance, co-founder and co-executive director Elie wrote:

Last year, we gave significantly more to GiveDirectly than GiveWell recommended because I put less weight on cost-effectiveness analysis than other staff. I remain ambivalent about putting so much weight on cost-effectiveness estimates because they are subject to numerous debatable assumptions, but I’ve changed my mind based on work we did over the past few months. We put significant effort into debating our cost-effectiveness model (.xls) and encouraged all GiveWell staff to input their own values into the model. The resulting strong consensus that cash is significantly less cost-effective than bednets/deworming persuaded me to put more weight on cost-effectiveness analysis in my giving this year.

Similarly, Alexander Berger notes his increased confidence in the recommendations and how much AMF strikes him as a great opportunity:

What I wrote last year is even more true this year: I think our recommendations this year are stronger than they were last year, and I’m excited to support them. And since I’ve been (even) less involved in our top charities research this year than in previous years, I’m also inclined to just follow the recommended GiveWell allocation (i.e. 100% to AMF).

I’m particularly struck this giving season by just how strong a recommendation AMF is. We believe that AMF can fairly reliably fill a gap in global bednet coverage and avert child deaths for something on the order of every $3,000 donated. I’m much more skeptical than some of my colleagues about how much we can learn from our cost-effectiveness models, particularly for deworming, but I see the case for bednets as being quite strong. And even if we are off by a factor of several times, which I think is absolutely possible, I see that as remarkably cost-effective: for comparison, common estimates put the value of a statistical life in the United States at around $5 million.

Increased coordinated doubling down on GiveWell recommendations by its allies

One major thing I see being done at a larger scale in 2015 is coordination by organizations that seek to move money to GiveWell top charities through careful marketing and messaging. GiveWell, which has historically remained very detached from efforts to spread the word about its top charities, has given a more explicit nod to fundraising efforts explicitly aligned with the goal of raising money for its top recommended charity. For instance, their Outreach Associate Catherine Hollander (the position didn't even exist last year) wrote a blog post recently on fundraising for AMF, the top charity. Charity Science's fundraising operation also appears to have matured this year, as exemplified in their post providing guidance for Christmas fundraisers. Giving What We Can also seems to have upped its social media presence and the publicity surrounding its Pledges. While not all the money moved through this will be attributed to GiveWell, it may raise the profile of GiveWell and move some money explicitly attributed to GiveWell.

More on the distribution of money moved to top charities

In 2014, AMF was not in the top charity list till the end of the year, but still raised $4.4 million from non-GV donors, higher than any other charity. This year, AMF was a top charity throughout the year, and GiveWell identified as the sole charity that it recommended GiveWell-aligned donors to give to. GiveWell itself, and its allies, are picking on AMF as the focal point of fundraising campaigns. AMF is also intrinsically appealing to GiveWell donors, since it's a scalable implementation of a proven intervention that saves lives. For all these reasons, I believe that AMF will raise the most money among GiveWell-recommended charities by a huge margin. Moreover, the uncertainty around money moved to AMF accounts for most of the uncertainty in total money moved.

I believe the other charities will generally receive money from people who have strong prior reasons for wanting to donate to the charities, with the GiveWell recommendation playing a Charity Navigator-like role of validating the organization's legitimacy. For instance, GiveDirectly excites a large number of effective altruists, due to the simplicity, non-paternalism, strong organizational growth, and significant scalability. People with whom these values resonate will likely continue to donate to GiveDirectly and use the huge funding gap GiveWell has found for the organization to bolster their case. Similarly for SCI and DtWI. It's also important to note that since the doubling down on AMF happened in November, the first few months probably saw a larger share of donations to these other charities.

Growth in coming years

Most of my discussion so far has been focused either on general models for money moved or what'll happen in 2015. However, my primary interest is in understanding growth over the next 5 or more years. The reason I devoted so many pixels to 2015 is that the statements I make about 2015 will be tested against reality in fairly short order, and I believe this calibration against reality will be crucial to evaluating my credibility.

Let's get back to three determinants of GiveWell's money moved: discovery and brand awareness, conversion, and retention/upsell.

The upshot: most of the growth in money moved to GiveWell top charities in the next two years will come from retention and upsell. More concretely, this means that the number of donors probably won't increase that much from 2015 to 2016 and 2017, but the money moved per donor will continue to increase, largely because of an increase in donors in high size buckets (and mostly this will be donors who donated smaller amounts in earlier years). More concretely, I expect the number of donors to increase by 10% per year for the next two years (2015 to 2016 and 2016 to 2017) and I expect the money moved per donor to increase by about 20% and 15% in the coming two years. A lot of this will come from the highest donation size buckets, via phenomena such as Founders Pledge donations.

I don't think the time is right to spell out the model in detail. I'd like to see how close my 2015 predictions are to reality and uncover any biases in how I'm thinking about things before elucidating a plausible story for the next few years.

Implications

To the extent that I believe the median case of my forecasts, it leaves me with mixed feelings. I believe that there is wisdom in having larger numbers of eyeballs engaging with the data and recommendations. The idea of growth being propelled more by retention than by user growth is a negative as far as quality is concerned. On the other hand, assuming some rationality on the part of donors, larger sums of money moved per donor incentivizes each donor to care more about their donations, and could in principle put more scrutiny on GiveWell. I believe that the former effect significantly outweighs the latter at the present margin, particularly because a lot of the growth in money moved per donor comes from donors just getting richer, rather than deciding to donate a larger fraction of their income to charity.

I also think the continued short-run increase in money moved to GiveWell top charities reduces the incentives for GiveWell to expand to more domains and undertake the hard task of finding new areas to donate to. This limits their long-term appeal (in particular, it puts a ceiling on how much conversion they can accomplish), but more importantly from a social welfare perspective, it limits the fraction of the donor population and the fraction of donor money that will be influenced by the kind of deep analysis that GiveWell strives to do. (With that said, Charity Navigator-level or GuideStar-level analysis of charities has taken off in a decent way and is probably informing over 1% of money moved already, but that's not the same type of analysis since it's based on a much smaller, though still important, set of metrics).

UPDATE 1: In a blog post comment on December 23, Elie Hassenfeld reported that as of December 22, GiveWell had moved $15.7 million in non-Good Ventures donations to its top charities. My original estimates did not include this information, but accounting for it would move the lower end of my estimates upwards somewhat while not significantly affecting the higher ends.

UPDATE 2: In a blog post on January 8, 2016, Tyler Heishman reports GiveWell's Q3 2015 money moved and also provides a preliminary estimate of the money moved between February and December 2015. This doesn't include money that might be moved in January 2016, though that is unlikely to materially affect the estimates much since the bulk of money moved is in December. Heishman reports that $28 million in money moved from sources other than Good Ventures is attributable to GiveWell. This is close to the 90th percentile in my estimate range ($28.6 million). So overall, my estimates were quite conservative, and it may have been possible to more accurately make higher estimates. With that said, my identification of the potential reasons for the money moved being a lot higher than my median estimate was broadly correct. As I wrote above:

The upper end of estimates is mainly governed by factors such as wealthy donors, highly successful fundraising groups or celebrity endorsements or media publicity, and other unforeseen factors.

The GiveWell blog post says:

Excluding Good Ventures, we have tracked about $28 million (of which, roughly half has come from donors giving $1 million or more).

Hence, at least the "wealthy donors" part seems right.

Moreover, I expect that the other predictions I made should roughly hold up with appropriate rescaling to the greater amount of money moved, but also accounting for the increased variability due to a small number of large donors.

UPDATE 3: Catherine Hollander linked to almost-final estimates of money moved by charity and size bucket. The total non-Good Ventures money moved was about $33.37 million (versus about 13 or 14 million last year, and a 50% confidence interval of $16.2-24.5 million in my post). Within the distribution specified in my post, the amount actually moved fell at the 97.5th percentile.

Size buckets: Money from those giving over $100,000 (including both the $100,000-$999,999 bucket and the $1,000,000+ bucket) was about $20.29 million, versus $4.42 million last year. Excluding these heavy donors, the money moved was $13.4 million, compared to about $8.4-10 million last year. The money moved per donor was $2,157.

I believe my primary source of error in estimates was at the high end. I correctly predicted that the money moved per donor would increase, but I underestimated the extent to which the total money moved by donors giving in excess of $100,000 would increase. If the giving in these buckets had increased in the same proportion as the giving in the sub-$100,000 buckets, the money moved would have been within my 50% interval. With that said, I also underestimated somewhat the sheer volume of donors: my 50% estimate for the number of donors was 9,500-12,500, whereas the actual number was 15,468.

Donations by recipient charity: Here, too, my estimates were off, but not systematically so. Three of the four top charities (Against Malaria Foundation, Schistosomiasis Control Initiative, and Deworm the World Initiative) fell in my 50% intervals for those charities (with AMF, at almost 15 million, being at the higher end of its range but still within it). The one outlier was GiveDirectly. My 50% estimate for GiveDirectly was between 1 and 4 million dollars, but actual money moved appears to have been over 13 million dollars. This is interesting in that it was unexpected not only to me but also (probably) to many other GiveWell watchers. It was also contrary to the message that GiveWell was focused on spreading, namely, that donating to AMF should be the top priority.

UPDATE 4: Natalie Stone-Crispin of GiveWell has just published a blog post on the GiveWell blog providing insight into GiveWell's research plans for 2016. Some of these updates are related to predictions I made, and make me more confident that my thinking was in the right direction. In my Facebook post that had been the original impetus for this Effective Altruists group post, I had said that GiveWell's top charities are unlikely to change in 2016, and Natalie's just-published post strengthens my view that the top charity list is likely to stay the same (with possible temporary removal of some top charities because of exhaustion of room for more funding or concerns about scaling challenges). GiveWell is also looking into switching focus from identifying top charities to outreach, something that I had not made a prediction on in either direction.

UPDATE 5: On Friday, May 13, 2016, GiveWell finally released the final report on money moved in 2015. I have published a new post going over my forecasts, how they compared with reality, and implications.

Disclosure: I am not financially affiliated with GiveWell, but have some past and ongoing association with them. You can get a detailed account of my past interaction and affiliation with GiveWell here.

9 comments

Comments sorted by top scores.

comment by Carl_Shulman · 2016-01-08T17:41:51.284Z · score: 2 (2 votes) · EA(p) · GW(p)

GiveWell writes that it has tracked ~$28 million in non-Good Ventures for 2015 (since February 2015), and they "plan to publish [their] annual metrics (covering February 1, 2015 – January 31, 2016) in March". So a top quartile outcome for the revised predictions. Again, I commend you for putting up careful predictions where others did not.

Estimate for 2015: Median estimate of $18.7 million, 50% probability estimate of $16.2 million to $24.5 million.

Estimate for 2016: Median estimate of $25.3 million, 50% probability estimate of $17.5 million to $37.5 million.

Estimate for 2017: Median estimate of $30.1 million, 50% probability estimate of $19.5 million to $58 million.

comment by Carl_Shulman · 2015-12-19T23:18:25.369Z · score: 2 (2 votes) · EA(p) · GW(p)

Thank you for putting all of this quality research effort and analysis, with concrete predictions, into this topic. This is a high-value information-rich post.

Have you looked at the historical time-series relationships between your web metrics and quarterly # of donors?

comment by vipulnaik · 2015-12-20T00:14:52.311Z · score: 4 (4 votes) · EA(p) · GW(p)

I did, and didn't find a strong relationship. That was one of the reasons why I shifted away from the original hypothesis where discovery and brand awareness directly lead to donations. Now I think of them as more like the first steps in the pipeline, with the actual donation happening so much later that it's hard to obtain any kind of reliable time series correlation other than what you'd get between two increasing things.

The web metrics don't even correlate well with each other more than what you'd expect any two generally increasing things to: the Wikipedia views, Google Trends and the website unique visitors all seem to have minds of their own.

comment by Robert_Wiblin · 2015-12-19T20:35:50.166Z · score: 2 (2 votes) · EA(p) · GW(p)

This is very impressive Vipul. I wonder if it could do with a TL;DR summary?

comment by vipulnaik · 2015-12-19T20:37:51.100Z · score: 0 (2 votes) · EA(p) · GW(p)

No, any TL;DR summary will be subject to ridicule for being simplistic and misleading.

comment by Owen_Cotton-Barratt · 2015-12-20T09:20:46.950Z · score: 3 (3 votes) · EA(p) · GW(p)

I wonder if it's worth trying anyway. People will inevitably go away from reading the piece remembering only an overview, and you can probably craft a better and more ball balanced overview than they would produce by themselves. The summary also serves as an abstract, letting people make a note informed decision about whether to read it in the first place.

comment by vipulnaik · 2015-12-20T17:23:33.832Z · score: 1 (1 votes) · EA(p) · GW(p)

Okay, I added a TL;DR, with the caveat that it may be misleading.