Thanks, Ben! That all seems fair enough for these purposes.
Fwiw, I think that number might be more the arithmetic mean of some observations. Interpreting it as a geometric mean seems like it doesn’t strongly violate much, but I think the geometric mean is going to be a little bit lower.
But yeah, I doubt it makes much of a difference in the scheme of things!
Thanks for writing this! I found it a really interesting series and kudos to you for sharing earlier-stage thinking publicly. I definitely know that I can find sharing such thinking pretty daunting!
>ACE estimates that the average vegetarian stays vegetarian for 3.9-7.2 years, implying a five-year dropout rate of 53%-77%.
I think that 3.9-7.2 is their estimate for (i) the averagevegetarian adherence length, but you might be interpreting that here as more like (ii) the medianlength of vegetarian adherence?
From that ACE report:
> We can see in the Guesstimate model that after multiplying, the length of adherence for current vegetarians is 32–49 years.
And then they calculate the length for former vegetarians using the following as a basis:
I’d say the underlying distribution here is pretty skewed, so the difference between that average vegetarian length and median vegetarian length might be pretty significant.
So I guess my pretty quick sense is that the median vegetarian adherence length may be a fair bit shorter than 3.9 - 7.2 years. And if you are interpreting that 3.9-7.2 as being more about the median length, and it was in fact a fair bit shorter, then that could meaningfully change some of your conclusions here.
I think that is all somewhat nitpicky though, and I could certainly be wrong about it! Regardless, thanks again for sharing all of this. :)
Here’s a relevant thread from ~5 years ago(!) when some people were briefly discussing points along these lines. I think it illustrates both some similar points and also offers some quick responses to them.
Please do hit see in context to see some further responses there!
And agree, I would also like to further understand the arguments here :)
Sure! Here are some of my quick(ish) thoughts that don’t necessarily represent those of others on the fund:
Generally not wanting the fund to be more than ~50% of any group's budget. That could cause over-reliance on the fund, hurt their fundraising efforts with other funders, and possibly disincentivize other funders from contributing to promising groups.
Larger groups often do a variety of programs, some of which may be much less impactful. There’s some reason to be wary of funging less impactful programs and that may generally lean one away from funding larger groups who are capable of easily absorbing grants of more than say $100k.
My sense is that smaller and medium sized groups aren’t able to absorb a significant amount of additional funding without significant diminishing returns. E.g., it seems that a lot of groups in our space have experienced really significant growing pains from trying to scale too quickly.
Believing a number of promising opportunities are in low and middle income countries. Dollars can quite go far overseas and RFMF of small-medium size international group can easily be significantly filled by a $10-$30k grant. It also seems important to build up a now nascent movement in a large number of countries.
Our granting cycle is every four months and we often fund some groups in multiple payouts over a year. E.g., while the meta fund may only make essentially one grant per group in a 12monthgranting cycle, we might make 2-3 grants to numerous groups over thesameperiod. Looking at individual payout reports might give the impression we are inclined to split funding across a large number of groups, perhaps significantly more so than viewing grant totals over a 12 month period.
I think there’s some comparative advantage reasoning going on, because we want to add value to what can easily be achieved elsewhere. For instance, it is much easier for us to fund international groups than it is for small individual donors. We might also be more risk-neutral than a lot of other funders in the space, which can lend itself to funding less-established groups. Some other funds and funders also seem to focus more on promising groups that have scaled, and given their allocation of funding it can make most sense for us to focus on smaller opportunities.
I think all those thoughts might go some way in explaining the apparent split of funds across a relatively large number of grantees.
Thanks for writing this up Saulius! I think it is a really useful addition to the literature on EAA and I could see myself returning to it multiple times in future. You seem good at writing such content! :)
Some thoughts that I had after reading this piece:
- I think there’s a decent chance that if one were to dive deeper into captive invertebrates then this could lead to discoveries of tens of billions of animals that are in captivity that the movement currently largely neglects
- One important point I think worth highlighting about the numbers is their differential growth rates. That is, for instance, not only are there many more farmed fish than pigs or cows but the annual increase in the number of farmed fish is much greater than that for pigs or cows
- “Captivity” seems a binary distinction applied to an underlying continuum of something like “the degree to which people control an animal’s habitat.” I wonder if there are some edge cases that could significantly impact the numbers reported here. For instance, and this could certainly be stretching the definition of “captivity” but if fish ladder-type structures were included then that could be another significant source of fish in captivity, even if each fish only spends a small amount of time in them
- I agree with the update towards China being even more important than previously thought given numbers of quail, frogs, and turtles. Relatedly, something that feels important is most, if not all, of the five countries with the most farmed vertebrate animals are Asian countries
Sorry for my slow reply! I think that I missed the notification for this.
You’re right I accidentally linked the wrong article. IIRC, this was the article that I should have linked. I believe that it outlines the high-moisture twin-screw extrusion method, a method which decades later proved important for the Beyond Burger and the Impossible Burger.
I hope this helps! Would be curious about any takes you have in this area.
[Fwiw: I previously worked at ACE and now work at Farmed Animal Funders. I'm also on the committee for EA Animal Welfare Fund]
Thanks for writing this, Ben! It is an interesting analysis. Here are some thoughts that I had while reading:
What could also be interesting would be looking at the concordance/ disconcordance between some of the regranting funds in this area (e.g., EAA fund, EA Animal Welfare Fund, and OWA.)
What could also be interesting for Open Phil and ACE is comparing and contrasting the funding allocations across the concordant groups.
Some of the discordance between Open Phil and ACE might also be explained by different views about the potential of cultivated meat, different overall capacities for funding, and different approaches on restricting funding.
Lastly, two quick notes:
Three charities which were named “Standout Charities” by ACE but did not receive Open Phil grants did receive grants from the Centre for Effective Altruism’s Animal Welfare Fund (Animal Ethics, Faunalytics, and Compassion in World Farming - USA).
I think Compassion in World Farming - USA has received three Open Phil grants.
None of the charities ACE comprehensively reviewed but did not recommend have received a grant from Open Phil.
I think Compassion in World Farming International has been comprehensively reviewed by ACE and then not recommended but have received several Open Phil grants.
Thanks for completing this analysis on the advantages and disadvantages of the INT framework! I particularly like you clearly enumerating your points.
I think there are some important points not adequately covered in the alternative INT framework and discussion of cost-effectiveness estimates. Namely:
(1) To a significant extent cause prioritization involves estimating long-term counterfactual impacts
(2) Neglectedness could be instrumental to estimating long-term counterfactual impacts because the more neglected a cause the more potential to translate to greater far future trajectory changes, as opposed to accelerating proximate changes
I think that your general model is wrong. Briefly, here’s a couple of reasons why:
First, producers strongest economic incentive is net-profit maximization.
Net-profit= #fish sold * (average revenue per fish sold - average cost to farmer per fish sold)
Farming fish at quite high stocking densities without counteracting aeration causes low dissolved oxygen levels. These high stocking densities cause a greater number of fish to be sold. As long as the increase in net-profit caused by the increase in the # fish sold is greater than the decrease in net-profit caused by the decrease in marginal profit per fish, farmers have economic incentive to do that. Therefore farmers would have their strongest economic incentive be in favor of some negative outcomes caused by lower oxygen levels as long as these were outwieghed and lead to them increasing their net-profit, namely through them being able to farm a greater number of fish.
Second, fish farming is still a young and rapidly evolving industry so farming practices may not totally align with economic incentives. One sentiment I have heard expressed is that for fish welfare there still are a lot of either welfare and economic wins-wins or win-ties. That is, there are opportunities for fish welfare to be improved without costing economic productivity.
Third, regardless of the incentives or possibility of welfare and economic win-wins or win-ties, it still empirically seems true that a) many farmed fish seem subject to sub-optimal dissolved oxygen levels and b) mass die-offs are not rare. Given the frequency of sub-optimal dissolved oxygen and mass die-offs this is probably evidence farmers currently don’t overall have very strong incentives to prevent these issues. So, while there is limited evidence in general on the topic, here are some of the counterexamples to the general model you seem to propose:
In channel catfish, the most farmed fish in the US, “[t]he traditional pond system typically produces 4,500–5,500 kg/ha of catfish with a maximum of 7,000 kg/ha (Brune, 1991; USDA, 2006). However, today, many farms in Alabama produce more than 10,000 kg/ha, and the amount of aeration provided is not adequate to consistently maintain minimum dissolved oxygen (DO) concentrations above 3 mg/L (Boyd and Hanson, 2010).” (Brown (2011), p. 72)
E.g, “For catfish, one of the major causes of significant die-offs is low oxygen, while oxygen levels are something that can be feasibly controlled.”
E.g., “Caged salmon companies have reported over 760 mass deaths to the Scottish Government in the last three years.” This article reports that oxygen shortages are a frequent killer.
E.g., The report linked in the OP estimates pre-slaughter mortality rates of 15%-80% for commonly farmed fish over the entire production cycle.
Thanks for writing this up Saulius! I think it is a really useful addition to the literature on EAA. You seem good at writing such content! :)
Some very quick thoughts that I had on this piece:
- My rough impression is that the “pre-slaughter mortality rate” of mice is relatively high. This matches my own experience when I had pet mice when I was younger and a quick google suggests that lab mice mortality seems high. E.g. > We examined the survival rate of 539 litters of mice from two of the most commonly used laboratory strains (C57Bl/6 and Balb/c) bred under normal husbandry procedures, and found that mortality rate (that is whole litters lost) was at average 28,9%.
- My rough impression is some pet snakes feed on eggs or fish predominantly rather than mice. I am not sure how big a proportion does that though but it could be significant. E.g., I think the Gartner Snake is a fairly common pet breed and that it is common to feed them fish.
- I have a feeling that rodents are farmed in larger numbers for human consumption in some asian countries but a quick google didn’t really confirm or deny that.
- I wonder if more mice are fed to captive/farmed crocodiles, alligators, and caimans then to captive snakes. These other reptiles are are much bigger than the average snake and eat more often, and I think it is common to feed them mice. Skimming this and it seems possible that the number of these other reptiles farmed is in the hundreds of thousands.
-I wonder if mice are fed in quite large amounts to captive predatory birds. E.g., this suggests one of these birds eats x5 the amount of mice p/w than a python does.
- I thinks there’s a decent chance that if one were to dive deeper into the farming of invertebrates then this could lead to discoveries of tens of billions of additional farmed animals the movement largely currently neglects.
But in all I mainly think this is an important area that not many have thought about (including me). Thanks for highlighting it! :)
I am also excited about Sentience Institute’s work and look forward to seeing more :)
We also produced Global Farmed & Factory Farmed Animals Estimates, suggesting that around 71% of farmed land animals and probably 96% of all farmed animals globally are factory farmed, and that probably 85% of the farmed animals alive at any time are fish.
The estimate of farmed fish numbers used in that estimate relies on Mood and Brooke (2012). The Mood and Brooke (2012) “estimate does not include the numbers of fish farmed for bait and it does not include fish mortalities arising in fish farms prior to harvest….” (p.2). Those exclusions seem significant. E.g.,
ACE estimates that for the four most consumed farmed finfish in the U.S. the mortality rate prior to slaughter is 18-60%, 5-35%, 10-38%, and 12-65% (as 90% subjective confidence intervals) for salmon, tilapia, pangasius and catfish respectively.
Including those quantities in the overall estimate seems like it will take the percentage estimate of the proportion of all farmed animals that are factory farmed much closer to 100%.
It is probably also worth noting that the Global Farmed & Factory Farmed Animals Estimates does not to include estimates for the number of farmed insects (e.g., silkworms and honey bees.) The sheer number of those insects could have a big impact on the percentage estimate of the proportion of all farmed animals that are factory farmed too!
(1) To what degree did your beliefs about the consciousness of insects (if insects are too broad a category please just focus on the common fruit fly) change from completing this report and what were the main reasons for those beliefs changing? I would be particularly interested in an answer that covers the following three points: (i) the rough probability that you previously assigned to them being conscious, (ii) the rough probability that you now assign to them being conscious and (iii) the main reasons for the change in that probability.
(2) Do you assign a 0% probability to electrons being conscious?
(3) In section 5.1 you write
I’d like to get more feedback on this report from long-time “consciousness experts” of various kinds. (So far, the only long-time “consciousness expert” from which I’ve gotten extensive feedback is David Chalmers.)
David Chalmers seems like an interesting choice for the one long-time “consciousness expert” to receive extensive feedback from. Why was he the only one that you got extensive feedback from? And of the other consciousness experts that you would like to receive extensive feedback from, do you think that most of them would disagree with some part of the report in a similar way, and if you think they would, what would that disagreement or those disagreements be?
(4) A while ago Carl Shulman put out this document detailing research advice. Can you please do the same, or if you already have a document like this can you please point me to it? I would probably find it useful and I would guess some others would too.
There are two big reasons why hiring altruists still makes sense in many cases:
You don’t have any alternative candidates worth hiring, or finding such a candidate would require a large investment.
A particular altruistic candidate looks sufficiently better than the alternative candidate that the difference between candidates exceeds the difference between the altruistic candidate’s value at your organization and their value elsewhere.
Another consideration for hiring an altruistic candidate is that altruistic candidates are more open to lower salaries which then makes it more likely additional hires can be made.
Agree may be a bad idea because there’s certainly a faction on the left which dislikes him. Though there could be reasons for not caring much about what they think.
He is a public intellectual, one of the very few that thinks there’s a legitimate risk of extinction from AI in the near future, is quite utilitarian and thinks non-human animals have significant moral value. It could be argued that the good which comes from him promoting of those ideas would outweigh the negative consequences.
I don’t feel strongly about him being a speaker, but thought it was interesting point to discuss.
IIRC in conversation at EA Global the maker of Speciesism said the total cost of that film was <$100,000 US. I believe he said he was in the process of making another animal advocacy documentary but I am not sure.
Probably also worh noting that:
There’s preliminary evidence (p.23-25) that documentaries could be one of the most effective diet change interventions.
I think Faunalytics is currently considering some type of evaluation of the effect of a documentary on diet.
There’s could already be an adequate supply of animal advocacy (excluding WAS) documentaries. It’s could be better to instead direct resources to promoting existing documentaries rather than funding additional documentaries.
Daniel Irving is currently making a documentary about EA. Little unsure of the specifics involved, but he has interviewed a number of EAs.
You mention the Easterlin paradox a few times in these comments and in you draft paper. I briefly looked into the literature a while ago and I place less weight on the Easterlin paradox than you. Here’s a quick summary of what I found.
Easterlin claimed (p.113-118) that average satisfaction in a country doesn’t increase as a country grows wealthier. Since then there has been a back and forth in the literature but there is now a growing body of correlational evidence that strongly conflicts with Easterlin’s initial claim (for instance, p.3, p.4, p.10, p.12). It seems that the literature now suggests that the relationship between income and life satisfaction is one of diminishing returns but that an increase in income is correlated with an increase in life satisfaction. A nice heuristic to use is that a doubling in income increases subjective well-being by 0.34 standard deviations (p.7).
There has been only limited research into the effect of income on emotional well-being or the affective aspect of subjective well-being. The research there has been suggests that particularly at low levels of income an increase in income correlates with an increase in emotional well-being (p.3, p.8, p.8). A meta analysis on subjective well-being noted that there is a weaker association between income and emotional well-being than income and cognitive well-being (p.3).
Here's another study from Rothgerber which could be relevant:
The present research examined pet ownership, current pet diet, and guilt associated with pet diet among a fairly large sample of non-meat-eaters (n = 515). It specifically focused on the conflict that pits feeding one’s pet an animal-based diet that may be perceived as best promoting their well-being with concerns over animal welfare and environmental degradation threatened by such diets, here labeled the vegetarian’s dilemma. Questionnaire responses indicated that ethically motivated meat abstainers were more likely to own pets and owned more of them than those motivated by health concerns or a combination of ethical and health concerns. Vegans and those resisting meat on ethical grounds were more likely to feed their pet a vegetarian diet and expressed the greatest concerns over feeding their pet an animal-based diet. For vegans and ethical meat abstainers, it is suggested that questions concerning what to feed their pet approaches a tragic tradeoff contrasting two sacred values: protecting the well-being of their pets and protecting the well-being of other animals and the environment. For meat abstainers motivated by health concerns, this constitutes a relatively easy moral problem because the primary concern for such individuals is the health of their pet with less or no regard for other ramifications of the decision, i.e., harming other animals or the environment.
I believe a plausible cumulative distribution function for the probability of extinction would have an asymptote—or else something like an asymptote, e.g., the probability of extinction between 100 and 1000 years from now is about the same as the probability of extinction between 1000 and 10,000 years from now, etc.
Using that example the probability of value existing could be roughly modelled as:
Where p is the probability of value existing n years into the future, r is the extinction probability between 10 and 100 years, log means the log base 10 and n is the number of years in the future. This relationship works for n>2.
I was curious about what the average of p(n) for that type of function would be over the next 10^11 years. Some available extinction estimates put r between 10% and 50%. I imagine there’s also similar variance within EAs’ r value. Using r= 10% the average of p(n) over 10^11 years seems like it would be 3 10^-1 . Using r= 50% the average of p(n) over 10^11 years would be ~7 10^-4. I used Wolfram Alpha’s integral calculator for these calculations and I am not that it’s performing the calculation correctly. These averages for p(n) could make the impact on far future EV significant.
I don’t have strong views on which CPONE model is best and the ones I mention here may be flawed. I softly lean towards including CPONE models because the posterior then more closely reflects the user’s view of reality, it’s not too difficult to include CPONE models, reasonable people may have different CPONE models, and the addition of a CPONE model may result in different cause prioritization conclusions.
I think multi-cellular life on earth makes for a better reference class. Multi-cellular life has taken several big hits over the years, but it's always bounced back.
Interesting. I hadn’t thought of that reference class before :)
When you're looking at interventions' effects on the far future, the numbers are so big that the prior does a lot of work—10^54 and 10^55 expected utility don't look that different after updating on the prior.
Excellent point :) I wasn’t fully taking that into consideration. Updates me towards thinking that CPONE models are less important than previously thought. I think reasonable people could have a CPONE model which causes more than one order of magnitude difference in EV and therefore cause a more significant difference after updating on the prior.
[edited originally I accidentally used the natural logarithm instead of log base 10 when calculating the average of the probability function over 10^11 years]
I think this quantitative model has some potential and it’s a great addition to the growing literature on cause selection. Thanks for taking the time Michael :)
One aspect of this model which I find problematic, and I feel is something that may be often overlooked when calculating the EV of the far future, is that there is some Cumulative Probability Of Non Existence (CPONE) that is currently not accounted for in the probabilities listed in the EV of the far future spreadsheet.
The CPONE relies on the following:
The extinction risk probability per year is always greater than 0 because extinction is possible in any year.
Extinction in any year means extinction in all future years. This property is what makes the probability of non-existence cumulative. By cumulative I mean it increases each additional year it is forecast into the future.
It follows that the probability of value existing x years in the future decreases as x increases.
I don’t have a great sense of what I feel the probability for extinction of humans or their descendants into the far future is but assigning zero probability to this outcome in the far future spreadsheet conflicts with my initial thoughts on this topic. For instance, available estimates seem to put the current annual probability of extinction at ~10^-4 and it seems that even much smaller annual extinction probabilities accumulate over large timescales to become significant.
These probabilities of extinction matter because future EV comes from the ∑(estimated value at future time point multiplied by the probability of value existing at that future time point) for all future time points. If we feel that, say, 10^10 years into the future there’s a 0.5 probability humans or their descendants are extinct then all estimated values after that time point have to be multiplied by <0.5 in order to find their EV.
Given this, I think there’s some chance that the inclusion of reasonable CPONE models into far future EV calculations can cause orders of magnitude difference relative to not including CPONE models.
Please note, I am not sure the points I made in this comment are correct. I haven’t thought about/ researched this much and as such there’s certainly a chance that I will update in future. It’s unclear to me what impact including CPONE on EV of the far future has, maybe one day I will attempt some calculations myself. I currently assign significant probability to it causing orders of magnitude difference and that makes me feel like CPONE should be attempted to be included in models like this. Another solution would be to make it clearer how the model is dealing with extinction probabilities into the far future and how this may conflict with some people’s views.
Which threshold was that and how did you arrive at that conclusion? I don't really know one way or another yet, but upgrading or downgrading confidence seems premature without concrete numbers.
The threshold was a 10% difference in animal product consumption between the experiment and the control group. I arrived at this conclusion because I thought that there was some chance that these ads would cause the experiment group to report a 10% or more decrease in animal product consumption when compared to the control group. Since the study didn’t detect a difference at this level I assign a lower probability to a change of this magnitude being present than I did previously.
A predicted change of 10% or more might have been overly optimistic and I didn’t have a great sense of what I thought the effectiveness of online ads would be prior to this experiment. The ads were targeted at what was thought to be the most receptive demographic and those who click on these ads seem particularly predisposed to decreasing their animal product consumption. You’re right though, upgrading or downgrading confidence might be premature without concrete numbers.
I think there are some other reasons for why I seem to be updating in the negative direction for the effectiveness of online ads. These other reasons are:
I feel that that my lower bound for the effectiveness of online ads also moved in the negative direction. I previously assigned next to no probability that the ads caused an increase in animal product consumption. However the results seem to suggest that there may have been an increase in animal product consumption in the experiment group. So I have increased the probability that I put on that outcome.
ACE also seems to be updating in the negative direction.
I did a very rough and simple calculation in this spreadsheet using that the experiment group would have 1% of people reduce their animal product consumption by 10%, 1% of people convert to vegetarianism and .1% of people convert to veganism. I don’t put too much weight on this because I did do these calculations after I had already somewhat committed to negatively updating in this post which may have induced a bias towards producing negative results. Still, this suggests that something like my best guess was systematically too positive across the board.
On this last bullet point I wonder if there is a way that we can do a bayesian analysis of the data. If we were to set our prior and then inform it with the results from this experiment. It would be very interesting to see if this would cause us to update.
It seems unfair to deallocate money from online ads where studies are potentially inconclusive to areas where studies don't exist, unless you have strong pre-existing reasons to distinguish those interventions as higher potential.
I think we agree that if the study is inconclusive it shouldn’t cause us to change the allocation of resources to online ads. However, I think if the study causes updates in the negative direction or positive direction about the effectiveness of online ads this is reason to change the allocation of resources to online ads. I currently interpret the study as causing me to update in the negative direction for online ads. I think this means that other interventions appear relatively more effective in comparison to online advertising compared to my prior views of their effectiveness in comparison to online advertising. This seems to be reason to allocate some increased amount of resources to these other interventions and some decreased amount of resources to online ads.
“Later, Edge Research will complete “a final data report including (a) an outline of the research methodology and rationale, (b) high level findings and takeaways, and a (c) drill downs on specific areas and audiences.
It’s currently unclear what precise methodology Edge Research will use to analyze the data, but the expectation is that they would use a Chi-Square test to compare the food frequency questionnaires between both the treatment and control groups, looking both for meat reduction and elimination.”
Or are you wanting them to have committed to something in the analysis that they didn't talk about there?
Yeah I would have liked a more detailed pre-analysis plan. I think there was perhaps too much researcher freedom in the data analysis. This probably makes questionable data analysis techniques and inaccurate interpretation of results more likely. Some things that I think could have been useful to mention in a pre-analysis plan are:
Information about the data weighting process.
How incomplete survey responses will be treated.
How the responses of those who aren’t females aged 13-25 will be treated.
KG: “Would it be better to do pre treatment/intervention and post treatment/intervention data collection rather than just post treatment/intervention data collection for future studies?”
JK: “The idea is, have a third, smaller, group that went immediately to a survey? That's a good idea, and not that expensive per survey result. That helps you see the difference between things like whether the video makes people more likely to go veg vs reduces recidivism.”
The idea you suggest sounds promising but it’s not what I meant. With my initial question I intended to ask: Would it be better for future studies to have both a baseline collection of data prior to intervention and an endline collection of data sometime after the intervention rather than just an endline collection of data sometime after the intervention? I ask because my general impression is the standard practice for RCTs in the social sciences is to do pre and post intervention data collection and there’s likely good reasons for why that’s the case. I understand that there may be significant costs increases for pre and post intervention data collection relative to just post intervention data collection but I wonder if the possibly increased usefulness of a study’s results outweigh these increased costs.
The Edge analysis doesn't look useful to me, since they didn't do anything that unusual and there are lots of people in the community in a position to analyze the data. Additionally, my impression is that working with them added months of delay. So I certainly wouldn't recommend this in the future!
Sounds like we probably have pretty similar views about the limited value of Edge’s collaboration. I also probably wouldn’t recommend using them in future.
My guess is they just failed to bring enough people back in for follow up through ads.
That makes sense as a likely reason why the study was low powered. I wonder if alternative options could have been explored when/if it looked like this was the case to prevent the study from being so low powered. For instance, showing more people the initial ad in this circumstance could have led to more people completing the survey which would have likely have increased the power of the survey. Although it may have been difficult to do this for a variety of reasons.
You take the combined experimental and control groups and you figure out for each characteristic (gender, country, age range) what the distribution is. Then if you happened to get extra UK people in your control group compared to your experimental group, instead of concluding that you made people leave the UK you conclude that you happened to over-sample the UK in the control and under-sample in the experimental. To fix this, you assign a weight to every response based on for each demographic how over- or under-sampled it is. Then if you're, say, totalling up servings of pork, instead of straight adding them up you first multiply the number of servings each person said they had by their weight, and then add them up.
Thanks for explaining this- it’s much clearer to me now :)
Why would CCTs have a larger counterfactual impact than other interventions? This seems like an important point to make explicit, both for you and for everybody else.
Without going too in depth some of the reasons we think this are:
The field is relatively uncrowded.
A conditional cash transfer charity has relatively high potential scalability.
There appear to be a number of relatively evidence-based and cost-effective conditions that a conditional cash transfer charity could base itself upon.
A conditional cash transfer charity seems like it would be able to update on new information at a faster rate and to a greater extent than charities based on most other intervention areas.
This wasn’t included in the original post was because we felt a shorter post would be able to generate useful feedback.
My gut says that administering the costs & monitoring the behavior you're promoting in a CCT program (depending on what that is) may cost more than simply giving out vaccines, vitamin-A supplements, etc.
Your gut could be right :). My understanding is that in some areas the demand for some health interventions may be lagging behind the supply of those health interventions. For instance, this article and this article suggest major reasons for partial or no immunization in India are demand side. In those circumstances it’s plausible that conditional cash transfers could be a very cost-effective intervention and perhaps more cost-effective than supplying vaccines or micronutrients.
It also seems like there are more ways to mess up a CCT intervention than a simple direct service intervention.”
This could be true. At the moment we aren’t highly confident in our understanding of the relative logistical difficulty of different interventions. A consideration like this may make us update away from conditional cash transfers in future.
“HOWEVER, all of this is bracketed with a huge disclaimer: just go talk to an expert who knows more.”
Okay will do :)
I'd add "graduating additional grades" to the list of potential conditions.”
Sure. That’s something we will consider although we are unsure what the returns to schooling in low income countries are. For instance, the 2009 GiveWell Developing-world education (in-depth review) observes that there is little reliable information regarding the true relationship between schooling and later-life outcomes such as income.
Not at the moment. We’re currently near the beginning of a shallow review of conditional cash transfers and we haven’t wrote our research up in a form that people can comment on because we have found this process to be really time consuming. We also feel that some of the best feedback may be gathered early on in the research process by less time consuming posts like this as well as direct conversations/ email exchanges with specialists.
Ideally, in the future we will have important aspects of research written up in a form that people can comment on but at this stage it isn’t clear if that will include research on conditional cash transfers.
[Softcore EAs] identify as EAs and donate money and time to effective charities, but otherwise lead regular lives, as opposed to devoting the brunt of their resources to advance human flourishing as do hardcore EAs.
Hardcore EAs can also devote the brunt of their resources to advancing the flourishing of all sentient animals :)
Even if you only focus on donations that have already been made, and ignore pledges, GWWC has a high positive multiplier. Moreover, completely ignoring the future value of pledges would be really pessimistic.
I totally agree and think that a really interesting question is what the future value of pledges should be. I think may also be worth mentioning that if we focus on donations that have already been made, my understanding is GWWC’s impact is an order of magnitude less than their current realistic impact estimate. I am not sure how exactly we should weigh that information.
With CS you should value the future value of legacy commitments, even though they will take 20-60 years (though you'd need to apply discounting). (Presumably you do give them positive value even though they're a long way in the future? :))
At CS we sure do value the future value of legacy commitments :). We haven’t yet determined exactly how we will calculate their expected value.
Thanks so much for providing such detailed answers to my questions :). I think that I am getting a much better idea now. Thanks Sam for linking me to the comments of GWWC’s fundraising prospectus- the discussions there cover most of my questions. If it’s not too much to ask I would really like it if you answered a few more questions that I have.
Just to follow up on Sam’s answer to my first and second questions.
To help me understand how I should feel about the ratio GWWC uses for actual donations to pledged donations could you please outline what percentage of members record enough information in My Giving for their information to be used in calculating the ratio of actual donations to pledged donations that was used in GWWC’s most recent realistic impact estimate?
To clarify, are you talking about the impact of changes to members' income over time, or asking whether we're accounting for potential changes to donation patterns over time which affect the counterfactual ratio?”
I was talking about accounting for potential changes to donation patterns over time which affect the counterfactual ratio- sorry that wasn’t clear :). I certainly agree that it’s really hard to know how accurate people’s counterfactual estimates are. I wonder what the best way of doing estimates for counterfactuals is. Do you guys think that it might be more accurate to ask people each year about their counterfactual estimates rather than asking people once and then using that number for the next 40 years to calculate the counterfactual impact of GWWC?
One last question. GWWC seems to be the only EA fundraising organization I know of which calculates its potential impact for 40 years into the future. Do you guys think that other EA fundraising organizations like Charity Science and Raising for Effective Giving should attempt to do this type of realistic impact calculation for 40 years into the future so that donors who are trying to decide between these organizations may be able to more accurately compare the different organizations?
On a sidenote, I think that up and down votes of comments provide a really valuable source of feedback. As a result I am interested if people have ideas about the possible reasoning behind the down vote or down votes of my previous comment so that I can hopefully improve my future comments :).
Thank you so much for answering questions like this. I think it’s really worthwhile :). When you have a free moment it would be great if you could answer these questions. Unlike Peter I have only 6 questions:
2.) On what information is the ratio of actual donations to pledged donations used in GWWC’s most recent realistic impact estimate based upon?
3.) Is the current technique that GWWC uses to calculate counterfactuals more likely to overestimate or underestimate GWWC’s impact given that the counterfactual percentage of all future donations is estimated when people initially take the pledge rather than when they make their donations in subsequent years?
6.) It seems that GWWC’s comparative advantage is in generating and maintaining pledges for effective charities. Given that there is a large overlap between GWWC’s and GiveWell’s charity recommendations and GiveWell being in a better position to continue charity evaluations, why is it worth GWWC continuing charity evaluations?
Any additional tips for people who ask for charitable donations several times throughout the year?
To avoid donor fatigue and possibly damaging relationships whilst still optimizing for donations:
Limit the number of asks you do per year to only those with the highest expected value.
Ensure that you have many interactions with individuals that don’t involve an ask for each interaction you have that does involve an ask.
Try to discuss effective charity in contexts where it’s clear you’re not asking for a particular donation to a fundraiser, but just happy to answer any questions they might have, and help them choose the charity they want to support.
Try to personalize asks as much as possible and ask in a non-confrontational manner.
Try not to sound repetitive.
Personally thank everyone who donates and do it relatively quickly after they donate.
The reasoning outlined in this post forms a significant part of why some of the Charity Science staff and board members will pursue founding an evidence-based, cost-effective charity, aiming for it to become GiveWell recommended.
To answer your questions:
Is this post an announcement of Charity Science's (CS) intention to explore the option of direct charity entrepreneurship?
No, Charity Science will continue with it’s usual activities.
Are some CS staff or board members already intending to found new charities, and this post serves as the rationale? Or, is this post just a general essay explaining why anyone outside of CS might also consider founding a new effective charity, if they are also well suited to found such a charity?