EA Malaysia Cause Prioritisation Report (2021)

post by yiyang · 2021-04-24T05:48:58.731Z · EA · GW · 8 comments

Contents

  Summary
  Disclaimer
  Research process
  Findings
  Implications
  Learnings
  Contact
None
8 comments

Summary

Disclaimer

We think it’s important to list the limitations of our research, so that readers can make better judgements on this research. 

Research process

We emulated a large part of our methodology based on Charity Entrepreneurship’s research process. However, we also reduced a large chunk of CE’s research process into a more manageable methodology due to a lack of capacity and research experience. Some key differences include:

Here is our research process: 

  1. Idea generation stage
    1. We brainstormed for ideas, which produced 43 cause areas.
  2. Initial ranking stage
    1. We brainstormed for factors that we wanted to use for weighing the cause areas. Most of the factors can be found here.
    2. Each researcher weighed the factors using a 100-point distribution privately and independently; then aggregated the weightages using the median. The top three weighted factors, which were used for the initial ranking of cause areas, were neglectedness, cost-effectiveness, and logistical difficulty.
    3. We evaluated the 43 cause areas using the top three weighted factors and a 5-point Likert scale privately and independently; then aggregated the scores using the median. This evaluation was done without any additional research on any cause areas.
    4. We picked the top 12 cause areas from the initial 43 cause areas. We chose 12 mostly due to capacity constraints.
    5. We then combined the several cause areas into eight due to overlapping definitions.
  3. Weighted factor model stage
    1. We chose 12 factors that we wanted to use for weighing the top eight cause areas, and grouped them into two categories.
      1. Overall impact: evidence base, cost-effectiveness, neglectedness, logistical difficulty, metric focus, flexibility, flow through effects, timing, and risk of negative or no impact
      2. Limiting factor: scale limit, and funding availability, and talent availability
    2. We weighed the 12 individual factors using a 100-point allocation privately and independently; then aggregated the weightages using the median. We conducted a brief discussion to recalibrate our weightages.
    3. We  weighed the two categories (overall impact and limiting factor) 60% and 40% respectively.
    4. We conducted a 1-hour literature review for each of the eight cause areas, and evaluated them the 12 factors on a 5-point Likert scale. This research and evaluation was done privately and independently.
    5. We then aggregated our individual scores using the median.
  4. Recalibration stage
    1. We conducted four 1.5-hour long discussions for each cause area and factor that had more than a two point difference in scores (i.e. we didn’t have similar conclusions). These discussions were used for understanding new evidence and perspectives from one another, which were subsequently used to update our beliefs and recalibrate our scores.
    2. We then aggregated our individual scores using the median.

Findings

Cause areasRankOverallDefinition
EA community building

1

71.22%

Interventions that form a community of individuals interested in EA, as well as interventions that increase the chances of people taking significant actions on maximising personal impact (e.g. volunteering for EA groups, effective donations, local priorities research, high impact career planning).
Farmed animal welfare (land)

2

70.76%

Interventions that reduce the suffering of land farmed animals 
Mental health

3

70.56%

Interventions that improve the mental well-being of individuals.
AI risks

4

65.13%

Interventions that reduce harm caused by the use of AI-related technology, from current AI capabilities to superintelligence capabilities. This also includes existential risks caused by AI-related technology. (further reading [EA · GW])
Financial literacy

5

64.62%

Interventions that improve the set of skills and knowledge that allows an individual to make informed and effective decisions with their financial resources.
Family planning

6

61.17%

Interventions that improve the ability of individuals and couples to anticipate and attain their desired number of children, as well as the spacing and timing of their births
Farmed animal welfare (aquatic)

7

56.74%

Interventions that reduce the suffering of aquatic farmed animals.
Improving diversity and inclusion

8

39.25%

Interventions that reduce harm causing from unjust and unfair discrimination.

For more detailed scores, you can find them here.

Implications

Learnings

This research did not generate highly decision-relevant insights at this time. However, it did confirm our initial intuition (or increase our confidence) about which cause area is most important. 

We should have been more specific with our research goals in the initial exploration stage by forecasting the value of information that we expected this research to generate. We should have then compared and prioritised which projects to take on with our limited capacity. We suspect doing this initial exercise would have helped us make a more informed decision and set better expectations. 

EDIT from Yi-Yang: please do check out Jamie Harris's comment and my response below. I think it incapsulates learnings that I wasn't able to write down in our forum post submission. Basically, I wouldn't recommend using this methodology as the first iteration of local priorities research for groups.   

Contact

If you’re interested in discussing more about our findings, please comment below or reach out to us at our Facebook page. You can also reach out to us by filling up this form.

8 comments

Comments sorted by top scores.

comment by BrianTan · 2021-04-24T13:39:21.233Z · EA(p) · GW(p)

Thanks so much for publishing this report Yi-Yang! I am really glad there is now one report written up properly on the EA Forum about an EA group's local priorities research.

I haven't been able to write a proper report yet for EA Philippines's local priorities research, but I hope to do that within the next few weeks or months.  (If anyone is interested in reading about our work on this though, I wrote a bit about this in our 2020 annual report [EA · GW]).

Your research process is more structured than ours,  and I like the idea of independent research and aggregating people's ratings, which we aren't doing (or haven't done yet). 

I think it would be a lot better though if you had "problem profiles" like 80,000 Hours's for those causes you listed, especially the top 2-4 causes. Maybe that's something to work on next, as something you can use to convince more Malaysians why those causes should be worked on. This is what we're doing at EA Philippines, and it also helps you easily give people resources they can read if they're interested to learn more about that cause area.

Or if not making full problem profiles, putting a few sentences or bullets about the scale and neglectedness of each of the causes would help. For example, you could put the number of farmed land animals and farmed aquatic animals in Malaysia slaughtered/produced every year, or the estimated % of Malaysians with a mental health disorder. You could also try putting a few sentences or bullets about tractability, but that's a lot harder to talk about or research about.

About your actual cause area rankings, my comments are:

  1. I think 6 out of the top 8 causes you put make a lot of sense for why they're there. The 2 that I think are very questionable though are financial literacy and improving diversity and inclusion. I don't see why these two could be in the top 8 causes for Malaysia. Maybe one of you could make the case for why these two causes are very impactful to work on, especially compared to other alternatives I list below?
  2. I've listed three causes notably not in your top 8  that I think have a strong case for being there. I'd be curious to hear if you considered them (and what their scores were if you considered them):
    1. Biosecurity and pandemic preparedness in Malaysia - If AI risks are there, I think this should be on the list too. I think any sufficiently large or rich country could probably do better work on biosecurity and pandemic preparedness.
    2. Improving institutional decision making in Malaysia - I assume that helping top institutions, especially the government, make better decisions is going to be very impactful in most countries around the world. Even just recruiting better talent into government, or finding ways to improve the government, are probably big in expected impact.
    3. Health and development in the poorest areas of Malaysia - if family planning is on your list, why isn't this on the list too? Cost-effective and evidence-based GiveWell interventions like cash transfers, deworming, vitamin a supplementation, and community health interventions might still be important and very cost-effective to do for the poorest areas of Malaysia.
  3. I think 2 other causes that you could consider looking into, but might not make it to your top 8,  are the following. I'd be curious to hear if you considered these causes:
    1. Climate change
    2. COVID-19 response in Malaysia (this relates to biosecurity and pandemic preparedness, and also improving institutional decision making, but is more short-term and specifically about COVID).
Replies from: tzukit, yiyang
comment by tzukit · 2021-04-25T08:06:31.579Z · EA(p) · GW(p)

thank you for your thoughtful feedback Brian! i'm unconfident to speak on behalf of my team so will bring up your feedback during our next meeting, notably on having problem profiles and the potential need to extend the report to clarify each cause area further (the present ones and some big absents). 

comment by yiyang · 2021-04-27T13:16:22.995Z · EA(p) · GW(p)

Hi Brian! Thank for your response.  I'll be using "we" (as a team) to address most of your comments, and "I" at the end to address one point. 
 

I think it would be a lot better though if you had "problem profiles" like 80,000 Hours's for those causes you listed, especially the top 2-4 causes.  

Yes if there is a case for conducting further research, we are definitely considering deeper research in the top causes, and producing “problem profiles”. 
 

Or if not making full problem profiles, putting a few sentences or bullets about the scale and neglectedness of each of the causes would help.

We realised that our last point at the disclaimer didn’t make clear an additional related issue, which addresses this concern of yours. We didn’t detail which piece of evidence or arguments that made us give a certain score. Technically we did - it’s probably somewhere in our meeting minutes and it’s very messy - hence we’ve decided not to address this issue at this time. However, if we were to conduct another research like this, we definitely want to be better at making explicit our assumptions, evidences, and arguments.

The 2 that I think are very questionable though are financial literacy and improving diversity and inclusion. I don't see why these two could be in the top 8 causes for Malaysia. Maybe one of you could make the case for why these two causes are very impactful to work on, especially compared to other alternatives I list below?


We actually found a huge variance of scores for the above two causes areas in both the initial ranking stage and weighted factor model stage. So some of us in our team do agree with you that these cause areas shouldn’t be in the top 8. It also might be the case that we didn’t brainstorm enough cause areas that may reach the top 8. 

As a side note, most of us in our team have a lot of strong feelings with diversity and inclusion issues in Malaysia (although some of us did put a lower score for this cause area, we weren’t that surprised it made it in the top 8). In a nutshell, issues of race and religion are often used as a dividing force within Malaysia at the legislative, political and social level in much of Malaysia’s modern history. 

On a personal note, I wouldn't be surprised if these two cause areas actually do drop out in the next iteration of research (unless there's really convincing evidence of a cost-effective intervention).  

Would love to check out EA PH's cause prioritisation report soon! :)

comment by Jamie_Harris · 2021-05-08T19:26:28.319Z · EA(p) · GW(p)

I imagine that this was a useful process for you guys to think through cause prioritisation research and relevant considerations. Looking at your methodology though, it seems as if you were attempting to essentially redo EA cause prioritisation research to date from scratch in a short timeframe?

My guess of the most useful process would have been to just take some of the most commonly / widely recommended EA cause areas (and maybe a couple of other contenders) and try to clarify how they seem more or less promising in the Malaysian context specifically.

If you agree with my characterisation of your process, with the benefit of hindsight, would you recommend that other national groups follow your methodology or my suggested alternative?

And, a somewhat separate question: what sorts of considerations do you think differ between Malaysia and other contexts in which these questions have been considered, if any?

Replies from: yiyang, yiyang
comment by yiyang · 2021-05-14T05:37:46.678Z · EA(p) · GW(p)

You've made some good points that I didn't get to write in our forum post, and I've made an edit to direct readers to your comment. 

comment by yiyang · 2021-05-14T05:30:33.982Z · EA(p) · GW(p)

Hi Jamie!

Looking at your methodology though, it seems as if you were attempting to essentially redo EA cause prioritisation research to date from scratch in a short timeframe?

My guess of the most useful process would have been to just take some of the most commonly / widely recommended EA cause areas (and maybe a couple of other contenders) and try to clarify how they seem more or less promising in the Malaysian context specifically.

If you agree with my characterisation of your process, with the benefit of hindsight, would you recommend that other national groups follow your methodology or my suggested alternative?

Yes I agree. I think national groups should highly consider that their first iteration of local priorities research be that - taking recommended EA cause areas and conducting shallow research on them. 

That's what we did EA Singapore, although not in a very deliberate way. Once that's out of the way, it saved a lot of time for deeper, more useful research.  Here are the reasons why I think EA Malaysia chose to do this instead: (a) we wanted to test out a methodology, (b) we want to have a stronger consensus in our team when many felt some non-EA recommended areas should be included, and (c) we wanted to be sure we didn't miss any potential promising cause areas out. 

I don't think they are good reasons per say, but I just wanted to put them out there. 

what sorts of considerations do you think differ between Malaysia and other contexts in which these questions have been considered, if any?

I'm not sure if I understood your question correctly, so please do let me know if I didn't.  

I don't think there's any significant considerations that are different, since most of these considerations (or the specific methodology) are from Charity Entrepreneurship. If I were to compare CE's methodologies to other methodologies used by other organisations, I imagine it would be significantly different. 

comment by zeshen · 2021-05-09T06:10:50.800Z · EA(p) · GW(p)

Thanks a lot for the work and the report. Interesting findings you got there. 

I have no experiences in EA so i'm just thinking out loud here - apologies in advance for my ignorance. I'm wondering if we have good reliable statistics on causes of deaths in the country (death being a proxy for suffering), and we could look into the categories of avoidable deaths (e.g. curable illnesses) and whether those areas are receiving enough support / funding. Also, from a poverty perspective, I'm curious if we have an idea how many Malaysians live in hardcore poverty and what can be done to get them out of it.

Also, what exactly is EA Malaysia's role as compared to EA global? I can imagine that global issues such as climate change and AI existential risks are also being heavily looked at by EA global and others, and depending on the issue, EA Malaysia's involvement could be either independent, complementary, or redundant. 

I'm sure you have already thought about these points. Thanks again for the study. 

Replies from: yiyang
comment by yiyang · 2021-05-14T05:00:47.945Z · EA(p) · GW(p)

Hi Zeshen! I'll be answering you from my own personal capacity, so my views are not EA Malaysia's.  

I'm wondering if we have good reliable statistics on causes of deaths in the country (death being a proxy for suffering), and we could look into the categories of avoidable deaths (e.g. curable illnesses)

For health specific statistics, I've used information from IHME. For animal consumption, I've used data from FAO

and whether those areas are receiving enough support / funding.

It's a bit tough finding exact information about this. I did find one example from this report on the Lancet

mental health spending (RM344·82 million or 1% of the health budget) remains below the average spending on mental health of upper-middle-income countries. 

 

Also, from a poverty perspective, I'm curious if we have an idea how many Malaysians live in hardcore poverty and what can be done to get them out of it.

I have only done a bit of research on poverty, but my intuition tells me that Khazanah Research Institute probably has some information about this.  One of the top Google search results is this report, which I find helpful in dealing with the "where is Malaysia's poverty line issue" that you may have seen in the news sometimes. 

Also, what exactly is EA Malaysia's role as compared to EA global? I can imagine that global issues such as climate change and AI existential risks are also being heavily looked at by EA global and others, and depending on the issue, EA Malaysia's involvement could be either independent, complementary, or redundant. 

I love how you framed the outcomes of our involvement. I might even add, "destructive", which is different from "redundant" - our involvement could caused more harm than good.   

Ideally, we want to be complementary, if working on a certain thing is not our comparative advantage. For example, I would imagine top AI governance research institutions elsewhere have a better comparative advantage than Malaysia's; this would mean that Malaysian wanting to work in this space using EA's perspective but still want to be in Malaysia, would probably have the most impact in localising AI governance research from elsewhere into policy recommendations. 

I don't feel confident giving specific recommendations on reduce risk of doing redundant or destructive work, and increase the chance of doing complementary work. My only intuition to this is to over-coordinate (or coordinate more than you're used to).