EA Giving Tuesday Donation Matching Initiative 2018 Retrospective

post by AviNorowitz (AviN) · 2019-01-06T16:23:57.019Z · score: 81 (27 votes) · EA · GW · 9 comments

Contents

Avi Norowitz, William Kiely, Anisha Zaveri, and Arushi Gupta, were on the organizer team for this project, and many others also provided valuable contributions. See "Project contributors" at the end of this post for more information.

Summary

Facebook’s Giving Tuesday matching program

In early October 2018, Facebook announced that Facebook and PayPal would match donations made on Facebook on Giving Tuesday (November 27, 2018) up to $7 million. Facebook later provided the following details on the match:

In support of #GivingTuesday 2018, we're partnering with PayPal to match donations made on Facebook to nonprofits up to a total of $7 million. Start a fundraiser for Giving Tuesday or donate to a nonprofit.
About the match
- Facebook and PayPal will start matching donations at 8:00am ET (5:00am PT) on Tuesday, November 27, 2018. The matching offer will expire when $7 million in donations is reached or at 11:59pm PT on November 27.
- Donations will be matched dollar for dollar on a first-come, first-served basis.

- Matching is available for any eligible US-based 501(c)(3) nonprofit that can receive donations on Facebook.

- Donations up to $250k per nonprofit and $20k per donor are eligible to be matched.

In addition to the limits described in the announcement, we determined that the maximum amount permitted per donation in the US was $2,500, so 8 donations were required to reach $20,000.

Facebook ran similar matching programs in 2016 for $900,000, and in 2017 for $2 million. In 2017 we attempted [EA · GW] to direct matching funds to EA-aligned organizations. Unfortunately, because we had less than a week to prepare, and because the matching funds ran out much faster than we had planned for (at 86 seconds), only 13% of our total donation amount was matched.

To learn more about the Facebook matching program and how 2018 compared to previous years, see our FAQ.

The opportunity for EAs

Facebook’s annual donation match on Giving Tuesday represents a rare opportunity to get donations to EA-aligned organizations counterfactually matched, similar to US employer matching programs that match employee donations to any US registered nonprofit organization. This opportunity is different than most matching opportunities in the following respects:

We see this donation-matching opportunity as one that avoids most of the pitfalls described by this GiveWell blog post on donation matching.

To learn more about the opportunity for EAs, see our FAQ.

Our strategy to direct the matching funds

In 2017 [EA · GW], the matching funds ran out in 86 seconds, which was much faster than most of us had anticipated. As a result, we estimate that only 13% ($48k) of our total donation amount ($379k) was matched. We learned from this experience that success in 2018 would likely require us to donate fast. As such, we emphasized that donors should try to donate within as few seconds as possible after the match begins. We also provided basic instructions, full instructions, and demonstration videos on donating fast.

We also wanted to give donors an easy way to donate to a large variety of EA-aligned organizations, and we wanted to be able to collect data on donations and matches to these organizations. To facilitate this, we created one Facebook fundraiser for each organization and added them to a spreadsheet.

Because only US registered 501(c)(3)s nonprofit organizations were eligible, we faced the risk that a large number of important organizations, programs, and funds would remain out of reach for donors. This could have reduced the amount of donors eligible to participate, or could have influenced donors to donate less effectively for the match. Thankfully, we were able to work with CEA, ACE, and GiveWell to provide donors the opportunity to donate to the majority of these difficult-to-reach organizations, programs, and funds through Facebook on Giving Tuesday.

In 2017, we became aware of the opportunity less than a week in advance, which didn’t give us much time to prepare. In 2018 we started our preparation around 2 months in advance. This gave us the opportunity to carefully plan our strategy, give organizations a heads up, arrange a plan to get donations and matches to difficult-to-reach organizations, prepare good materials, and work on donor outreach.

To learn more about our strategy, see our FAQ.

Pledge and donation amounts

By the Saturday morning before Giving Tuesday, we had $200k in pledges. By the time the match began on Tuesday morning, this had increased to $614k in pledges from 160 individuals.

After our fundraisers ended at midnight on Wednesday, we had a total of $719k in donations. The vast majority of these donations were made by 209 donors within the first 10 minutes after the match began.

Within 2 minutes after the match began, Facebook began reporting on fundraiser pages that the matching funds had run out.

You can see a more complete breakdown of pledges and donations on our website.

Organizations on Facebook Payments have likely received donation amounts by mid-December. Organizations not on Facebook Payments should receive donations in mid-January as a check or an electronic funds transfer from Network For Good.

Match amounts

Facebook had reported that they would communicate match amounts to donors, fundraiser creators, and organizations on or after the Thursday, November 29. Unfortunately, Facebook did not follow through as it said it would. Nevertheless, by mid-December, we were able to accumulate enough data to estimate the match end time and the match amounts. We estimate that the match ran out within 15 seconds, and that donations made between seconds 0 and 14 were matched. Even though the match ran out as fast as it did, we estimate that $469k (65%) in donations to our fundraisers were made before the match ended and were therefore matched.

Our estimates are supported by multiple lines of (incomplete) data that donors, fundraiser creators, and organizations have been able to obtain from Facebook. At this point, we believe our estimates are accurate within a few percentage points.

You can find a more complete breakdown of donations and matches on our website.

In addition to the fundraisers we created, other people created Facebook fundraisers for EA-aligned organizations as well. A few of these received a large amount of matching funds from Facebook and PayPal. Specifically, I estimate that this fundraiser for Animal Charity Evaluators received $20k in matching funds, and that this fundraiser for the Machine Intelligence Research Institute received around $25k in matching funds. If we were to include these in our total amount matched, this would increase the amount to $514k.

Organizations should receive match amounts in mid-January as a check or an electronic funds transfer from Network For Good.

Follow-up with organizations

We’re currently in the process of following up with organizations that we expect to receive a nontrivial amount of donations and/or matches through our initiative. We’re providing them our estimated total donation and match amounts, as well as the timestamps, donor names, donation amounts, and match amounts for each donation. Since organizations won’t be able to see contact information for their donors, we’re also providing contact information for pledgers who gave us permission to do so.

We’ve also asked organizations to provide us with donation and match amounts they’ve received on Facebook on Giving Tuesday, and we’ll update our website with these amounts when we have them available. We expect to receive this information from only some organizations.

So far, the donation and match amounts reported by organizations have been consistent with our expectations. That being said, as far as we know, Facebook has not provided any information to organizations not registered for Facebook Payments, so we don’t have information from any of these organizations yet. This includes donations and matches via CEA and GiveWell, which combined make up around half of all donations and matches.

We’re also following up with each regranting organization -- CEA, GiveWell, and ACE -- to provide instructions on how to allocate the donations and matches they’ll be receiving to the appropriate organizations, programs, and funds. These instructions will be based on the donation and match amounts we estimate from our fundraisers.

Follow-up with donors

At around 20 minutes after the match began, we sent a follow-up survey to donors. We performed an analysis on the survey results, which you can find here. We seem to have captured the vast majority of donors in our survey, weighted by either donation amounts or number of donors. In this section, I review some of the survey findings.

Payment problems

In 2017, payment problems caused some donations to get delayed. We wanted to learn about payment problems that donors experienced in 2018 so we could help mitigate those problems in 2019. We found the following:

Donors outside the US

25% of donors were outside the US, but because of Facebook donation amount limits and associated payment problems, their donations represented only 4% of the total donation amount.

Were EAs motivated to donate more?

We believe the $469k in matching funds are counterfactually valid [EA · GW]. That is, without the efforts of the EA community, we believe the matching funds would have otherwise gone to organizations of average effectiveness.

Some people have suggested that some donations might be counterfactually valid as well: It’s possible that without the EA Giving Tuesday donation matching initiative, some people would not have donated, or would have donated less. To provide some insight into this, we asked donors whether the matching opportunity motivated them to donate more, not including donations they would have otherwise made in 2018 or 2019. Donors reported that 12% of their donations ($85k) would have not otherwise been made in 2018 or 2019. We consider this amount to be highly uncertain, given questions about the accuracy of self-reports, the time period considered, and the question phrasing.

How we improved since 2017

Our estimated match amount increased from $48k in 2017 to $469k in 2018, while our estimated match percent increased from 13% to 65%. I believe much of this increase was caused by improvements in our work in 2018.

Focus on donating fast

In 2017, the key to getting donations matched was donating within the first 86 seconds. We did not know in advance when the matching funds would run out in 2018, but had reasons to believe it may run out much faster. We suggested that people make their donations within as few seconds as possible. This turned out to be good advice, because the match ended within 15 seconds.

Starting early

In 2017, a number of us discovered the match opportunity about a week in advance, giving us just a week to prepare. In 2018, we began preparation work at the end of September, around 2 months in advance. Most of the improvements described in this section were only possible because we had started early, giving us enough time to implement them.

Focus on mitigating payment problems

In 2017, many large donors had payments declined by their payment issuers, causing delays in their donations. We were also uncertain about the maximum amounts permitted per donation. Facebook said they would match up to $20k in donations per donor, but would Facebook permit single donations as large as $20k?

We spent a considerable amount of time investigating payment limits and determining on the optimal amount per donation to recommend to large donors. In the end, it’s unclear how much of this work was helpful. Since payment problems could have been very costly, however, I think the expected value of this work still made it worthwhile. The information we learned from this may also help us in 2019.

Improved outreach and marketing

We spent a considerable amount of time on outreach and marketing to donors, including the following:

Improved communications with organizations

Given our time constraints in 2017, we were unable to notify organizations in advance of our plans. This resulted in a number of problems:

In 2018 we made an effort to address these problems by starting a dialog with most organizations 1-2 months in advance of Giving Tuesday.

Coordination with regranting organizations

In 2017, some organizations, programs, and funds were unable to receive donations because they were not their own 501(c)(3)s. This could have resulted in lost opportunities to get donations matched. This could have also influenced some people to donate less effectively just for the match. In addition, follow-up restriction requests to CEA and GiveWell created inconveniences for these organizations.

To mitigate these concerns, we began a dialog early with CEA, GiveWell, and ACE to work on arrangements so they could accept donations and matches for organizations, programs, and funds that could not directly receive them through Facebook. We proposed arrangements that would minimize the amount of work these regranting organizations would need to undertake. So, for instance to avoid the need for regranting organizations to communicate with donors individually about follow-ups, we arranged to inform the regranting organizations of the total amounts donated and matched to each organization, program, and fund.

This coordination turned out to be highly valuable, since 40% of donations and 45% of matches were from restricted donations going through CEA, GiveWell, and ACE.

Data collection and analytics

We anticipated that high-quality data and analytics would be important for a number of reasons:

For these reasons, we took a number of approaches to facilitate data collection and analysis:

Areas for improvement for 2019

If the Facebook donation matching opportunity is available again in 2019, we could likely improve in a number of areas.

Improved communication with donors

Even though donors outside the US represented 25% of donors, Facebook donation limits and associated payment problems prevented them from donating more than 4%. I think we could have communicated better that donors outside the US may face payment problems and probably won’t be able to donate large amounts.

We also receive some feedback from a large donor that they were not aware that advance preparation on their side was required. I think we could have communicated this better as well.

Earlier preparation of instructions

Although we had an FAQ ready about 5 weeks in advance, we only began working on instructions (including demonstration videos) about 2.5 weeks in advance. The instructions took longer than expected to finalize, and we were only ready to share them about 1.5 weeks in advance. This meant that we were barely able to have instructions ready on time for GWWC to share us in their newsletter. This also meant that we didn’t give organizations much time to adapt the instructions for their own fundraising purposes.

Helping organizations fundraise

Since organizations have expertise in fundraising, they may have been well-positioned to help motivate their donor base to get their donations matched through Facebook. We informed organizations they could use our instructions to help with their fundraising efforts, but I think we probably did not spend as much effort working with them on this as we could have. A few organizations ran effective fundraising efforts to help get donations matched through Facebook, but there may have been more opportunity to get more organizations onboard.

Investing more in technology

We increased our usage of technology in a number of areas, including making better use of Google Forms and Google Sheets, registering domain names (GoDaddy), creating a website (Google Sites), and making use of a URL shortening service (Rebrandly). Because of time constraints and a preference not to spend money, however, I think we didn’t invest as much as we should have.

Incorporating donation survey results

If the Facebook donation matching opportunity is available again in 2019, I think we could likely use our donation survey results to help us do better. For instance:

Lessons learned

We learned a number of lessons from our work on this, some of them specific to the Facebook donation matching and others that might be more generalizable to other EA projects.

EAs can follow instructions and work well as a community

In our 2017 retrospective, I noted [EA · GW] that I was impressed with how well EAs worked well as a community. Although our match percent was only 13%, this was because we did not provide instructions to donate within as few seconds as possible. With regard to the instructions we did provide, EAs followed those instructions impressively well.

In 2018, I anticipated that a nontrivial amount of preparation would be necessary for donors to donate fast enough to get matched, especially large donors making multiple donations. We had provided instructions and demo videos, but I had concerns that people might not read them, and that we might end up with a low match percent once again. In the end, these problems did not materialize: 65% of the total donation amounts by EAs were made within 15 seconds and were therefore matched. This suggests that, in general, EAs read the necessary instructions and did the necessary preparation.

Coordination with a small number of organizations yielded large benefits

I saw it as especially important for people to have the ability to donate to a large variety of EA-aligned organizations, programs, and funds. We spent some time arranging ways for some organizations to accept donations for others, without creating too much administrative overhead. In the end, we were able to make arrangements for CEA, GiveWell, and ACE to accept donations for a large number of organizations, programs, and funds in a way that minimized administrative overhead for the organizations involved. This coordination yielded large benefits, since 40% of donations and 45% of matches were from restricted donations going through CEA, GiveWell, and ACE.

Most EAs waited until the day before to get involved

In 2017, the vast majority of pledges, fundraiser creations, and other activity by the EA community occurred within the 24 hours leading up to the match. I saw this as problematic, because it left little time for people to prepare, or tell their EA friends about the opportunity.

In 2018, we tried to increase early involvement by having a full plan in place with FAQs, instructions, and videos available more than a week in advance. We also began promoting the opportunity a month in advance. Even so, most pledges and other activity occurred within the 24 hours leading up to match.

In 2019, I’d like to continue making an effort to get people involved earlier. But it might be the case that no matter what we do, most people will wait until the 24 hours leading up to the match before getting involved.

Facebook Donations Support responses were mostly unreliable

In 2018, I opened a large number of support requests with Facebook Donations Support to request clarifications about donating in general and Giving Tuesday in particular. The replies I received were largely unreliable, and often contained confirmed inaccuracies. For the most part, we had to make educated guesses and find creative ways to get the information we needed.

On the other hand, we later found that automatic replies from Facebook Donations Support indicating whether particular fundraisers or donations were matched were accurate.

In 2019, it might still be worthwhile to open support requests with Facebook Donations Support, but we should start with the assumption that answers written by humans are unreliable unless we can independently confirm them.

Near misses

We discovered a few unexpected problems along the way that could have substantially reduced the amount of donations that were matched. This section describes those problems and how we were able to avoid them.

Organizations could have become ineligible

In our initial communications with organizations who were not already signed up for Facebook Payments, we had suggested they sign up so they could receive donations earlier, and also see the names of their donors. Shortly after, Arushi Gupta discovered that signing up could cause organizations to become temporarily ineligible to receive donations while they undergo a vetting process. After investigating the issue further, we reversed our recommendations. If we had not noticed this problem, this could have led to a large number of organizations becoming ineligible to receive donations on Giving Tuesday.

We saw this problem materialize with one organization that had signed up for Facebook Payments in preparation for Giving Tuesday. The organization became ineligible to receive donations, and became eligible again just two days before Giving Tuesday.

It’s unlikely we could have predicted this behavior, but I think this illustrates the importance of carefully reading through potentially relevant documentation.

People could have donated before the match began

We observed that timestamps on Facebook fundraiser pages were delayed by around 2 seconds, and suggested in our demonstration videos that donors may be able to donate 1-2 seconds before the match began. The night before, William Kiely realized this was a mistake: Facebook began printing e-mail timestamps in donation receipts, and these timestamps did not reflect this 2 second delay. We quickly reversed course and communicated that people should not donate early. If we had not noticed this, we could have lost as much as $100k in matches, depending on how many people had planned to donate early.

Questions for 2019

If the Facebook donation matching opportunity is available again in 2019, our experience in 2018 raises some questions for 2019.

How much time will we have?

In 2016, Facebook reported that the match ran “within hours,” although we don’t have independent evidence of this. In 2017 the match ended at 86 seconds, and in 2015 the match ended at 15 seconds. If trends continue in 2019, the match could run out even faster. Our strategy will likely vary depending on how fast we expect the matching funds to run out.

What unexpected problems could we face?

In 2017, most of us failed to anticipate the importance of donating fast, leading to a match percent of only 13%. In 2018, we avoided this and other problems, and achieved a match percent of around 65%. But as described previously, we had a few near misses in 2018 that could have led to much fewer donations getting matched.

Given that we now know about these particular problems, we’ll make sure to avoid them in 2019. But I think we should make sure to be vigilant about other unexpected problems similar to these that could occur.

How open should we be about our work?

In 2018 we struggled with tension between wanting to promote the opportunity broadly within the EA community, but not wanting to promote it widely outside of it. Although we wanted to get as many donations to EA-aligned organizations matched as possible, we anticipated a number of risks in promoting it outside of EA:

For these reasons, we tried to avoid generating attention outside the EA community. For instance, we had two opportunities to get positive media coverage of our efforts, but declined them. We also took steps such as avoiding mention of “Giving Tuesday” in our Facebook group name and description. This made us less easily searchable by people outside of the EA community, but one EA said it made it difficult for them to find us.

It’s not clear whether or not we struck the right balance, and this is an issue we should continue to think about in 2019.

Project costs

Most of our project costs were in the form of hours worked by our organizer team and others. We also had a small amount of financial costs, which we self-funded.

Hours worked

I estimate that our organizer team worked a total of 330 hours. In addition to this, many other EAs were involved (see the “Project contributors”), and combined this might have added hundreds of more hours of work on the project. Given the large match amount ($469k), I believe the number of hours worked was worthwhile. The main reservation I have to this is that we did involve some highly valuable individuals at the EA organizations we coordinated with, and the counterfactual use of their time could have been highly valuable.

Financial costs

Our financial costs were self-funded among our organizer team. I estimate we spent $30 on technology, $75 in prepaid debit cards for testing payment limits, and $650 in test donations.

Most of the prepaid debit cards were later redeemed for personal use.

The vast majority of test donations were made by William Kiely to CEA, and will be presumably put to good use.

Project contributors

A large number of individuals from the EA community contributed to the EA Giving Tuesday donation matching initiative. This section notes many of these contributors, but is not comprehensive. I apologize to any contributors I neglected to mention below.

9 comments

comment by Larks · 2019-01-06T18:22:46.296Z · score: 10 (7 votes) · EA · GW

Congratulations guys, this is really impressive. Thanks for all the work you put into this.

comment by agdfoster · 2019-01-07T20:36:04.629Z · score: 7 (5 votes) · EA · GW

Really impressed by both how you've executed this as well as the write-ups. 🙌

Thank you!!

comment by Khorton · 2019-01-06T21:44:58.578Z · score: 5 (5 votes) · EA · GW

Super impressive! If you're looking to increase investment for next year, you could apply for funding from the EA Meta Fund [EA · GW]

comment by WilliamKiely · 2019-01-06T23:55:04.502Z · score: 8 (4 votes) · EA · GW

Thanks for mentioning that the EA Meta Fund might be interested in funding things like this ("A project experimenting with novel fundraising strategies or target groups").

The question of whether having external funding would have helped seems complicated.

I think that there was a lot more valuable work that we could have done to make this initiative an even greater success, but I don't think that having external funding would have caused me or Avi to do more of this work. Firstly because we were capable of funding ourselves, but primarily (at least for me personally) because the challenge I was facing was how I could find the time/energy to do extra work on this project while maintaining my standing at my day job, which seemed important so I could continue working there in the future.

Due to a lack of foresight I had taken a 19-day vacation (Oct 19 - Nov 6) to Europe to attend EA Global London and then do some more traveling, which meant that I couldn't take any more time off from my job later in November upon my return without potentially significantly affecting my standing at my company. This meant that I just had nights and weekends to do work for Giving Tuesday.

I considered that I ought to have taken the extra time off from my job anyway despite this cost, but at the time this seemed too extreme a measure given that I didn't have any plans for what I would do come December/January had things taken a turn for the worse at my job due to my absence.

As I'm writing this now, it actually seems like I definitely should have taken more time off so I could do more work for Giving Tuesday despite the consequences of that for my day job. But my bias is to be risk-averse and not do things that seem to crazy or extreme, so it's not surprising to me that I didn't do this.

Perhaps external funding would have been a benefit in that it would have provided social approval for me taking more time off from my day job to work on the project. My employer probably also would have been more understanding of me doing this if I had external funding from a reputable organization to signal that what I was taking time off to do was valuable.

Actually, now that I think about it, I don't even know for sure how my employer would have reacted to me e.g. taking the entire month of November off since I didn't ask. I think it's likely that I would have been let go, but I could easily be wrong, especially if I took the time to explain why I wanted to take the time off and why I wanted to come back afterward.

comment by AviNorowitz (AviN) · 2019-01-07T13:06:44.979Z · score: 2 (2 votes) · EA · GW

Thanks for the suggestion. I already had EA Grants in mind as an option, but it's interesting to know that the EA Meta Fund lists a fundraising project as an example. As William noted, we were more time constrained rather than funding constrained this year. However, I'll keep this in mind as an option for future years if our circumstances change, or if we can come up with effective ways to convert funding into time.

One other complication with applying for a grant is that we wont know whether there's a worthwhile opportunity until a month or so in advance. After that, if there is a worthwhile opportunity, then we'd need to start working immediately. So we'd probably need to apply for a grant early and, if a worthwhile opportunity doesn't materialize, then we'd need to (a) use the grant to work on something else or (b) return it.

comment by anonymous_ea · 2019-01-08T18:37:32.220Z · score: 3 (3 votes) · EA · GW

I want to echo other people in saying that this was both incredibly impressive and very impactful! Thank you so much :)

comment by Khorton · 2019-01-07T23:27:40.977Z · score: 3 (3 votes) · EA · GW

From your comments, it sounds like it would be useful to have a team of people (maybe 5 or 6) who have precommitted 10-100 hours in November to Giving Tuesday prep if their effort is needed. Do you think that would have helped this year? Do you think that's practical for next year?

comment by AviNorowitz (AviN) · 2019-01-09T03:53:22.000Z · score: 2 (2 votes) · EA · GW

I think improving recruitment could help. I did recruit people in late September 2018 after Facebook dropped a hint that they would do a repeat, but in retrospect it may have helped to focus on recruiting individuals who would be able to commit to 80 hours (or whatever) if necessary.

This might not be so easy though, since I suspect most people with the skills to do this well are already working on other things and would have difficulty sparing that many hours. Perhaps compensation (e.g. through a grant) could help, but I'm not sure.

I also don't think having large numbers of people each working 10 hours would help, because managing, training, and delegating tasks to that many people would be impractical.