Final update on EA Norway's Operations Project

post by Jorgen_Ljones, eirine · 2020-01-10T11:27:14.032Z · score: 28 (14 votes) · EA · GW · 6 comments

Contents

      Executive summary:
  What we’ve done since the last forum post
    Core
    Advisors
  Results of the operations project
    Pressing challenges and top priorities
    Key considerations in helping operations staff
    Create meeting places for operations staff
    Engage skilled people to build on this work in later projects
  Lessons learned from running a project funded by EA Meta Fund
  Final remarks
None
6 comments

Eirin M. Evjen [EA · GW], Exec. Dir. EA Norway

Jørgen R. Ljønes [EA · GW], Ass. Exec. Dir. EA Norway

In a forum post in February [EA · GW], we posted our plans for organising an operations camp aimed at improving recruitment of operations talent for EA organisations. The EA Meta Fund gave us a grant for the project in March. As we wrote in September [EA · GW], we decided not to run an operations camp as part of our project, because our research on the needs at EA orgs showed that recruitment isn’t one of the main bottlenecks. In this update, we’ll provide an overview of the progress of the project in this last phase, and the lessons we have learned about running a project funded by the EA Meta Fund.

Executive summary:

What we’ve done since the last forum post

When we decided to not run an operations camp, but rather identify top challenges and priorities for operations staff and ways to solve them, we quickly realised that the EA Norway team did not have the necessary skills or experiences to continue the project. To account for this, we recruited and onboarded new team members and decided to reduce our own involvement in the project. Moving forward, the team consists of:

Core

Advisors

To find potential solutions for the top challenges for operations staff at EA organisations, we ran a one-hour facilitation session at EAG London in October. A total of sixteen participants from twelve organisations attended. The session consisted of four parts:

  1. Short recap of the EA operations bottlenecks project and goals of this session.
  2. Priorities & challenges: Participants wrote down three priorities/challenges they’re facing in the next 6-12 months and verbally shared their top challenge/priority.
  3. Small group discussions: We grouped participants based on their top priorities/challenges, after which they discussed shared challenges in groups.
  4. Debrief and discussion about next steps to carry the work forward.

Overall we see the session as a success. Most of the operations staff we invited attended, and the feedback was positive. Out of the sixteen participants, twelve answered our survey at the end of the session, while the rest had to leave the session early. The Net Promoter Score was 42. When asked what the most valuable part of the session was, most of the respondents responded it was meeting other operations staff and hearing the challenges of the other organisations.

However, there are multiple ways we could have improved the session. For example, we received feedback from several participants that it could have been longer. In addition, the grouping of the participants could have been greatly improved. We tried to get data on the top priorities of select organisations leading up to EAG London, but were unsuccessful. Because of this, we attempted to match attendees based on their reported top priorities during the session. In hindsight, we probably should have allowed the participants to self-select their groups. Lastly, several participants reported that some form of mandatory preparation would have made the session more valuable. Given the participants’ level of engagement leading up to the session, such requirements did not seem feasible at the time. Still, the overall results may have been better if we pushed for the participants to prepare for the session, even if some wouldn’t comply.

Results of the operations project

The four key outcomes from the project were that we:

  1. Identified the most pressing challenges and priorities for EA operations staff.
  2. Uncovered key considerations in helping operations staff.
  3. Created meeting places for operations staff.
  4. Engaged skilled people to build on this work in later projects.

Pressing challenges and top priorities

As we wrote about in the previous forum post [EA · GW], we held a workshop at EA Global San Francisco and sent out an online form to organisations that didn’t participate at the workshop. Based on responses from eighteen operations staff, we found that there were five particularly pressing operations challenges:

  1. Standardisation and documentation of internal policies and processes.
  2. Talent retention.
  3. Making time for strategic thinking.
  4. Sharing best practices between organisations.
  5. Cultural challenges.

During the facilitation session at EAG London, we asked the participants what their top priorities were for the next six to twelve months. Here are the main categories that were mentioned:

The responses from the workshop at EAG SF and the session at EAG London aren’t directly comparable since the questions and circumstances were different. In SF we gathered data on the high-level challenges to find the largest pain points, while in EAG London we asked the participants to mention specific priorities in the near future.

Key considerations in helping operations staff

Throughout this project, we have coordinated and worked with multiple operations staff members. Based on our experience and conversations with them, we have generated some key takeaways and lessons learned on how to best help people in their positions.

Any future projects or efforts aiming to help operations staff with their challenges should consider these takeaways. In particular, next steps should include tailored and targeted support addressing the main challenges faced by operations staff, should build on the experiences and lessons learned from other organisations, and should include operations staff meeting each other.

Create meeting places for operations staff

At both EAGs, our events became a forum for operations staff to be introduced to each other. We had a total of twenty-six participants from sixteen different organisations between the two events. Participants said that they really value meeting other operations staff, hearing others’ challenges, and getting some input on their own challenges. We know that the session at EAG London led to one-on-one conversations between participants who had not planned these beforehand.

Engage skilled people to build on this work in later projects

As mentioned above, we have engaged a team of experienced and motivated people to build on the work we have done so far to continue identifying and solving the most pressing challenges in operations at EA organisations. In agreement with the EA Meta Fund managers, we consider the project we received funding for as completed with this report. We are glad skilled people are continuing to work on operations challenges and solutions, but now as a new project not funded by the EA Meta Fund. With EAG SF in March 2020 as the next milestone, the new project team will explore potential paths to leverage their capacity and expertise to the benefit of operations staff.

In addition, many of the operations staff we have encountered have expressed interest in being involved moving forward, either as a participant or as an advisor. This is promising for the team’s work ahead. Still, as the phase leading up to the facilitation session at EAG London showed, operations staff have limited available time.

We believe one of the most valuable results of the operations project has been to ensure skilled and experienced people are exploring and implementing solutions to the challenges that operations staff are facing. We hope that we have given them what they need to continue the work successfully, through getting in contact with operations staff and advisors, understanding key considerations when cooperating with operations staff, and being motivated to do the work.

Lessons learned from running a project funded by EA Meta Fund

This has been EA Norway’s first time running a project funded by the EA Meta Fund. For anyone considering doing the same, here are some of the lessons we have learned that might be useful.

As with most application-based funding sources, there seems to be a fundamental conflict between the grantor’s need for reliable signals when evaluating applications, and the grantee’s need to secure funding before sinking resources into a project. The best signal for a grantor is the quality of completed work. The trade-off between these two needs depends on the risk appetite and risk portfolio of each party. Before we submitted our funding application, we researched the need for help with recruitment at EA organisations with running an operations camp in mind, but we did not spend time asking broader questions of how pressing hiring needs are compared to other operations bottlenecks. After receiving the grant from the EA Meta Fund we had the time to explore such questions, which turned out to be crucial for our decision to not go through with the camp as our vehicle to improve the operations capacity at EA organizations. Ideally, we and the EA Meta Fund would have benefited from having this information before deciding to spend the resources, but this was not clear to us before we applied. We are unsure about the best takeaway here, and we are interested in suggestions and similar experiences from others in the comments. But it seems like grantors and grantees should discuss this problem at an early stage and reveal their level of uncertainty, willingness to spend time getting more information and risk appetite to make better trade-offs.

A related trade-off is whether it’s best to plan for and apply to carry out a concrete project or to achieve a certain goal. We applied for funds to organise a camp, but our overall goal was to help mitigate operations challenges in the EA community. When we decided not to organise the camp after all, we changed the means of achieving the goal, but the goal persisted. While getting a grant to carry out a concrete project is much easier to communicate and provides clearer expectations between grantee and grantor, this is much more rigid than having an overall goal as the focus of a project.

We believe it was the right choice to not organise an operations camp, but in hindsight we see that communicating changes and expectation management proved more challenging than we expected. Gradual changes and many marginal adaptations to new information may result in stakeholders outside the working team perceive a larger deviation from the original message than the working team realises. If we end up applying for a grant in the future, we are likely to have the application and project description focus more on the goal we are trying to achieve rather than the activity we are guessing is our best way of achieving it.

Another lesson we learned was to discuss communication expectations at the very beginning with both the EA Meta Fund and CEA. In calls with the fund managers, our impression was that the Fund and CEA did not want general updates, and that we were overall given a lot of flexibility in how we spent the money. Because of this perception, we did not frequently communicate with representatives from the EA Meta Fund or CEA about project expenses or updates, and the EA Meta Fund did not ask for further updates even after being offered it explicitly. In hindsight, it would have been better if we had clarified the boundaries of our grant, and what happens if our plans changed after we began working on the project.

Working on an ecosystem-level challenge such as operations bottlenecks in EA has required a lot of involvement of key stakeholders. In this forum post we have mentioned that people we talked to were busy and rightfully prioritised other things when asked. However, our experience has been that most EAs we have asked for input and data for our project are friendly, helpful, and honest. This is also true for the contacts we have had at the EA Meta Fund and CEA - they have always been eager to cooperate. If anything, we believe we have been too slow and hesitant to ask for help and input from people in our movement. We are very grateful for everyone that has contributed to this project so far.

Making significant changes mid-project and working on such complex issues has been challenging for us. Not going through with our initial plan has sometimes given us a perception of failing. Furthermore, when we realised that we lacked sufficient experience and skills to continue solving bottlenecks in operations, it was discouraging. Because of this, we have been really thankful for all the support and positive feedback we have received on this operations project. It is great to be part of a community that appreciates uncertainty and values updating on new evidence.

Final remarks

This project has been both exciting and challenging for us at EA Norway. Trying to solve these types of ecosystem-level problems is complicated and difficult, but has the potential to unlock a lot of value if done successfully. Working closely with skilled people - both our team members and operations staff - has been a motivating and important learning experience for us.

If you want information about the operations project as it ran until November 2019, don’t hesitate to reach out to EA Norway at post@effektivaltruisme.no. If you want information about the work moving ahead and what the plans of the new team are (or if you want to see how you can help), contact Mark at ea@markmccoy.me.

6 comments

Comments sorted by top scores.

comment by SiebeRozendal · 2020-01-11T11:46:33.102Z · score: 5 (4 votes) · EA(p) · GW(p)

(Off-topic @JPAddison/@AaronGertler/@BenPace:)

Is tagging users going to be a feature on the Forum someday? It'd be quite useful! Especially for asking a question to non-OP's where the answer can be shared and would be useful publicly.

comment by SiebeRozendal · 2020-01-11T11:42:02.920Z · score: 3 (3 votes) · EA(p) · GW(p)

Props for making a no-go decision and switching the focus of the project - I think that is very commendable!

I am very curious about what is going to happen further, and have a few questions:

@EA Norway: Do you have any ideas/opinions on addressing operations bottlenecks that might also be highly impactful, such as

a) organisations doing highly impactful work but not explicitly branded as EA (e.g. top charities, research labs) and

b) other EA projects, such as large local/national groups, and early-stage projects.


comment by SiebeRozendal · 2020-01-11T11:43:54.008Z · score: 2 (2 votes) · EA(p) · GW(p)

(@Meta Fund:)

Will any changes be made to the application and funding process in light of how this project went? I can imagine that it would be valuable to plan a go/no-go decision for projects with medium to large uncertainty/downside risk, and perhaps add a question or two (e.g., 'what information would you need to learn to make a go/no-go decision?') if that does not bloat the application process too much. I think this could be very valuable to explore more risky funding opportunities. For example, a two-stage funding commitment can be made where the involved parties can pre-agree to a number of conditions that would decide the go/no-go decision, making follow-up funding much more efficient than going through a new complete funding round.

comment by SiebeRozendal · 2020-01-11T11:42:52.485Z · score: 1 (1 votes) · EA(p) · GW(p)

(@Mark McCoy:)

I wonder what is currently happening with Good Growth and how it relates to this current so-far nameless operations project. It seems like it is an unfunded merging of the two projects? Could you briefly elaborate on the plans and funding situation for the project?

comment by casebash · 2020-01-12T12:32:31.026Z · score: 2 (1 votes) · EA(p) · GW(p)

What's Good Growth?

comment by SiebeRozendal · 2020-01-13T21:29:31.622Z · score: 1 (1 votes) · EA(p) · GW(p)

A consulting organisation aimed at EA(-aligned) organisations, as far as I'm aware: https://www.goodgrowth.io/.

Mark McCoy, mentioned in this post, is the Director of Strategy for it.