How we think about the Forum

post by JP Addison (jpaddison), Centre for Effective Altruism · 2019-10-15T16:24:04.447Z · score: 24 (12 votes) · EA · GW · 3 comments

Contents

  Models
  Plans
    Quarterly goals for Q4
    Later goals
  Final Thoughts
  Footnotes
None
3 comments

This is the second post out of two in a series about how the Forum gets made. In the last post [EA · GW], we introduced ourselves. In this post, we’ll try to lay out the models that guide our work and the plans we have for the future.

Models

The Forum is not a program executing a single plan; it’s a distributed ecosystem of many people trying to achieve many goals. The impacts are as numerous as the aims of every post and comment. Still, we can classify them using a few broad categories:

(Credit to Oliver Habryka [EA · GW] for an initial list upon which we built to produce the above.)

To achieve those goals (or really, a much broader class of goals), we think the Forum needs both of the following intermediate goals:

We’ve developed a metric we use to figure out whether this is happening (and thus, whether we’re doing a good job). We evaluate our plans largely based on how we think they’ll impact the number of good posts [1] and the amount of views those posts receive. [2]

Don’t read too much confidence into this section. We view it as a better-than-nothing operating framework. In an ideal world, we’d have much more empirical data backing up our model. We’ve also had an actual, updating graph of the metric for all of a few weeks. Finally, we also track other metrics (from “time on site” to “mentions on popular blogs”) to get a better picture of the Forum’s impact.

Plans

I know the title of the section is “Plans,” but first, here are some things we’ve already done that we think of as aiming towards the “views on good posts” metric:

Quarterly goals for Q4

A lot of our goals aren’t explicit features. They’re things like “merge in LW with maximum 4 weeks delay” or “meet these goals every two-week sprint”. Our main relevant feature-goals are:

There are other goals that aren’t large enough to quite be “quarterly goals” in the OKR system we use, but will likely be a sprint goal, such as fixing our (currently LessWrong-green) emails, adding an option to hide karma, and improving our error reporting.

The content side (aka Aaron), also has some goals pertaining to the Forum:

Later goals

We might do these, but not this quarter.

Final Thoughts

We hope, after reading this, you have a good sense of the actual dynamics that shape this Forum. Don’t be shy to ask questions! We’re especially interested in reactions to the proposed features.

Footnotes

[1] What’s a “good post”? Currently, we mark certain posts as representing the kind of content we most want on the Forum (this corresponds to roughly 20% of posts). We also include posts that sparked lively/productive conversations, even if the posts themselves were very brief/just asking a question. This doesn’t mean we don’t value lots of other content — we do! A lot! — but tracking views to all posts seems less informative than watching views on the posts we think offer the most value to those who read them. (This evaluation is cause-impartial and does not affect the visibility of posts in any way.)

[2] Currently, we’re focusing on views from logged-in users. This has several good properties:

3 comments

Comments sorted by top scores.

comment by Peter_Hurford · 2019-10-16T16:09:24.751Z · score: 16 (9 votes) · EA(p) · GW(p)

I'd just like to +1 that SEO could be very valuable.

More generally, I'd like to see more work into solving the "lost content" problem. My guess is that great posts from last year are rarely read or remembered today while they might still be rather relevant. “Best posts since you last visited” could be helpful, but so would SEO and also maybe tags.

comment by anon_987234 · 2019-10-19T08:31:09.749Z · score: 0 (3 votes) · EA(p) · GW(p)

To my knowledge:

  • The CEA has two full-time developers (Sam Deere and JP Addison). JP has been at CEA since at least June 2018, and Sam has been there for many years.
  • The EA forum appears to be a copy of the LessWrong forum, with some minor cosmetic modifications (like having a 'community' section).
  • The two web development achievements mentioned in this post ('Built the timeframe feature into the All Posts page to enable easier post discovery', 'Ported and modified the Community Favorites section and added it to the homepage') are features directly ported from LessWrong.

Please correct me if I'm wrong about any of these.

My interpretation of these facts is that the developer team at CEA isn't very productive.

(There's also EA funds and the EA donor lotteries, but what new features have been added there in the last 12 months?)

comment by JP Addison (jpaddison) · 2019-10-19T10:27:51.737Z · score: 9 (7 votes) · EA(p) · GW(p)

Hi,

I'll first clarify / correct some facts, then discuss my interpretation.

  • I've been at CEA since September 2017. I have been working full time on the Forum since May. Previously, I was only trying to keep it up to date with LessWrong. As mentioned in the previous post in this series, Sam does not work directly on the Forum.
  • Correct [EA · GW]. The developer term of art is "fork". In this case it's a fork where we push our changes upstream to LessWrong, and keep up to date with their changes.
  • When I say 'Built the timeframe feature' I do in fact mean that I built it. It exists on LessWrong because I submitted a Pull Request. For Community Favorites section I refactored a small amount of code to make it fit back into our frontpage, (as LessWrong's is used quite differently) and to have it include posts from the "Community" section at the user's option. These two features were selected for relevance to the metric, but don't represent the entirety of my work.

I’ll let Sam reply with the amount of behind-the-scenes work that he’s needed to do on the Funds & Giving What We Can (if he wants, also fair warning that he might want to wait till a workday).

As an open source project, we're quite a bit more open than the average software project, but I realize it’s hard to assess the work that goes into a tech project from the outside.

Given the state it was in previously and the commercial alternatives, I'm pretty happy with the progress made on the Forum.