Posts
Comments
[weirdness-filter: ur weird if you read m commnt n agree w me lol]
Doing private capabilities research seems not obviously net-bad, for some subcategories of capabilities research. It constrains your expectations about how AGI will unfold, meaning you have a narrower target for your alignment ideas (incl. strategies, politics, etc.) to hit. The basic case: If an alignment researcher doesn't understand how gradient descent works, I think they're going to be less effective at alignment. I expect this to generalise for most advances they could make in their theoretical understanding of how to build intelligences. And there's no fundamental difference between learning the basics and doing novel research, as it all amounts to increased understanding in the end.
That said, it would in most cases be very silly to publish about that increased understanding, and people should be disincentivised from doing so.
(I'll delete this comment if you've read it and you want it gone. I think the above can be very bad advice to give some poorly aligned selfish researchers, but I want reasonable people to hear it.)
EA: We should never trust ourselves to do act utilitarianism, we must strictly abide by a set of virtuous principles so we don't go astray.
Also EA: It's ok to eat animals as long as you do other world-saving work. The effort and sacrifice it would take to relearn my eating patterns just isn't worth it on consequentialist grounds.
Sorry for the strawmanish meme format. I realise people have complex reasons for needing to navigate their lives the way they do, and I don't advocate aggressively trying to make other people stop eating animals. The point is just that I feel like the seemingly universal disavowment of utilitarian reasoning has been insufficiently vetted for consistency. If we claim that utilitarian reasoning can be blamed for the FTX catastrophe, then we should ask ourselves what else we should apply that lesson to; or we should recognise that FTX isn't a strong counterexample to utilitarianism, and we can still use it to make important decisions.
(I realised after I wrote this that the metaphor between brains and epistemic communities is less fruitfwl than it seems like I think, but it's still a helpfwl frame in order to understand the differences anyway, so I'm posting it here. ^^)
TL;DR: I think people should consider searching for giving opportunities in their networks, because a community that efficiently capitalises on insider information may end up doing more efficient and more varied research. There are, as you would expect, both problems and advantages to this, but it definitely seems good to encourage on the margin.
Some reasons to prefer decentralised funding and insider trading
I think people are too worried about making their donations appear justifiable to others. And what people expect will appear justifiable to others, is based on the most visibly widespread evidence they can think of.[1] It just so happens that that is also the basket of information that everyone else bases their opinions on as well. The net effect is that a lot less information gets considered in total.
Even so, there are very good reasons to defer to consensus among people who know more, not act unilaterally, and be epistemically humble. I'm not arguing that we shouldn't take these considerations into account. What I'm trying to say is that even after you've given them adequate consideration, there are separate social reasons that could make it tempting to defer, and we should keep this distinction is in mind so we don't handicap ourselves just to fit in.
Consider the community from a bird's eye perspective for a moment. Imagine zooming out, and seeing EA as a single organism. Information goes in, and causal consequences go out. Now, what happens when you make most of the little humanoid neurons mimic their neighbours in proportion to how many neighbours they have doing the same thing?
What you end up with is a Matthew effect not only for ideas, but also for the bits of information that get promoted to public consciousness. Imagine ripples of information flowing in only to be suppressed at the periphery, way before they've had a chance to be adequately processed. Bits of information accumulate trust in proportion to how much trust they already have, and there are no well-coordinated checks that can reliably abort a cascade past a point.
To be clear, this isn't how the brain works. The brain is designed very meticulously to ensure that only the most surprising information gets promoted to universal recognition ("consciousness"). The signals that can already be predicted by established paradigms are suppressed, and novel information gets passed along with priority.[2] While it doesn't work perfectly for all things, consider just the fact that our entire perceptual field gets replaced instantly every time we turn our heads.
And because neurons have been harshly optimised for their collective performance, they show a remarkable level of competitive coordination aimed at making sure there are no informational short-circuits or redundancies.
Returning to the societal perspective again, what would it look like if the EA community were arranged in a similar fashion?
I think it would be a community optimised for the early detection and transmission of market-moving information--which in a finance context refers to information that would cause any reasonable investor to immediately make a decision upon hearing it. In the case where, for example, someone invests in a company because they're friends with the CEO and received private information, it's called "insider trading" and is illegal in some countries.
But it's not illegal for altruistic giving! Funding decisions based on highly valuable information only you have access to is precisely the thing we'd want to see happening.
If, say, you have a friend who's trying to get time off from work in order to start a project, but no one's willing to fund them because they're a weird-but-brilliant dropout with no credentials, you may have insider information about their trustworthiness. That kind of information doesn't transmit very readily, so if we insist on centralised funding mechanisms, we're unknowingly losing out on all those insider trading opportunities.
Where the architecture of the brain efficiently promotes the most novel information to consciousness for processing, EA has the problem where unusual information doesn't even pass the first layer.
(I should probably mention that there are obviously biases that come into play when evaluating people you're close to, and that could easily interfere with good judgment. It's a crucial consideration. I'm mainly presenting the case for decentralisation here, since centralisation is the default, so I urge you keep some skepticism in mind.)
There are no way around having to make trade-offs here. One reason to prefer a central team of highly experienced grant-makers to be doing most of the funding, is that they're likely to be better at evaluating impact opportunities. But this needn't matter much if they're bottlenecked by bandwidth--both in terms of having less information reach them and in terms of having less time available to analyse what does come through.[3]
On the other hand, if you believe that most of the relevant market-moving information in EA is already being captured by relevant funding bodies, then their ability to separate the wheat from the chaff may be the dominating consideration.
While I think the above considerations make a strong case for encouraging people to look for giving opportunities in their own networks, I think they apply with greater force to adopting a model like impact markets.
They're a sort of compromise between central and decentralised funding. The idea is that everyone has an incentive to fund individuals or projects where they believe they have insider information indicating that the project will show itself to be impactfwl later on. If the projects they opportunistically funded at an early stage do end up producing a lot of impact, a central funding body rewards the maverick funder by "purchasing the impact" second-hand.
Once a system like that is up and running, people can reliably expect the retroactive funders to make it worth their while to search for promising projects. And when people are incentivised to locate and fund projects at their earliest bottlenecks, the community could end up capitalising on a lot more (insider) information than would be possible if everything had to be evaluated centrally.
(There are of course, more complexities to this, and you can check out the previous discussions on the forum.)
- ^
This doesn't necessarily mean that people defer to the most popular beliefs, but rather that even if they do their own thinking, they're still reluctant to use information that other people don't have access to, so it amounts to nearly the same thing.
- ^
This is sometimes called predictive processing. Sensory information comes in and gets passed along through increasingly conceptual layers. Higher-level layers are successively trying to anticipate the information coming in from below, and if they succeed, they just aren't interested in passing it along.
(Imagine if it were the other way around, and neurons were increasingly shy to pass along information in proportion to how confused or surprised they were. What a brain that would be!)
- ^
As an extreme example of how bad this can get, an Australian study on medicinal research funding estimated the length of average grant proposals to be "between 80 and 120 pages long and panel members are expected to read and rank between 50 and 100 proposals. It is optimistic to expect accurate judgements in this sea of excessive information." -- (Herbert et al., 2013)
Luckily it's nowhere near as bad for EA research, but consider the Australian case as a clear example of how a funding process can be undeniably and extremely misaligned with the goal producing good research.
Hm, I think you may be reading the comment from a perspective of "what actions do the symbols refer to, and what would happen if readers did that?" as opposed to "what are the symbols going to cause readers to do?"[1]
The kinds of people who are able distinguish adequate vs inadequate good judgment shouldn't be encouraged to defer to conventional signals of expertise. But those are also disproportionately the people who, instead of feeling like deferring to Eliezer's comment, will respond "I agree, but..."
- ^
For lack of a better term, and because there should be a term for it: Dan Sperber calls this the "cognitive causal chain", and contrasts it with the confabulated narratives we often have for what we do. I think it summons up the right image.
When you read something, aspire to always infer what people intend based on the causal chains that led them to write that. Well, no. Not quite. Instead, aspire to always entertain the possibility that the author's consciously intended meaning may be inferred from what the symbols will cause readers to do. Well, I mean something along these lines. The point is that if you do this, you might discover a genuine optimiser in the wild. : )
Ideally, EigenTrust or something similar should be able to help with regranting once it takes off, no? : )
Really intrigued by the idea of debates! I was briefly reluctant about the concept at first, because what I associate with "debates" is usually from politics, religious disputes, debating contests, etc. where the debaters are usually lacking so much of essential internal epistemic infrastructure that the debating format often just makes it worse. Rambly, before I head off to bed:
- Conditional on it being good for EA to have more of a culture for debating, how would we go about practically bring that about?
- I wonder if EA Global features debates. I haven't seen any. It's mostly just people agreeing with each other and perhaps adding some nuance.
- You don't need to have people hostile towards each other in order for it to qualify as "debate", I do think one of the key benefits of debates is that the disagreement is visible.
- For one, it primes the debaters to hone in on disagreements, whereas perhaps EA in-group are overly primed to find agreements with each other in order to be nice.
- Making disagreements more visible will hopefwly dispel the illusion that EA as a paradigm is "mostly settled", and get people to question assumptions. This isn't always the best course of action, but I think it's still very needed on the margin, and could get into why if asked.
- If the debate (and the mutually-agreed-upon mindset of trying to find each others' weakest points) is handled well, it can onlookers feel like head-on disagreeing is more ok. I think we're mostly a nice community, reluctant to step on toes, so if we don't see any real disagreements, we might start to feel like the absence of disagreement is the polite thing to do.
- A downside risk is that debating culture is often steeped in the "world of arguments", or as Nate Soares put it: "The world is not made of arguments. Think not "which of whese arguments, for these two opposing sides, is more compelling? And how reliable is compellingness?" Think instead of the objects the arguments discuss, and let the arguments guide your thoughts about them."
- We shouldn't be adopting mainstream debating norms, it won't do anything for us. What I'm excited about is the idea making spaces for good-natured visible disagreements where people are encouraged to attack each others' weakest points. I don't think that mindset comes about naturally, so it could make sense to deliberately make room for it.
- Also, if you want people to debate you, maybe you should make a shortlist of the top things you feel would be productive to debate you on. : )
Just wanted to say, I love the 500-word limit. A contest that doesn't goodhart on effort moralization!
Oh, this is excellent! I do a version of this, but I haven't paid enough attention to what I do to give it a name. "Blurting" is perfect.
I try to make sure to always notice my immediate reaction to something, so I can more reliably tell what my more sophisticated reasoning modules transforms that reaction into. Almost all the search-process imbalances (eg. filtered recollections, motivated stopping, etc.) come into play during the sophistication, so it's inherently risky. But refusing to reason past the blurt is equally inadvisable.
This is interesting from a predictive-processing perspective.[1] The first thing I do when I hear someone I respect tell me their opinion, is to compare that statement to my prior mental model of the world. That's the fast check. If it conflicts, I aspire to mentally blurt out that reaction to myself.
It takes longer to generate an alternative mental model (ie. sophistication) that is able to predict the world described by the other person's statement, and there's a lot more room for bias to enter via the mental equivalent of multiple comparisons. Thus, if I'm overly prone to conform, that bias will show itself after I've already blurted out "huh!" and made note of my prior. The blurt helps me avoid the failure mode of conforming and feeling like that's what I believed all along.
Blurting is a faster and more usefwl variation on writing down your predictions in advance.
- ^
Speculation. I'm not very familiar with predictive processing, but the claim seems plausible to me on alternative models as well.
If Will thought SBF was a "bad egg", then it could be more important to establish influence with him, because you don't need to establish influence (as in 'willingness to cooperate') with someone who is entirely value-aligned with you.
Yes! That should work fine. That's 21:00 CET for me. See you then!
My email is emrik.asheim@gmail.com btw.
I think it'd be easy to come up with highly impactfwl things to do with free reign over Twitter? Like, even before I've thought about it, there should be a high prior on usefwl patterns. Brainstorming:
- Experiment with giving users control over recommender algorithms, and/or designing them to be in the long-term interests of the users themselves (because you're ok with foregoing some profit in order to not aggressively hijacking people's attention)
- Optimising the algorithms for showing users what they reflectively prefer (eg. what do I want to want to see on my Twitter feed?)[1]
- Optimising algorithms for making people kinder (eg. downweighting views that come from bandwagony effects and toxoplasma), but still allowing users to opt-out or opt-in, and clearly guiding them on how to do so.
- Trust networks
- Liquid democracy-like transitive trust systems (eg. here, here)
- I can see several potential benefits to this, but most of the considerations are unknown to me, which just means that there could still be massive value that I haven't seen yet.
- This could be used to overcome Vingean deference limits and allow for hiring more competent people more reliably than academic credentials (I realise I'm not explaining this, I'm just pointing to the existence of ideas enabled with Twitter)
- This could also be a way to "vote" for political candidates or decision-makers in general too, or be used as a trust metric to find out whether you want to vote for particular candidates in the first place.
- Liquid democracy-like transitive trust systems (eg. here, here)
- Platform to arrange vote swapping and similar, allow for better compromises and reduce hostile zero-sum voting tendencies.
- Platform for highly visible public assurance contracts (eg. here), could be potentially be great for cooperation between powerfwl actors or large groups of people.
- This also enables more visibility for views that held back by pluralistic ignorance. This could be both good and bad, depending on the view (eg. both "it's ok to be gay" and "it's not ok to be gay" can be held back by pluralistic ignorance).
- Could also be used to coordinate actions in a crisis
- eg. the next pandemic is about to hit, and it's a thousand times more dangerous than covid, and no one realises because it's still early on the exponential curve. Now you utilise your power to influence people to take it seriously. You stop caring about whether this will be called "propaganda" because what matters isn't how nice you'll look to the newspapers, what matters is saving people's lives.
- Something-something nudging idk.
Mostly, even if I thought Sam was in the wrong for considering a deal with Elon, I find it strange to cast a negative light on Will for putting them in touch. That seems awfwly transitive. I think judgments for transitive associations are dangerous, especially given incomplete information. Sam/Will probably thought much longer on this than I have, so I don't think I can justifiably fault their judgment even if I had no ideas on how to use twitter myself.
- ^
This idea was originally from a post by Paul Christiano some years ago where he urged FB to adopt an algorithm like this, but I can't seem to find it rn.
Hmm, I suspect that anyone who had the potential to be bumped over the threshold for interest in EA, would be likely to view the EA Wikipedia article positively despite clicking through to it via SBF. Though I suspect there are a small number of people with the potential to be bumped over that threshold. I have around 10% probability on that the negative news has been positive for the movement, primarily because it gained exposure. Unlikely, but not beyond the realm of possibility. Oo
Mh, I had in mind both, and wanted to leave it up for interpretation. A public debate about something could be cool because I've never done that, but we'd need to know what we're supposed to disagree about first. Though, I primarily just wish to learn from you, since you have a different perspective, so a private call would be my second offer.
I'm confused. You asked whether I had in mind a public or private call, and I said I'd be fine with either. Which question are you referring to?
Happy to either, but I'll stay off-camera if it's going to be recorded. Up to you, if you wish to prioritise it. : )
This is excellent. I hadn't even thought to check. Though I think I disagree with the strength of your conclusion.
If you look back since 2015 ("all time"), it looks like this.[1] Keep in mind that Sam has 1.1M pageviews since before November[2] (before anyone could know anything about anything). Additionally, if you browse his Wikipedia page, EA is mentioned under "Careers", and is not something you read about unless particularly interested. On the graph, roughly 5% click through to EA. (Plus, most of the news-readers are likely to be in the US, so the damage is localised?)
A point of uncertainty for me is to what extent news outlets will drag out the story. Might be that most of the negative-association-pageviews are yet to come, but I suspect not? Idk.
Not sure how to make the prediction bettable but I think I'm significantly less pessimistic about the brand value than you seem to be.
Feel free to grab these if you want to make a post of it. Seems very usefwly calibrating.
Hmm, not sure how that job would work, but if someone could be paid to fix all of EAs coordination problems, that'd probably be worth the money. It is the responsibility of anyone who wishes to assume that responsibility. And if they can, I really hope they do.
I'm a very slow reader, but do you wish to discuss (or debate it!) over a video call sometime? I'm eager to learn things from someone who's likely to have a different background on the questions I'm interested in. : )
I also made a suggestion on Sasha's post related to nudging people's reading habits by separating out FTX posts by default. I don't endorse the design, but it could look something like this.[1] Alternatively, could introduce 'tag profiles' or something, where you can select a profile, and define your filters within each profile.[2]
(P.S. Sorry for the ceaseless suggestions, haha! Brain goes all sparkly with an idea and doesn't shut up until I make a comment about it. ^^')
I have an unusual perspective on this. I skim nearly everything, and rarely see the benefit of completionism in reading. And I don't benefit from the author being nuanced or give different views a balanced take. I don't care about how epistemically virtuous the author is being! I just care about learning usefwl patterns from the stuff I'm reading, and for that, there's no value to me knowing whether the author is obviously biased in favour of a view or not. There are more things to say here, but I say some of them here.
Btw, I appreciate your contributions to the forum, at least for what I've seen of it. : )
I feel like quite a few people are working on things related to this, with approaches I have different independent impressions about, but I'm very happy there's a portfolio.
Manifold Markets, Impact Markets, Assurance Contracts, Trust Networks, and probably very obvious stuff I'm forgetting right now but I thought I'd quickly throw these in here. I'm also kinda working on this, but it's in parallel with other things and it mostly consists of a long path of learning and trying to build up understanding of things.
I don't know of standardised methods that I think are likely to more reliably generate that, and I am worried about rushing standardisation prematurely. But this worry is far from insurmountable, and I'd be happy to hear suggestions for things that should be standardised. You have any? : )
There are both positives and negatives to being addicted to the EA Forum, so I'm not sure I'm against it entirely. Also, I think the current gatekeeper(s) (mainly Lizka atm, I think) have better judgment than the aggregated wisdom of accumulated karma, so I really appreciate the curation. Weighted upvotes are controversial, and I could change my mind, but for now I suspect the benefits outweigh the costs because I trust the judgment of high-karma-power users more on average. For an alternative way to filter posts, I think Zoe William's weekly summaries are excellent.
Ideally, it would be cool if EA karma could be replaced with a distributed trust network of sorts, such as one I alluded to on LW.
Big agree! I call it the "frontpage time window". To mitigate the downsides, I suggested resurfacing curated posts on a spaced-repetition cycle to encourage long-term engagement, and/or breaking out a subset of topics into "tabs" so the forum effectively becomes something like a hybrid model of the current system and the more traditional format.
Huh! I literally browsed this paper of his earlier today, and now I find it on the forum. Weird!
Anyway, Michael Nielsen is great. I especially enjoy his insights on spaced repetition.
"What is the most important question in science or meta science we should be seeking to understand at the moment?"
Imo, the question is not "how can we marginally shift academic norms and incentives so the global academic workforce is marginally more effective," but instead "how can we build an entirely independent system with the correct meta-norms at the outset, allowing knowledge workers to coordinate around more effective norms and incentive structures as they're discovered, while remaining scalable and attractive[1] enough to compete with academia for talent?"
Even if we manage to marginally reform academia, the distance between "marginal" and "adequate" reform seems insurmountable. And I think returns from precisely aimed research are exponential, such that a minority of precisely aimed researchers achieve good compared to a global workforce mostly compelled to do elaborate rituals of paper pushing. To be clear, this isn't their fault. It's the result of a tightly regulated bureaucratic process that's built up too much design debt to refactor. It's high time to jump ship.
- ^
in profit and prestige
I'm in favour of this!
While you can click "hide" topic, I had forgotten about it, and it wasn't obvious how to before I read the comments. Besides, I agree that it would be healthy for the community if the Frontpage wasn't inundated with FTX posts. (After I hid the topic, the forum suddenly felt less angry somehow. Oo)
Here's a suggestion for how it could look.[1]
- ^
When you're on the Frontpage, you see fewer Community posts. With a separate tab for Community, I think it would be fine if FTX crisis posts were hidden by default or something, or hot-topic posts were given a default negative modifier. (We don't want the forum to waste time.)
I suppose it would also be nice to have an option to collapse the tags so they take up less space.[2]
- ^
Not sure I agree, but then again, there's no clear nailed-down target to disagree with :p
For particular people's behaviour in a social environment, there's a high prior that the true explanation is complex. That doesn't nail down which complex story we should update towards, so there's still more probability mass in any individual simpler story than in individual complex stories. But what it does mean is that if someone gives you a complex story, you shouldn't be surprised that the story is complex and therefore reduce your trust in them--at least not by much.
(Actually, I guess sometimes, if someone gives you a simple story, and the prior on complex true stories is really high, you should distrust them more. )
To be clear, if someone has a complex story for why they did what they did, you can penalise that particular story for its complexity, but you should already be expecting whatever story they produce to be complex. In other words, if your prior distribution over how complex their story will be is nearly equal to your posterior distribution (the complexity of their story roughly fits your expectations), then however much you think complexity should update your trust in people, you should already have been distrusting them approximately that much based on your prior. Conservation of expected evidence!
I made an entry to Arbital on absorbing barriers to test it out, copied below. Sorta want to bring Arbital back (with some tweaks), or implement something similar with the tags on the EA forum. It's essentially a collaborative knowledge net, and it could have massive potential if people were taught how to benefit from and contribute to it.
When playing a game that involves making bets, avoid naïvely calculating expected utilities without taking the expected cost of absorbing barriers into account.
An absorbing barrier in a dynamical system is the state in possibility space from which it may never return.
It's a term from Taleb, and the canonical example is when you're playing poker and you've lost too much to keep playing. You're out of the game.
- In longtermism, the absorbing barrier could be extinction or a dystopian lock-in.
- In the St. Petersburg Paradox, the absorbing barrier is the first lost bet.
- In conservation biology, the extinction threshold of a species is an absorbing barrier where a parameter (eg. population size) dips below a critical value where they are no longer able to reproduce to replace their death rate, leading to gradual extinction.
- In evolutionary biology, the error threshold is the rate of mutation above which DNA loses too much information between generations that beneficial mutations cannot reach fixation (stability in the population). In the figure below, the model shows the proportion of population carrying a beneficial hereditary sequence over the mutation rate. The sequence only reaches fixation when the mutation rate (1-Q) goes about ~0.05. This either prevents organisms from evolving in the first place, or, when the mutation rate suddenly increases due to environmental radiation, may constitute an extinction threshold.
A system is said to undergo a Lindy effect if its expected remaining lifespan is proportional to its age. It also describes a process where the distance from an absorbing barrier increases over time. If a system recursively accumulates robustness to extinction, it is said to have Lindy longevity.
- In biology, offspring of r-selected species (Type III) usually exhibit a Lindy effect as they become more adapt to the environment over time.
- In social dynamics, the longer a social norm has persisted, the longer it's likely to stick around.
- In programming, the longer a poor design choice at the start has persisted, the longer it's likely to remain. This may happen due to build-up of technical debt (eg. more and more other modules depend on the original design choice), making it harder to refactor the system to accommodate a better replacement.
- In relationships, sometimes the longer a lie or an omission has persisted, the more reluctant its originator is likely to be correct it. Lies may increase in robustness over time because they get entangled with other lies that have to be maintained in order to protect the first lie, further increasing the reputational cost of correction.
When playing a game--in the maximally general sense of the term--that you'd like to keep playing for a very long time, avoid naïvely calculating expected utilities without taking the expected cost of absorbing barriers into account. Aim for strategies that take you closer to the realm of Lindy longevity.
...what? I haven't been outside for a spell, and I knew things were bad, but that's just broken.
(We can fix it, though! With faith, patience, and a whole lot of sitting in office chairs!)
Actually appreciate this comment. I should've been more clear about when I was using universal vs existential quantifiers and anything in between. I do not advocate that everyone should withhold anger, because perhaps (as is likely) some people do in fact know much more than me, and they know enough that anger is justified.
I should clarify that the harm I envision is not mostly about Sam or others at FTX. It's the harm I imagine indirectly caused to the movement, and by the movement, by condoning insufficiently-informed bandwagons of outrage and pile-on ridicule. It harms our alignment, our epistemic norms, and our social culture; and thereby harms our ability to do good in the world.
Anger, ostracism--heck, even violence--seems less likely to misfire than ridicule. Ridicule is about having fun at another's expense, and that's just an exceedingly dangerous tool even when wielded with good intentions (which I highly doubt has been the primary motivation most people have had for using it).
(Thanks for highlighting these questions.)
I don't object to people condemning him if they know more than me. I clarified what I meant in response to David's comment below.
Mh, crux is wrong. My objections are consistent with my past behaviour in similar situations.
- I am not categorically defending Sam from everything. I am conditionally defending him from a subset of things. Though I think his welfare is important, my primary purpose here isn't about that. (I do think his welfare matters, just as anyone should have their core dignity as a sentient being respected, regardless of who they are or what they've done.)
- I would write something equivalent to this post regardless of whether I believed Sam had done something unethical,[1] because I think some of the community's response was, in part, unhealthy and dangerous either way.
- When it involves outrage, our epistemic rigour and reluctance to defer to mass opinion should be much stricter than baseline. What happened instead was that people inferred Sam's guilt by how confidently outraged their peers were. And because in our present culture it's really hard to believably signal condemnation without also signalling confidence, this is a recipe for explosive information cascades/bandwagons. This is extremely standard psychology, and something we should as a high-trust community be unceasingly wary of. For this reason primarily, we should be very--but not infinitely--reluctant to enforce "failing to condemn" as a crime.
- I don't object to people condemning his actions, especially not to the people who are clearly conditionalising their condemnation on seeing specific evidence. I'm not claiming other people don't know more than me, and they might have much stronger reasons to be confident.
- Ridicule is more tangential to the harm, and has much more associative overlap with cruelty compared to anger and condemnation. Ridicule doesn't even pretend to be about justice (usually). If ridicule must be used, it works better as a tool for diminishing people in power, when you want them to have less power; when someone is already at the bottom, ridicule is cruelty. (Maybe the phase shift in power was so sudden that people failed to notice that they are now targeting someone who's suddenly very vulnerable to abuse.)
- ^
I have my object-level probabilities, but part of my point is how expected I am to reveal them, which makes me think I should leave it ambiguous, at least in public. "Which side are you on?!" <- Any social instincts that pressure a resolution to this question should be scrutinised with utmost suspicion.
[Wasn't trying to attack the FB group. I'm glad it exists.]
But I was more curious to ask: why do you say Against Pandemics is extremely shady? It's the first time I've heard of them.
I assume nonprofits and independents who were kept afloat by FTX money have to find alternative sources of funding (or otherwise be forced to shut down). But do you mean they should abstain from even doing that?
Welcome to the forum! I agree that EAs often have a really troubling relationship with their own feelings, and scruples to a fault. If you have strong reason to believe that Sam acted unethically, I have no objections against directing your feelings of anger at him. But I would urge people to carry their anger with dignity, both for the sake of community norms and their own sense of self-worth.
While I agree that humour is a great de-stressor, I have faith in our ability to find alternative ways to entertain ourselves that don't involve kicking someone while they're down.
:)
I haven't updated appreciably in this direction after this, but I appreciate you calling attention to it.
Curated posts could resurface to the frontpage at exponentially decaying intervals.
- Counteracts recency bias. Enables longer-term discussions.
- Increases exposure (and over a more varied reader population) to the most important ideas.
- Efficiently[4] increases collective memory of the best contributions.
- We might uncover and dislodge some flawed assumptions that reached universal acceptance in the past due to information cascades.
- Given recency bias combined with the fact that people are very reluctant to write things that have been written about before, we could theoretically be losing wisdom over time instead of accumulating it. Especially since the movement is growing fast, and newcomers weren't here when a particular piece of wisdom was under discussion the first time around.
Wrote a shortform on it. Would be cool to have, imo! : )
The 'frontpage time window' is the duration a post remains on the frontpage. With the increasing popularity of the forum, this window becomes shorter and shorter, and it makes everyone scramble to participate in the latest discussions before the post disappears into irrelevancy.
I call it the "jabber loop". As long as we fear being exposed as clueless about something, we're incentivised to read what we expect other people will have read, and what they're likely to bring up in conversation.
This seems suboptimal if it biases us towards only appreciating what's recent and crowds out the possibility of longer, more patient, discussions. One solution to this could be spaced-repetition curated posts.[1]
When a post gets curated, that should indicate that it has some long-term value. So instead of (or in addition to) pinning it on top of the frontpage, the star could indicate that this post will resurface to the frontpage at a regular interval that decays exponentially (following the forgetting curve).[2][3]
Some reasons this could be good
- It lets readers know that this discussion will resurface, so their contributions could also have lasting value. Comments are no longer write-and-forget, and you have a real chance at contributing more long-term.
- It efficiently[4] increases collective memory of the best contributions.
- It can help us scrutinise ideas that got inculcated as a fundamental assumption early on. We might uncover and dislodge some flawed assumptions that reached universal acceptance in the past due to information cascades.
- As we gain more information and experience over time, we might stand a better chance at discover flaws in previously accepted premises. But unless there's something (like spaced-repetition curation) that prompts the question into public debate again, we don't get to benefit from that increased capacity, and we may just be stuck with the beliefs that got accepted earlier.
- Given that there's a large bias to discuss what's recent, combined with the fact that people are very reluctant to write up ideas that have already been said before, we might be stuck with a bit of a paradox. If the effects were strong enough, we could theoretically be losing wisdom over time instead of accumulating it. Especially since the movement is growing fast, and newcomers weren't here when a particular piece of wisdom was a hot topic the first time around.
- ^
There are other creative ways the forum could use spaced repetition to enhance learning, guide attention, and maximise the positive impact of reminders at the lowest cost.
It could either be determined individually (e.g. personal flashcards similar to Orbit), collectively (e.g. determined by people upvoting it for 'long-term relevancy' or something), or centrally (e.g. by Lizka and other content moderators).
- ^
Andy Matuschak, author of Quantum Country and Evergreen Notes, calls this a timefwl text. He also developed Orbit, a tool that should help authors integrate flashcards into their educational writings. If the forum decided to do something like this, he might be eager to help facilitate it. Idk tho.
- ^
Curators could still decide to un-curate a post if it's no longer relevant or they don't think the community benefits from retaining it in their epistemic memepool.
- ^
I highly recommend Andy's notes on spaced repetition and learning in general.
I don't have opinions on any of the specific details, but what I do have is a feeling that darkness has stuck a great blow against us. There's stuff that needs to get done, and recent events have imperiled our ability to do them. I'm scared, but most of all I want to help make up for the loss. Not by trying harder, but by persevering in the things I do with what I have to do them with.
"To save the world, I will start by doing the proper and humble things I know how to do within the confines of my own life."
To clarify, was it this sentence you found confrontational? (I'm not counter-arguing, I am genuinely asking, because I seem to lack an eye for this sort of thing, or alternatively I'm usually right and most people are wrong. The truth is probably in the middle somewhere if I were to guess.)
I haven't followed this much but someone who seems to know things says that a lot of Nathan Young's claims (here and on Twitter) are sort of false/overblown.
Thanks for writing this up! One of the big advantages to independent research imo is that it's much easier to pivot whenever you want, without having to justify yourself to anyone (well, depends on what kind of independence you have). Let's you iterate across projects much faster, so it's easier to end up finding a project that's truly worth your time.
(There are more arguments why subforums would be cool, but I shouldn't be spending time elucidating them rn unless anyone's curious >.<)
Yes, and I would love a meta-research/rationality/socioepistemology subforum!
It would be usefwl because it might get easier to find and talk to the people who are especially interested in the topic. And because posts on the frontpage gets eaten by the time window in a swish and a swosh, it rarely leads to lasting discussion and people feel like they have to hurry (which is partly positive, partly negative).
The tag system would work just the same if people used it (I would actually prefer it), but since tag usage is inconsistent, people can't rely on it.[1]
- ^
Solutions to inconsistent tag usage could be to prevent people from clicking the "post" button before they've added at least one tag, or paying someone to manually tag posts, or making an AI help you do it, etc.
I found this image from Lin's map of pandemic preparedness really helpfwl. I think it could be profoundly usefwl to have a list of dense infographics like these for various cause areas. Backchaining is something that experts can do for the community; forward-chaining involves more near-term and individual variables that newcomers have to sort out for themselves. Having maps backchained like this lets newcomers identify the available proxy-goals, and forward-chain from their own situation to connect up to any of the pathways on the graphic.
Sorry if this is only tangentially relevant, but I honestly think more courses, discussion groups, and especially virtual programs could benefit from using the EA Gather Town for their sessions. This doesn't suit everyone, of course, but I think there are a lot of people for whom it would be optimal on the margin. I would be happy to help with this in any way I can.[1] Get in touch if you're interested. : )
Yellow hosted some unofficial intro course cohorts here, and one of them became a regular coworker, and several others have returned to the space every now and then. (Yellow actually invited the students and hosted the courses on their own initiative, and they made a guide! Needless to say, Yellow is pretty awesome.)
One-off events that people travel to are really great for inspiration, learning seriously, and strong connections. But there are significant obstacles to keeping up those connections after people return home to their daily routines. The environments (locale, incentives, activities) where they made the connections are often very different from their habitual environments where they'd have to find a way to maintain the connections. If they live far apart, they might not be the kind of people who have much bandwidth for communicating online, so the connection fades despite wanting to keep in touch.
For fostering long-term high-communication connections between EAs, I suspect local or online activities are underexplored. Events that are more specifically optimised for kickstarting a perpetual social activity (e.g. coworking, or regular meetups in a place they can always return to) for those who want it seem more likely to enable people to keep in touch, and EA Gather is great for that. Probably locally hosted activities work too, but I don't know much about them.
- ^
Either me or any of the other stewards could give quick intro tours to newcomers on e.g. how to connect with others via the space, community norms, benefits of coworking, etc. We could also build out or customise the space for what people want to use it for, but we have plenty of space so we might already have what you need for what you want to do.
Thanks a lot : )
(Honestly just posting comments on posts linking to relevant stuff you can think of is both cheap and decent value.)
Fair points. On the third hand, the more AGI researchers there are, the more "targets" there are for important arguments to reach, and the higher impact systematic AI governance interventions will have.
At this point, I seem to have lost track of my probabilities somewhere in the branches, let me try to go back and find it...
Good discussion, ty. ^^
You'd be well able to compute the risk on your own, however, if you seriously considered doing any big outreach efforts. I think people should still have a large prior on action for anything that looks promising to them. : )
This is confused, afaict? When comparing the impact of time-buying vs direct work, the probability of success for both activities is negated by the number of people pushing capabilities. So it cancels out, and you don't need to think about the number of people in opposition.
The unique thing about time-buying is that its marginal impact increases with the number of alignment workers,[1] whereas the marginal impact of direct work plausibly decreases with the number of people already working on it (due to fruit depletion and coordination issues).[2]
If there are 300 people doing direct alignment and you're an average worker, you can expect to contribute 0.3% of the direct work that happens per unit time. On the other hand, if you spend your time on time-buying instead, you only need to expect to save 0.3% units of time per unit of time you spend in order to break even.[3]
- ^
Although the marginal impact of every additional unit of time scales with the number of workers, there are probably still diminishing returns to more people working on time-buying.
- ^
Probably direct work scales according to some weird curve idk, but I'm guessing we're past the peak. Two people doing direct work collaboratively do more good per person than one person. But there are probably steep diminishing marginal returns from economy-of-scale/specialisation, coordination, and motivation in this case.
- ^
Impact is a multiplication of the number of workers , their average rate of work , and the time they have left to work . And because multiplication is commutative, if you increase one of the variables by a proportion , that is equivalent to increasing any of the other variables with the same proportion. .