How to make independent research more fun (80k After Hours) 2023-03-17T22:25:11.419Z
80k podcast episode on sentience in AI systems 2023-03-15T20:19:09.424Z
What to think when a language model tells you it's sentient 2023-02-20T02:59:26.808Z
Key questions about artificial sentience: an opinionated guide 2022-04-25T13:42:59.799Z
The pretty hard problem of consciousness 2021-08-20T10:30:41.967Z
Writing about my job: Research Fellow, FHI 2021-07-29T13:53:50.082Z


Comment by rgb on Common-sense sentience is dubious · 2023-03-22T14:39:45.970Z · EA · GW

Hi Timothy! I agree with your main claim that "assumptions [about sentience] are often dubious as they are based on intuitions that might not necessarily ‘track’ sentience", shaped as they are by potentially unreliable evolutionary and cultural factors. I also think it's a very important point! I commend you for laying it out in a detailed way.

I'd like to offer a piece of constructive criticism if I may. I'd add more to the piece that answers, for the reader:

  1. what kind of piece am I reading? What is going to happen in it?
  2. why should I care about the central points? (as indicated, I think there are many reasons to care, and could name quite a few myself)
  3. how does this piece relate to what other people say about this topic?

While getting 'right to the point' is a virtue, I feel like more framing and intro would make this piece more readable, and help prospective readers decide if it's for them.

[meta-note: if other readers disagree, please do of course vote 'disagree' on this comment!]

Comment by rgb on What to think when a language model tells you it's sentient · 2023-03-17T17:14:02.069Z · EA · GW

Hi Brian! Thanks for your reply. I think you're quite right to distinguish between your flavor of panpsychism and the flavor I was saying doesn't entail much about LLMs. I'm going to update my comment above to make that clearer, and sorry for running together your view with those others.

Comment by rgb on What to think when a language model tells you it's sentient · 2023-02-28T20:51:06.510Z · EA · GW

Ah, thanks! Well, even if it wasn't appropriately directed at your claim, I appreciate the opportunity to rant about how panpsychism (and related views) don't entail AI sentience :)

Comment by rgb on What to think when a language model tells you it's sentient · 2023-02-27T22:33:22.264Z · EA · GW

The Brian Tomasik post you link to considers the view that fundamental physical operations may have moral weight (call this view "Physics Sentience"). 

[Edit: see Tomasik's comment below. What I say below is true of a different sort of Physics Sentience view like constitutive micropsychism, but not necessarily of Brian's own view, which has somewhat different motivations and implications]

But even if true, [many versions of] Physics Sentience [but not necessarily Tomasik's] doesn't have straightforward implications about what high-level systems, like organisms and AI systems, also comprise a sentient subject of experience. Consider: a human being touching a stove is experiencing pain on Physics Sentience; but a pan touching a stove is not experiencing pain. On Physics Sentience, the pan is made up of sentient matter, but this doesn't mean that the pan qua pan is also a moral patient, another subject of experience that will suffer if it touches the stove.

To apply this to the LLMs case: 

-Physics Sentience will hold that the hardware on which LLMs run is sentient - after all, it's a bunch of fundamental physical operations.

-But Physics Sentience will also hold that the hardware on which a giant lookup table is running is sentient, to the same extent and for the same reason.

-Physics Sentience is silent on whether there's a difference between (1) and (2), in the way that there's a difference between the human and the pan.

The same thing holds for other panpsychist views of consciousness, fwiw. Panpsychist views that hold that fundamental matter is consciousness don't tell us anything, themselves, about what animals or AI systems are sentient. It just says they are made of conscious (or proto-conscious) matter.

Comment by rgb on What to think when a language model tells you it's sentient · 2023-02-24T07:28:48.615Z · EA · GW

I like it! I think one thing the post itself could have been clearer on is that reports could be indirect evidence for sentience, in that they are evidence of certain capabilities that are themselves evidence of sentience. To give an example (though it’s still abstract), the ability of LLMs to fluently mimic human speech —> evidence for capability C—> evidence for sentience. You can imagine the same thing for parrots: ability to say “I’m in pain”—> evidence of learning and memory —> evidence of sentience. But what they aren’t are reports of sentience.

so maybe at the beginning: aren’t “strong evidence” or “straightforward evidence”

Comment by rgb on What to think when a language model tells you it's sentient · 2023-02-24T02:07:29.781Z · EA · GW

Thanks for the comment. A couple replies:

I want to clarify that these are examples of self-reports about consciousness and not evidence of consciousness in humans.

Self-report is evidence of consciousness in Bayesian sense (and in common parlance): in a wide range of scenarios, if a human says they are conscious of something, you should have a higher credence than if they do not say they are. And in the scientific sense: it's commonly and appropriately taken as evidence in scientific practice; here is Chalmers's "How Can We Construct a Science of Consciousness?" on the practice of using self-reports to gather data about people's conscious experiences:

Of course our access to this data depends on our making certain assumptions: in particular, the assumption that other subjects really are having conscious experiences, and that by and large their verbal reports reflect these conscious experiences. We cannot directly test this assumption; instead, it serves as a sort of background assumption for research in the field. But this situation is present throughout other areas of science. When physicists use perception to gather information about the external world, for example, they rely on the assumption that the external world exists, and that perception reflects the state of the external world. They cannot directly test this assumption; instead, it serves as a sort of background assumption for the whole field. Still, it seems a reasonable assumption to make, and it makes the science of physics possible. The same goes for our assumptions about the conscious experiences and verbal reports of others. These seem to be reasonable assumptions to make, and they make the science of consciousness possible .

It's suppose it's true that self-reports can't budge someone from the hypothesis that other actual people are p-zombies, but few people (if any) think that. From the SEP:

Few people, if any, think zombies actually exist. But many hold that they are at least conceivable, and some that they are possible....The usual assumption is that none of us is actually a zombie, and that zombies cannot exist in our world. The central question, however, is not whether zombies can exist in our world, but whether they, or a whole zombie world (which is sometimes a more appropriate idea to work with), are possible in some broader sense.

So yeah: my take is that no one, including anti-physicalists who discuss p-zombies like Chalmers, really thinks that we can't use self-report as evidence, and correctly so.

Comment by rgb on What to think when a language model tells you it's sentient · 2023-02-24T01:18:22.470Z · EA · GW

Agree, that's a great pointer! For those interested, here is the paper and here is the podcast episode.

[Edited to add a nit-pick: the term 'meta-consciousness' is not used, it's the 'meta-problem of consciousness', which is the problem of explaining why people think and talk the way they do about consciousness]

Comment by rgb on What to think when a language model tells you it's sentient · 2023-02-20T19:06:01.826Z · EA · GW

Thank you!

Comment by rgb on [deleted post] 2022-12-21T12:15:24.773Z

I enjoyed this excerpt and the pointer to the interview, thanks. It might be helpful to say in the post who Jim Davies is.

Comment by rgb on Announcing the Future Fund's AI Worldview Prize · 2022-09-26T12:01:31.147Z · EA · GW

That may be right - an alternative would be to taboo the word in the post, and just explain that they are going to use people with an independent, objective track record of being good at reasoning under uncertainty.

Of course, some people might be (wrongly, imo) skeptical of even that notion, but I suppose there's only such much one can do to get everyone on board. It's a tricky balance of making it accessible to outsiders while still just saying what you believe about how the contest should work.

Comment by rgb on Announcing the Future Fund's AI Worldview Prize · 2022-09-24T10:56:08.257Z · EA · GW

I think that the post should explain briefly, or even just link to, what a “superforecaster” is. And if possible explain how and why this serves an independent check.

The superforecaster panel is imo a credible signal of good faith, but people outside of the community may think “superforecasters” just means something arbitrary and/or weird and/or made up by FTX.

(The post links to Tetlock’s book, but not in the context of explaining the panel)

Comment by rgb on Schisms are bad, actually: Community breakdown sucks · 2022-09-02T15:16:05.132Z · EA · GW

I think you mean “schisms”

Comment by rgb on (p-)Zombie Universe: another X-risk · 2022-07-30T17:22:45.202Z · EA · GW

You write,

Those who do see philosophical zombies as possible don’t have a clear idea of how consciousness relates to the brain, but they do think...that consciousness is something more than just the functions of the brain. In their view, a digital person (an uploaded human mind which runs on software) may act like a conscious human, and even tell you all about its ‘conscious experience’, but it is possible that it is in fact empty of experience.

It's consistent to think that p-zombies are possible but to think that, given the laws of nature, digital people would be conscious. David Chalmers is someone who argues for both views.

It might be useful to clarify that the questions of

(a) whether philosophical zombies are metaphysically possible (and the closely related question of physicalism about consciousness)

is actually somewhat orthogonal to the question of

(b) whether uploads that are functionally isomorphic to humans would be conscious

David Chalmers thinks that philosophical zombies are metaphysically possible, and that consciousness is not identical to the physical. But he also argues that, given the laws of nature in this world, uploaded minds, of sufficiently fine-grained functional equivalence to human minds, that act and talk like conscious humans would be conscious. In fact, he's the originator of the 'fading qualia' argument that Holden appeals to in his post.

On the other side, Ned Block thinks that zombies are not possible, and is a physicalist. But he also thinks that only biological-instantiated minds can be conscious.

Here's Chalmers (2010) on the distinction between the two issues:

I have occasionally encountered puzzlement that someone with my own property dualist views (or even that someone who thinks that there is a significant hard problem of consciousness) should be sympathetic to machine consciousness. But the question of whether the physical correlates of consciousness are biological or functional is largely orthogonal to the question of whether consciousness is identical to or distinct from its physical correlates. It is hard to see why the view that consciousness is restricted to creatures with our biology should be more in the spirit of property dualism! In any case, much of what follows is neutral on questions about materialism and dualism.

Comment by rgb on Reducing nightmares as a cause area · 2022-07-19T15:37:45.007Z · EA · GW

You might be interested in this LessWrong shortform post by Harri Besceli, "The best and worst experiences you had last week probably happened when you were dreaming." Including a comment from gwern.

Comment by rgb on What if we don't need a "Hard Left Turn" to reach AGI? · 2022-07-16T10:33:49.665Z · EA · GW

Thanks for the post! Wanted to flag a typo: “ To easily adapt to performing complex and difficult math problems, Minerva has That's not to say that Minerva is an AGI - it clearly isn't.”

Comment by rgb on Searle vs Bostrom: crucial considerations for EA AI work? · 2022-07-13T14:47:24.083Z · EA · GW

Well, I looked it up and found a free pdf, and it turns out that Searle does consider this counterargument.

Why is it so important that the system be capable of consciousness? Why isn’t appropriate behavior enough? Of course for many purposes it is enough. If the computer can fly airplanes, drive cars, and win at chess, who cares if it is totally nonconscious? But if we are worried about a maliciously motivated superintelligence destroying us, then it is important that the malicious motivation should be real. Without consciousness, there is no possibility of its being real.

But I find the arguments that he then gives in support of this claim quite unconvincing / I don't understand exactly what the argument is. Notice that Searle's argument is based on comparing a spell-checking program on a laptop with human cognition. He claims that reflecting on the difference between the human and the program establishes that it would never make sense to attribute psychological states to any computational system at all. But that comparison doesn't seem to show that at all.

And it certainly doesn't show, as Searle thinks it does, that computers could never have the "motivation" to pursue misaligned goals, in the sense that Bostrom needs to establish that powerful AGI could be dangerous.

I should say—while Searle is not my favorite writer on these topics, I think these sorts of questions at the intersection of phil mind and AI are quite important and interesting, and it's cool that you are thinking about them. (Then again, I *would *think that given my background). And it's important to scrutinize the philosophical assumptions (if any) behind AI risk arguments.

Comment by rgb on Announcing Non-trivial, an EA learning platform for teenagers · 2022-07-13T13:23:16.348Z · EA · GW

Feedback: I find the logo mildly unsettling. I think it triggers my face detector, and I see sharp teeth. A bit like the Radiohead logo.

On the other hand, maybe this is just a sign of some deep unwellness in my brain. Still, if even a small percentage of people get this feeling from the logo, could be worth reconsidering.

Comment by rgb on Searle vs Bostrom: crucial considerations for EA AI work? · 2022-07-13T11:08:19.805Z · EA · GW

Since the article is paywalled, it may be helpful to excerpt the key parts or say what you think Searle's argument is. I imagine the trivial inconvenience of having to register will prevent a lot of people from checking it out.

I read that article a while ago, but can't remember exactly what it says. To the extent that it is rehashing Searle's arguments that AIs, no matter how sophisticated their behavior, necessarily lack understanding / intentionality/ something like that, then I think that Searle's arguments are just not that relevant to work on AI alignment.

Basically I think what Chalmers says in his paper The Singularity: a Philosophical Analysis.

As for the Searle and Block objections, these rely on the thesis that even if a system duplicates our behavior, it might be missing important “internal” aspects of mentality: consciousness, understanding, intentionality, and so on. Later in the paper, I will advocate the view that if a system in our world duplicates not only our outputs but our internal computational structure, then it will duplicate the important internal aspects of mentality too. For present purposes, though, we can set aside these objections by stipulating that for the purposes of the argument, intelligence is to be measured wholly in terms of behavior and behavioral dispositions, where behavior is construed operationally in terms of the physical outputs that a system produces. The conclusion that there will be AI++ in this sense is still strong enough to be interesting. If there are systems that produce apparently superintelligent outputs, then whether or not these systems are truly conscious or intelligent, they will have a transformative impact on the rest of the world. (emph mine)

Comment by rgb on Some potential lessons from Carrick’s Congressional bid · 2022-05-18T06:36:24.427Z · EA · GW

Just wanted to say that I really appreciated this post. As someone who followed the campaign with interest, but not super closely, I found it very informative about the campaign. And it covered all of the key questions I have been vaguely wondering about re: EAs running for office.

Comment by rgb on List of lists of EA-related open philosophy research questions · 2022-05-03T16:41:39.110Z · EA · GW

opinionated (per its title) and non-comprehensive, but "Key questions about artificial sentience: an opinionated introduction" by me:

Comment by rgb on My Job: EA Office Manager · 2022-05-02T18:33:21.270Z · EA · GW

I work at Trajan House and I wanted to comment on this:

But a great office gives people the freedom to not worry about what they need for work, a warm environment in which they feel welcome and more productive, and supports them in ways they did not think were necessary.

By these metrics, Trajan House is a really great office! I'm so grateful for the work that Jonathan and the other operations staff do. It definitely makes me happier and more productive.

Trajan House in 2022 is a thriving hub of work, conversation, and fun.

Comment by rgb on Updates from Leverage Research: history, mistakes and new focus · 2022-04-29T16:42:44.059Z · EA · GW

Leverage just released a working paper, "On Intention Research". From the post:

Starting in 2017, some of Leverage’s psychology researchers stumbled across unusual effects relating to the importance and power of subtle nonverbal communication. Initially, researchers began by attempting to understand and replicate some surprising effects caused by practitioners in traditions like bodywork and energy healing. Over time researchers investigated a wide range of phenomena in subtle nonverbal communication and developed an explanation for these phenomena according to which one’s expectations about what will happen (one’s intentions) in part determine what information is communicated and received nonverbally. This area of research is known as “intention research.”

Those involved in intention research report encountering phenomena that they found quite surprising and challenging to explain. Their findings led many of Leverage’s psychology researchers to conclude that nonverbal communication is at least as expressive and psychologically central as verbal communication. Unfortunately, it also led to some negative psychological and psychosomatic effects and contributed to a significant increase in social tension at Leverage prior to its dissolution in 2019.

This research report describes what intention research was, why researchers pursued it, what they discovered, and the historical antecedents for these discoveries. The piece concludes with a discussion of the risks and challenges associated with further research.

Comment by rgb on Key questions about artificial sentience: an opinionated guide · 2022-04-28T16:06:51.201Z · EA · GW

Thanks for the comment! I agree with the thrust of this comment.

Learning more and thinking more clearly about implementation of computation in general and neural computation in particular, is perennially on my intellectual to-do list list.

We don't want to allow just any arbitrary gerrymandered states to count as an adequate implementation of consciousness's functional roles

maybe the neurons printed on each page aren't doing enough causal work in generating the next edition

I agree with the way you've formulated the problem, and the possible solution - I'm guessing that an adequate theory of implementation deals with both of them. Some condition about there being the right kind of "reliable, counterfactual-supporting connection between the states" (that quote is from Chalmers' take on these issues).

But I have not yet figured out how to think about these things to my satisfaction.

Comment by rgb on How many EAs failed in high risk, high reward projects? · 2022-04-26T14:07:19.045Z · EA · GW

Some past example that come to mind. Kudos to all of the people mentioned for trying ambitious things, and writing up the retrospectives:

  1. Not strictly speaking "EA", but an early effort from folks in the rationality community started an evidence-based medicine organization called MetaMed

Zvi Mowshowitz's post-mortem:

Sarah Constantin's post-mortem:

  1. Michael Plant has a post-mortem of his mental health app, Hippo

  2. Looking at around, I also found this list

Some other posts are the Good Technology Projects' postmortema postmortem of a mental health app by Michael Plant, organisations discuss their learnings in retrospectives like Fish Welfare Initiative or in posts announcing decisions to shut down like Students for High Impact Charities. In the Rationalist community, there was the Arbital Postmortem. You can see more examples on the Forum postmortems and retrospectives tag, and examples from the LessWrong community in their analogous tag.

Comment by rgb on The Bearable Brightness of Wellbeing: The life of an HLI researcher · 2022-04-16T16:21:14.640Z · EA · GW

Thanks for writing this! Your work sounds super interesting. You write, “ But you could be rewarded by the euphoric sense of revelation. Some of that sense may even be authentic; most of it will be fool’s gold.” What are some times you got that euphoric sense in your research for HLI?

Comment by rgb on The pretty hard problem of consciousness · 2021-08-27T12:01:58.486Z · EA · GW

[Replying separately with comments on progress on the pretty hard problem; the hard problem; and the meta-problem of consciousness]

The meta-problem of consciousness is distinct from both a) the hard problem: roughly, the fundamental relationship between the physical and the phenomenal b) the pretty hard problem, roughly, knowing which systems are phenomenally consciousness

The meta-problem is c) explaining "why we think consciousness poses a hard problem, or in other terms, the problem of explaining why we think consciousness is hard to explain" (6)

The meta-problem has a very interesting relationship to the hard problem. To see what this relationship is, we need a distinction between what the "hard problem" of explaining consciousness, and what Chalmers calls the 'easy' problems of explaining "various objective behavioural or cognitive functions such as learning, memory, perceptual integration, and verbal report".

(Much like 'pretty hard', the 'easy' is tongue in cheek - the easy problems are tremendously difficult and thousands of brilliant people with expensive fancy machines are constantly hard at work on them).

Ease of the easy problems: "the easy problems are easy because we have a standard paradigm for explaining them. To explain a function, we just need to find an appropriate neural or computational mechanism that performs that function. We know how to do this at least in principle."

Hardness of the hard problem: "Even after we have explained all the objective functions that we like, there may still remain a further question: why is all this functioning accompanied by conscious experience?...the standard methods in the cognitive sciences have difficulty in gaining purchase on the hard problem."

The meta problem is interesting because it is deeply related to the hard problem, but it is strictly speaking an 'easy' problem: it is about explaining certain cognitive and behavioral functions. For example: thinking "I am currently seeing purple and it seems strange to me that this experience could simply be explained in terms of physics" or "It sure seems like Mary in the black and white room lacks knowledge of what it's like to see red"; or sitting down and writing "boy consciousness sure is puzzling, I bet I can funding to work on this."

Chalmers hopes that cognitive science can make traction on the meta-problem, by explaining how these cognitive functions and behaviors come about in 'topic neutral' terms that don't commit to any particular metaphysical theory of consciousness. And then if we have a solution to the meta problem, this might shed light on the hard problem.

One particular intriguing connection is that it seems like a) a solution to the meta problem should at least be possible and b) if it is, then it gives us a really good reason not to trust our beliefs about consciousness!

  1. A solution to the meta problem is possible, so there is a correct explanation of our beliefs about consciousness that is independent of consciousness.
  2. If there is a correct explanation of our beliefs about consciousness that is independent of consciousness, those beliefs are not justified.
  3. Our beliefs about consciousness are not justified.

Part of the aforementioned growing interest in illusionism is that I think this argument is pretty good. Chalmers came up with it and elaborated it - even though he is not an illusionist - and I like his elaboration of it more than his replies!

Comment by rgb on The pretty hard problem of consciousness · 2021-08-26T09:20:32.780Z · EA · GW

[Replying separately with comments on progress on the pretty hard problem; the hard problem; and the meta-problem of consciousness]

Progress on the hard problem

I am much less sure of how to think about this than about the pretty hard problem. This is in part because in general, I'm pretty confused about how philosophical methodology works, what it can achieve, and the extent to which there is progress in philosophy. This uncertainty is not in spite of, but probably because of doing a PhD in philosophy! I have considerable uncertainty about these background issues.

One claim that I would hang my hat on is that the elaboration of (plausible) philosophical positions in greater detail, and more detailed scrutiny of them, is a kind of progress. And in this regard, I think the last 25 years have seen a lot of progress on the hard problem. The possible solution space has been sketched more clearly, and arguments elaborated. One particularly interesting trend is the elaboration of the more 'extreme' solutions to the hard problem: panpsychism and illusionism. Panpsychism solves the hard problem by making consciousness fundamental and widespread; illusionism dissolves the hard problem by denying the existence of consciousness.

Funnily enough, panpsychists and illusionists actually agree on a lot - they are both skeptical of programs that seek to identify consciousness with some physical, computational, or neural property; they both think that if consciousness exists it then it has some strange-sounding relation to the physical. For illusionists, this (putative) anomalousness of consciousness is part of why they conclude it must not exist. For panpsychists, this (putative) anomalousness of consciousness is part of why they are led to embrace a position that strikes many as radical. You can think of this situation by analogy: theologically conservative religious believers and hardcore atheists are often united in their criticisms of theologically liberal religious believers. Panpsychists and illusionists are both united in their criticisms of 'moderate' solutions to the hard problem.

I think the elaboration of these positions is progress. And this situation also forces non-panpsychist consciousness realists, who reject the 'extremism' of both illusionism and panpsychism, to respond and elaborate their views in a stronger way.

For my part, reading the recent literature on illusionism has made me far more sympathetic to it as a position than I was before. (At first glance, illusionism can just sound like an immediate non-starter. Cartoon sketch of an objection: How could consciousness be an 'illusion' - illusions are mismatches between appearance and reality, and with consciousness the appearance is the reality. Anyway, illusionists can respond to this objection - that's a subject for another day). If I continue to be sympathetic to illusionism, then I can say: the growing elaboration of and appeal of illusionism in the last decade represents progress.

But I think there is at least a 40% chance that my mind will have changed significantly regarding illusionism within the next three months.

Comment by rgb on The pretty hard problem of consciousness · 2021-08-20T14:47:55.451Z · EA · GW

That's a great question. I'll reply separately with my takes on progress on a) the pretty hard problem, b) the hard problem, and c) something called the meta-problem of consciousness [1].

[1] With apologies for introducing yet another 'problem' to distinguish between, when I've already introduced two! (Perhaps you can put these three problems into Anki?)

Progress on the pretty hard problem

This is my attempt to explain Jonathan Birch's recent proposal for studying invertebrate consciousness. Let me know if it makes rough sense!

The problem with studying animal consciousness is that it is hard to know how much we can extrapolate from what we know about what suffices for human consciousness. Let's grant that we know from experiments on humans that you will be conscious of a visual perception if you have a neural system for broadcasting information to multiple sub-systems in the brain. (This is the Global Workspace Theory mentioned above), and that visual perception is broadcast. Great, now we know that this sophisticated human Global Workspace suffices for consciousness. But how much of that is necessary? How much simpler could the Global Workspace be and still result in consciousness?

When we try to take a theory of consciousness "off the shelf" and apply it to animals, we face a choice of how strict to be. We could say that the Global Workspace must be as complicated as the human case. Then no animals count as conscious. We could say that the Global Workspace can be very simple. Then maybe even simple programs count as conscious. To know how strict or liberal to be in applying the theory, we need to know what animals are conscious. Which is the very question!

Some people try to get around this by proposing tests for consciousness that avoid the need for theory--the Turing Test would be an example of this in the AI case. But these usually end up sneaking theory in the backdoor.

Here's Birch's proposal for getting around this impass.

  1. Make a minimal theoretical assumption about consciousness.

The 'facilitation hypothesis':

Phenomenally conscious perception, relative to unconscious perception, facilitates a "cluster" of cognitive abilities.

It's a cluster because it seems like “the abilities will come and go together, co-varying in a way that depends on whether or not a stimulus is consciously perceived” (8). Empirically we have evidence that some abilities in the cluster include: trace conditioning, rapid reversal learning, cross-modal learning.

  1. Look for these clusters of abilities of animals.

  2. See if things which are able to make perceptions unconscious in humans--flashing them quickly and so forth--seems to 'knock out' that cluster in animals. If we can make the clusters come and go like this, it's a pretty reasonable inference that the cause of this is consciousness coming and going.

As I understand it, Birch (a philosopher) is currently working with scientists to flash stuff at bees and so forth. I think Birch's research proposal is a great conceptual advance and I find the empirical research itself very exciting and am curious to see what comes out of it.

Comment by rgb on Writing about my job: Research Fellow, FHI · 2021-08-16T16:21:49.751Z · EA · GW

Oh and I should add: funnily enough, you are on my list of people to reach out to! :D

Comment by rgb on Writing about my job: Research Fellow, FHI · 2021-08-16T16:21:00.026Z · EA · GW

Great question, I'm happy to share.

One thing that makes the reaching out easier in my case is that I do have one specific ask: whether they would be interested in (digitally) visiting the reading group. But I also ask if they'd like to talk with me one-on-one about their work. For this ask, I'll mention a paper of theirs that we have read in the reading group, and how I see it as related to what we are working on. And indicate what broad questions I'm trying to understand better, related to their work.

On the call itself, I am a) trying to get a better understanding of the work and b) let them know what FHI is up to. The very act of preparing for the meeting forces me to understand their work a lot better - I am sure that you have had a similar experience with podcasting! And then the conversations themselves are informative and also enjoyable (for me at least!).

The questions vary according to each person's work. But one question I've asked everyone is:

If you could fund a bunch of work with the aim of making the most progress on consciousness in the next 40 years (especially with an eye to knowing which AIs are conscious), what would you fund? What is most needed for progress?

One last general thought: reaching out to people can be aversive, but in fact it has virtually no downside (as long as you are courteous with your email, of course). The email might get ignored, which is fine. But the best case - and the modal case, I think - is that people are happy that someone is interested in their work.

Comment by rgb on Writing about my job: Research Fellow, FHI · 2021-08-02T12:52:06.998Z · EA · GW

Thanks Darius! It was my pleasure.

Comment by rgb on Writing about my job: Research Fellow, FHI · 2021-07-29T16:52:30.221Z · EA · GW

That's a great point. A related point that I hadn't really clocked until someone pointed it out to me recently, though it's obvious in retrospect, is that (EA aside) in an academic department it is structurally unlikely that you will have a colleague who shares your research interests to a large extent. Since it's rare that a department is big enough to have two people doing the same thing, and departments need coverage of their whole field, for teaching and supervision.

Comment by rgb on Writing about my job: Economics Professor · 2021-07-29T15:14:26.680Z · EA · GW

"I've learned to motivate myself, create mini-deadlines, etc. This is a constant work in progress - I still have entire days where I don't focus on what I should be doing - but I've gotten way better."

What do you think has led to this improvement, aside from just time and practice? Favorite tips / tricks / resources?

Comment by rgb on Retrospective on thinking about my career for a year · 2020-07-22T01:56:42.245Z · EA · GW

Thanks for this. I was curious about "Pick a niche or undervalued area and become the most knowledgeable person in it." Do you feel comfortable saying what the niche was? Or even if not, can you say a bit more about how you went about doing this?

Comment by rgb on Parallels Between AI Safety by Debate and Evidence Law · 2020-07-22T01:50:42.996Z · EA · GW

This is very interesting! I'm excited to see connections drawn between AI safety and the law / philosophy of law. It seems there are a lot of fruitful insights to be had.

You write,

The rules of Evidence have evolved over long experience with high-stakes debates, so their substantive findings on the types of arguments that prove problematic for truth-seeking are relevant to Debate.

Can you elaborate a bit on this?

I don't know anything about the history of these rules about evidence. But why think that over this history, these rules have trended towards truth-seeking per se? I wouldn't be surprised if the rules have evolved to better serve the purposes of the legal system over time, but presumably the relationship between this end and truth-seeking is quite complex. Also, people changing the rules could be mistaken about what sorts of evidence do in fact tend to lead to wrong decisions.

I think all of this is compatible with your claim. But I'd like to hear more!

Comment by rgb on AMA or discuss my 80K podcast episode: Ben Garfinkel, FHI researcher · 2020-07-20T18:44:17.514Z · EA · GW

Thanks for the great summary! A few questions about it

1. You call mesa-optimization "the best current case for AI risk". As Ben noted at the time of the interview, this argument hasn't yet really been fleshed out in detail. And as Rohin subsequently wrote in his opinion of the mesa-optimization paper, "it is not yet clear whether mesa optimizers will actually arise in practice". Do you have thoughts on what exactly the "Argument for AI Risk from Mesa-Optimization" is, and/or a pointer to the places where, in your opinion, that argument has been made (aside from the original paper)?

2. I don't entirely understand the remark about the reference class of ‘new intelligent species’. What species are in that reference class? Many species which we regard as quite intelligent (orangutans, octopuses, New Caledonian crows) aren't risky. Probably, you mean a reference class like "new species as smart as humans" or "new 'generally intelligent' species". But then we have a very small reference class and it's hard to know how strong that prior should be. In any case, how were you thinking of this reference class argument?

3. 'The Boss Baby', starring Alec Baldwin, is available for rental on Amazon Prime Video for $3.99. I suppose this is more of a comment than a question.