Comment by michaelstjules on Announcing the launch of the Happier Lives Institute · 2019-06-21T16:22:40.894Z · score: 7 (3 votes) · EA · GW
Consider John Rawls' grass-counter case: imagine a brilliant Harvard mathematician, fully informed about the options available to her, who develops an overriding desire to count the blades of grass on the lawns. Suppose this person then does spend their time counting blades of grass and is miserable while doing so. On the subjectivist view, this person's life is going well for them. I think this person's life is going poorly for them because they are unhappy.

I think the example might seem absurd because we can't imagine finding satisfaction in counting blades of grass; it seems like a meaningless pursuit. But is it any more meaningful in any objective sense than doing mathematics (in isolation, assuming no one else would ever benefit)? The objectivist might say that this is exactly the point, but the subjectivist could just respond that it doesn't matter as long as the individual is (more) satisfied.

Furthermore, I think life satisfaction and preference satisfaction are slightly different. If we're talking about life satisfaction rather than preference satisfaction, it's not an overriding desire (which sounds like addiction), but, upon reflection, (greater) satisfaction with the choices they make and their preferences for those choices. If we are talking about preference satisfaction, people can also have preferences over their preferences. A drug addict might be compelled to use drugs, but prefer not to be. In this case, does the mathematician prefer to have different preferences? If they don't, then the example might not be so counterintuitive after all. If they do, then the subjectivist can object in a way that's compatible with their subjectivist intuitions.

Also, a standard objection to hedonistic (or more broadly experiential) views is wireheading or the experience machine, of which I'm sure you're aware, but I'd like to point them out to everyone else here. People don't want to sacrifice the pursuits they find meaningful to be put into an artificial state of continuous pleasure, and they certainly don't want that choice to be made for them. Of course, you could wirehead people or put them in experience machines that make their preferences satisfied (by changing these preferences or simulating things that satisfy their preferences), and people will also object to that.

Comment by michaelstjules on Invertebrate Sentience Table · 2019-06-18T00:06:00.743Z · score: 12 (7 votes) · EA · GW

There's some criticism here:

Is the report by Cammaerts and Caemmaerts (2015) positive evidence of self-recognition in ants? Our answer is an emphatic no. Too many crucial methodological details are not given. No formal period between marking the subjects and then exposing them to the mirror was included; the reader is simply asked to accept that no self-cleaning movements occurred before marked ants first saw themselves in the mirror and that marked ants without any mirror did not do so. There is no clear mention of how these data were collected. Were the ants recorded on video? Were they observed directly? In other studies of ant behavior some means of magnification are used, but Caemmerts and Cammaerts provide no information about this, and it is not even clear if any attempt to assess inter-observer reliability was made.
It also remains a possibility that responses to the mirror on the mark test were confounded by chemical cues from the ant’s antennae and chemoreceptors on the mandibles. For instance, if the blue dye was chemically different from the brown dye, chemoreception could explain why ants marked with blue dye were more likely to be attacked by other ants. It is also important to note that the ants must have sensed that they had the marks on themselves through these and other olfactory channels prior to being exposed to the mirror, which would invalidate the mark test.
Notwithstanding the absence of evidence for vision-based individual facial recognition in ants, it would be astonishing if such poorly sighted, small brained insects − especially those without any mirror experience − could immediately use their reflection to try to remove a freshly applied foreign mark that was only visible in the mirror
Comment by michaelstjules on Invertebrate Sentience Table · 2019-06-15T22:20:20.083Z · score: 5 (4 votes) · EA · GW

Some ideas for the presentation of the table to make it more digestible:

1. Is the table downloadable? Can it be made downloadable?

2. Can the table cell/font sizes and table height be made adjustable? It would be nice to be able to fit more of it (ideally all of it) on my screen at once. Just zooming out in my browser doesn't work, since the table shrinks, too, and the same cells are displayed.

3. What about description boxes that pop up when you click on (or hover over) a cell (description/motivation of the feature itself, a box with the footnotes/text/sources when you click on the given cell)? Could also stick to informal recognizable names (cows, ants) where possible and put the taxon in a popup to save on space.

4. Different colour cells for "Likely No", "Lean No", "Unknown", "Lean Yes", "Likely Yes" (e.g. red, pink, grey, light green, green).

Comment by michaelstjules on Invertebrate Sentience Table · 2019-06-15T21:58:18.601Z · score: 2 (2 votes) · EA · GW

Was the mirror test experiment with ants missed or was it intentionally excluded? If the latter, why? It seems the journal it was published in is not very reputable, and the results have not been replicated independently.

https://pdfs.semanticscholar.org/6025/a64f817d6ef770e88449d9c0dea1a7a1c952.pdf

https://en.wikipedia.org/wiki/Mirror_test#Ants

Comment by michaelstjules on Invertebrate Sentience Table · 2019-06-15T21:04:27.848Z · score: 4 (4 votes) · EA · GW

What are the plans for maintaining/expanding this database? Would you consider making a wiki or open source version and allowing contribution from others (possibly through some formal approval process)?

I imagine it could be a useful resource not just for guiding our beliefs about the consciousness of invertebrates, but also the consciousness of other forms of life (and AI in the future).

One suggestion: I think it could be useful to have a column for the age at which each feature is first observable in humans on average (or include these in the entries for humans, as applicable).

Comment by michaelstjules on [Link] The Optimizer's Curse & Wrong-Way Reductions · 2019-04-27T21:38:05.877Z · score: 1 (1 votes) · EA · GW

tl;dr: even using priors, with more options and hazier probabilities, you tend to increase the number of options which are too sensitive to supporting information (or just optimistically biased due to your priors), and these options look disproportionately good. This is still an optimizer’s curse in practice.

This is an issue of the models and priors. If your models and priors are not right... then you should update over your priors and use better models. Of course they can still be wrong... but that's true of all beliefs, all reasoning, etc.
If you assume from the outside (unbeknownst to the agent) that they are all fair, then you're not showing a problem with the agent's reasoning, you're just using relevant information which they lack.

In practice, your models and priors will almost always be wrong, because you lack information; there's some truth of the matter of which you aren't aware. It's unrealistic to expect us to have good guesses for the priors in all cases, especially with little information or precedent as in hazy probabilities, a major point of the OP.

You'd hope that more information would tend to allow you to make better predictions and bring you closer to the truth, but when optimizing, even with correctly specified likelihoods and after updating over priors as you said should be done, the predictions for the selected coin can be more biased in expectation with more information (results of coin flips). On the other hand, the predictions for any fixed coin will not be any more biased in expectation over the new information, and if the prior's EV hadn't matched the true mean, the predictions would tend to be less biased.

More information (flips) per option (coin) would reduce the bias of the selection on average, but, as I showed, more options (coins) would increase it, too, because you get more chances to be unusually lucky.

My prior would not be uniform, it would be 0.5! What else could "unbiased coins" mean?

The intent here again is that you don't know the coins are fair.

Bayesian EV estimation doesn't do hypothesis testing with p-value cutoffs. This is the same problem popping up in a different framework, yes it will require a different solution in that context, but they are separate.

Fair enough.

The proposed solution applies here too, just do (simplistic, informal) posterior EV correction for your (simplistic, informal) estimates.

How would you do this in practice? Specifically, how would you get an idea of the magnitude for the correction you should make?

Maybe you could test your own (or your group's) prediction calibration and bias, but it's not clear how exactly you should incorporate this information, and it's likely these tests won't be very representative when you're considering the kinds of problems with hazy probabilities mentioned in the OP.

Comment by michaelstjules on Why does EA use QALYs instead of experience sampling? · 2019-04-24T03:51:16.229Z · score: 3 (2 votes) · EA · GW

I suspect experience sampling is much more costly and time-consuming to get data on than alternatives, and there's probably much less data. Life satisfaction or other simple survey questions about subjective wellbeing might be good enough proxies, and there's already a lot of available data out there.

Here's a pretty comprehensive post on using subjective wellbeing:

A Happiness Manifesto: Why and How Effective Altruism Should Rethink its Approach to Maximising Human Welfare by Michael Plant

Another good place to read more about this is https://whatworkswellbeing.org/our-work/measuring-evaluating/

Comment by michaelstjules on Reasons to eat meat · 2019-04-24T03:38:01.392Z · score: 10 (5 votes) · EA · GW

Deliberately offsetting a harm through a "similar" opposite benefit means deliberately restricting that donation to a charity from a restricted subset of possible charities, and it may be less effective than the ones you've ruled out.

https://forum.effectivealtruism.org/posts/Yix7BzSQLJ9TYaodG/ethical-offsetting-is-antithetical-to-ea

Offsetting could also justify murder, because there are life-saving charities.

https://slatestarcodex.com/2015/01/04/ethics-offsets/

Also related: https://forum.effectivealtruism.org/posts/eeBwfLfB3iQkpDhz6/at-what-cost-carnivory

Comment by michaelstjules on Reasons to eat meat · 2019-04-24T03:26:31.450Z · score: 11 (8 votes) · EA · GW

I know the post is satirical, but I think it's worth pointing out that ego depletion, the idea that self-control or willpower draws upon a limited pool of mental resources that can be used up, is on shaky ground, i.e. the effect was not replicated in a few meta-analyses, although an older meta-analysis did replicate it.

Comment by michaelstjules on [Link] The Optimizer's Curse & Wrong-Way Reductions · 2019-04-21T00:00:43.866Z · score: 6 (3 votes) · EA · GW

This paper (Schuyler, J. R., & Nieman, T. (2007, January 1). Optimizer's Curse: Removing the Effect of this Bias in Portfolio Planning. Society of Petroleum Engineers. doi:10.2118/107852-MS; earlier version) has some simple recommendations for dealing with the Optimizer's Curse:

The impacts of the OC will be evident for any decisions involving ranking and selection among alternatives and projects. As described in Smith and Winkler, the effects increase when the true values of alternatives are more comparable and when the uncertainty in value estimations is higher. This makes intuitive sense: We expect a higher likelihood of making incorrect decisions when there is little true difference between alternatives and where there is significant uncertainty in our ability to asses value.
(...) Good decision-analysis practice suggests applying additional effort when we face closely competing alternatives with large uncertainty. In these cases, we typically conduct sensitivity analyses and value-of-information assessments to evaluate whether to acquire additional information. Incremental information must provide sufficient additional discrimination between alternatives to justify the cost of acquiring the additional information. New information will typically reduce the uncertainty in our values estimates, with the additional benefit of reducing the magnitude of OC.

The paper's focus is actually on a more concrete Bayesian approach, based on modelling the population from which potential projects are sampled.

Comment by michaelstjules on [Link] The Optimizer's Curse & Wrong-Way Reductions · 2019-04-20T22:32:02.652Z · score: 1 (1 votes) · EA · GW

I made a long top-level comment that I hope will clarify some problems with the solution proposed in the original paper.

I ask the same question I asked of OP: give me some guidance that applies for estimating the impact of maximizing actions that doesn't apply for estimating the impact of randomly selected actions.

This is a good point. Somehow, I think you’d want to adjust your posterior downward based on the set or the number of options under consideration and how unlikely the data that makes the intervention look good. This is not really useful, since I don't know how much you should adjust these. Maybe there's a way to model this explicitly, but it seems like you'd be trying to model your selection process itself before you've defined it, and then you look for a selection process which satisfies some properties.

You might also want to spend more effort looking for arguments and evidence against each option the more options you're considering.

When considering a larger number of options, you could use some randomness in your selection process or spread funding further (although the latter will be vulnerable to the satisficer's curse if you're using cutoffs).

What do you mean by "the priors"?

If I haven’t decided on a prior, and multiple different priors (even an infinite set of them) seem equally reasonable to me.

Comment by michaelstjules on [Link] The Optimizer's Curse & Wrong-Way Reductions · 2019-04-20T22:00:14.348Z · score: 2 (2 votes) · EA · GW

I’m going to try to clarify further why I think the Bayesian solution in the original paper on the Optimizer’s Curse is inadequate.

The Optimizer's Curse is defined by Proposition 1: informally, the expectation of the estimated value of your chosen intervention overestimates the expectation of its true value when you select the intervention with the maximum estimate.

The proposed solution is to instead maximize the posterior expected value of the variable being estimated (conditional on your estimates, the data, etc.), with a prior distribution for this variable, and this is purported to be justified by Proposition 2.

However, Proposition 2 holds no matter which priors and models you use; there are no restrictions at all in its statement (or proof). It doesn’t actually tell you that your posterior distributions will tend to better predict values you will later measure in the real world (e.g. by checking if they fall in your 95% credence intervals), because there need not be any connection between your models or priors and the real world. It only tells you that your maximum posterior EV equals your corresponding prior’s EV (taking both conditional on the data, or neither, although the posterior EV is already conditional on the data).

Something I would still call an “optimizer’s curse” can remain even with this solution when we are concerned with the values of future measurements rather than just the expected values of our posterior distributions based on our subjective priors. I’ll give 4 examples, the first just to illustrate, and the other 3 real-world examples:

1. Suppose you have different fair coins, but you aren’t 100% sure they’re all fair, so you have a prior distribution over the future frequency of heads (it could be symmetric in heads and tails, so the expected value would be for each), and you use the same prior for each coin. You want to choose the coin which has the maximum future frequency of landing heads, based on information about the results of finitely many new coin flips from each coin. If you select the one with the maximum expected posterior, and repeat this trial many times (flip each coin multiple times, select the one with the max posterior EV, and then repeat), you will tend to find the posterior EV of your chosen coin to be greater than , but since the coins are actually fair, your estimate will be too high more than half of the time on average. I would still call this an “optimizer’s curse”, even though it followed the recommendations of the original paper. Of course, in this scenario, it doesn’t matter which coin is chosen.

Now, suppose all the coins are as before except for one which is actually biased towards heads, and you have a prior for it which will give a lower posterior EV conditional on heads and no tails than the other coins would (e.g. you’ve flipped it many times before with particular results to achieve this; or maybe you already know its bias with certainty). You will record the results of coin flips for each coin. With enough coins, and depending on the actual probabilities involved, you could be less likely to select the biased coin (on average, over repeated trials) based on maximum posterior EV than by choosing a coin randomly; you'll do worse than chance.

(Math to demonstrate the possibility of the posteriors working this way for heads out of : you could have a uniform prior on the true future long-run average frequency of heads for the unbiased coins, i.e. for in the interval , then , and , which goes to as goes to infinity. You could have a prior which gives certainty to your biased coin having any true average frequency , so any of the unbiased coins which lands heads out of times will beat it for large enough.)

If you flip each coin times, there’s a number of coins, , so that the true probability (not your modelled probability) of at least one of the other coins getting k heads is strictly greater than , i.e. (for , you need , and for , you need , so grows pretty fast as a function of ). This means, with probability strictly greater than , you won’t select the biased coin, so with probability strictly less than , you will select the biased coin. So, you actually do worse than random choice, because of how many different coins you have and how likely one of them is to get very lucky. You would have even been better off on average ignoring all of the new coin flips and sticking to your priors, if you already suspected the biased coin was better (if you had a prior with mean ).

2. A common practice in machine learning is to select the model with the greatest accuracy on a validation set among multiple candidates. Suppose that the validation and test sets are a random split of a common dataset for each problem. You will find that under repeated trials (not necessarily identical; they could be over different datasets/problems, with different models) that by choosing the model with the greatest validation accuracy, this value will tend to be greater than its accuracy on the test set. If you build enough models each trial, you might find the models you select are actually overfitting to the validation set (memorizing it), sometimes to the point that the models with highest validation accuracy will tend to have worse test accuracy than models with validation accuracy in a lower interval. This depends on the particular dataset and machine learning models being used. Part of this problem is just that we aren’t accounting for the possibility of overfitting in our model of the accuracies, but fixing this on its own wouldn’t solve the extra bias introduced by having more models to choose from.

3. Due to the related satisficer’s curse, when doing multiple hypothesis tests, you should adjust your p-values upward or your p-value cutoffs (false positive rate, significance level threshold) downward in specific ways to better predict replicability. There are corrections for the cutoff that account for the number of tests being performed, a simple one is that if you want a false positive rate of , and you’re doing tests, you could instead use a cutoff of .

4. The satisficer’s curse also guarantees that empirical study publication based on p-value cutoffs will cause published studies to replicate less often than their p-values alone would suggest. I think this is basically the same problem as 3.

Now, if you treat your priors as posteriors that are conditional on a sample of random observations and arguments you’ve been exposed to or thought of yourself, you’d similarly find a bias towards interventions with “lucky” observations and arguments. For the intervention you do select compared to an intervention chosen at random, you’re more likely to have been convinced by poor arguments that support it and less likely to have seen good arguments against it, regardless of the intervention’s actual merits, and this bias increases the more interventions you consider. The solution supported by Proposition 2 doesn’t correct for the number of interventions under consideration.

Comment by michaelstjules on [Link] The Optimizer's Curse & Wrong-Way Reductions · 2019-04-16T06:42:19.149Z · score: 3 (2 votes) · EA · GW
You seem to be using "people all agree" as a stand-in for "the optimizer's curse has been addressed". I don't get this. Addressing the optimizer's curse has been mathematically demonstrated. Different people can disagree about the specific inputs, so people will disagree, but that doesn't mean they haven't addressed the optimizer's curse.

Maybe we're thinking about the optimizer's curse in different ways.

The proposed solution of using priors just pushes the problem to selecting good priors. It's also only a solution in the sense that it reduces the likelihood of mistakes happening (discovered in hindsight, and under the assumption of good priors), but not provably to its minimum, since it does not eliminate the impacts of noise. (I don't think there's any complete solution to the optimizer's curse, since, as long as estimates are at least somewhat sensitive to noise, "lucky" estimates will tend to be favoured, and you can't tell in principle between "lucky" and "better" interventions.)

If you're presented with multiple priors, and they all seem similarly reasonable to you, but depending on which ones you choose, different actions will be favoured, how would you choose how to act? It's not just a matter of different people disagreeing on priors, it's also a matter of committing to particular priors in the first place.

If one action is preferred with almost all of the priors (perhaps rare in practice), isn't that a reason (perhaps insufficient) to prefer it? To me, using this could be an improvement over just using priors, because I suspect it will further reduce the impacts of noise, and if it is an improvement, then just using priors never fully solved the problem in practice in the first place.

I agree with the rest of your comment. I think something like that would be useful.

Comment by michaelstjules on [Link] The Optimizer's Curse & Wrong-Way Reductions · 2019-04-14T23:09:51.766Z · score: 3 (4 votes) · EA · GW
What do you mean by "a good position"?
(...)
I'm getting a little confused about what sorts of concrete conclusions we are supposed to take away from here.

I'm not saying we shouldn't use priors or that they'll never help. What I am saying is that they don't address the optimizer's curse just by including them, and I suspect they won't help at all on their own in some cases.

Maybe checking sensitivity to priors and further promoting interventions whose value depends less on them (among some set of "reasonable" priors) would help. You could see this as a special case of Chris's suggestion to "Entertain multiple models".

Perhaps you could even use an explicit model to combine the estimates or posteriors from multiple models into a single one in a way that either penalizes sensitivity to priors or gives less weight to more extreme estimates, but a simpler decision rule might be more transparent or otherwise preferable. From my understanding, GiveWell already uses medians of its analysts' estimates this way.

Ah, I guess we'll have to switch to a system of epistemology which doesn't bottom out in unproven assumptions. Hey hold on a minute, there is none.

I get your point, but the snark isn't helpful.

Comment by michaelstjules on [Link] The Optimizer's Curse & Wrong-Way Reductions · 2019-04-14T17:57:45.144Z · score: 3 (4 votes) · EA · GW
Yes, but it's very hard to attack any particular prior as well.

I don't think this leaves you in a good position if your estimates and rankings are very sensitive to the choice of "reasonable" priors. Chris illustrated this in his post at the end of part 2 (with the atheist example), and in part 3.

You could try to choose some compromise between these priors, but there are multiple "reasonable" ways to compromise. You could introduce a prior on these priors, but you could run into the same problem with multiple "reasonable" choices for this new prior.

Comment by michaelstjules on Existential risk as common cause · 2019-02-24T19:54:01.114Z · score: 2 (2 votes) · EA · GW

I think even more people have things in the bads set, and there will be more agreement on these values, too, e.g. suffering, cruelty and injustice. The question is then a matter of weight.

Most people (and probably most EAs) aren't antinatalists, so you would expect, for them, the total good to outweigh the total bad. Or, they haven't actually thought about it enough.

Comment by michaelstjules on Cause profile: mental health · 2018-12-31T20:08:56.093Z · score: 3 (3 votes) · EA · GW

OTOH, while current mental health issues may prevent altruism, prior experiences of suffering may lead to increased empathy and compassion.

https://reducing-suffering.org/how-important-is-experiencing-suffering-for-caring-about-suffering/

Comment by michaelstjules on What’s the Use In Physics? · 2018-12-30T22:40:11.719Z · score: 4 (4 votes) · EA · GW

A few more: energy (nuclear fusion, green tech, energy storage), medical physics, quantum computing (and its medical applications), risks from space and preparedness for worst case scenarios (like ALLFED).

Comment by michaelstjules on How High Contraceptive Use Can Help Animals? · 2018-12-30T18:31:07.557Z · score: 1 (1 votes) · EA · GW
By preventing one pregnancy in Vietnam, we save approximately: 30 mammals 850 chickens 1395 fish from being produced in factory-farmed conditions (or 35 626 welfare points).

Is this only from the animal products the child would have eaten themself? Should the consumption from that child's descendants be included?

None of the GiveWell/ACE top or standout charities are working in these areas.

FWIW, TLYCS recommends PSI and DMI, and DMI is one of GiveWell's standout charities, and both do family planning work.

Comment by michaelstjules on How High Contraceptive Use Can Help Animals? · 2018-12-30T10:10:03.112Z · score: 2 (2 votes) · EA · GW

FWIW, this is aimed at developing countries.

Couldn't you say the same about GiveWell's evaluation of AMF, TLYCS's evaluation of PSI or the evaluation of any other charity or intervention that would predictably affect population sizes? ACE doesn't consider impacts on wild animals for most of the charities/interventions it looks into, either, despite the effects of agriculture on wild animals.

My impression is that Charity Science/Entrepreneurship prioritizes global health/poverty and animal welfare, so we shouldn't expect them to consider the effects on technological advancement or GCRs anymore than we should expect GiveWell, TLYCS or ACE to.

They have worked on evaluating animal welfare, though, so it would be nice to see this work applied here for wild animals.

EDIT: Oh, is the concern that they're looking at a more biased subset of possible effects (by focusing primarily on effects that seem positive)?

Comment by michaelstjules on Detecting Morally Significant Pain in Nonhumans: Some Philosophical Difficulties · 2018-12-29T01:07:15.354Z · score: 9 (3 votes) · EA · GW

For the Rethink Priorities project, why not also look into consciousness in plant species (e.g. mimosa and some carnivorous plants), AI (especially reinforcement learning) and animal/brain simulations (e.g. OpenWorm)? Whether or not they're conscious (or conscious in a way that's morally significant), they can at least provide some more data to adjust our credences in the consciousness of different animal species; they can still be useful for comparisons.

I understand that there will be little research to use here, but I expect this to mean proportionately less time will be spent on them.

Comment by michaelstjules on The harm of preventing extinction · 2018-12-26T06:19:06.450Z · score: 5 (3 votes) · EA · GW
My rough answer to this is: If someone wants to die (after thinking about it for a long time and having time to reflect on it), let them die.

Some people don't have the choice to die, because they're prevented from it, like victims of abuse/torture or certain freak accidents.

I don't see how the atrocities that are experienced by humans outweigh the benefits, given that the vast majority of humans seem to have a pretty decent will to live.

I think this is a problem with the idea of "outweigh". Utilitarian interpersonal tradeoffs can be extremely cruel and unfair. If you think the happiness can aggregate to outweigh the worst instances of suffering:

1. How many additional happy people would need to be born to justify subjecting a child to a lifetime of abuse and torture?

2. How many extra years of happy life for yourself would you need to justify subjecting a child to a lifetime of abuse and torture?

The framings might invoke very different immediate reactions (2 seems much more accusatory because the person benefitting from another's abuse and torture is the one making the decision to subject them to it), but for someone just aggregating by summation, like a classical utilitarian, they're basically the same.

I think it's put pretty well here, too:

There’s ongoing sickening cruelty: violent child pornography, chickens are boiled alive, and so on. We should help these victims and prevent such suffering, rather than focus on ensuring that many individuals come into existence in the future. When spending resources on increasing the number of beings instead of preventing extreme suffering, one is essentially saying to the victims: “I could have helped you, but I didn’t, because I think it’s more important that individuals are brought into existence. Sorry.”
Comment by michaelstjules on The expected value of extinction risk reduction is positive · 2018-12-23T20:54:48.327Z · score: 1 (1 votes) · EA · GW

Isn't it equally justified to assume that their welfare in the conditions they were originally optimized/designed for is 0 in expectation? If anything, it makes more sense to me to make assumptions about this setting first, since it's easier to understand their motivations and experiences in this setting based on their value for the optimization process.

Apart from that, I am not sure if the two assumptions listed as bullet points above will actually hold for the majority of "sentient tools".

We can ignore any set of tools that has zero total wellbeing in expectation; what's left could still dominate the expected value of the future. We can look at sets of sentient tools that we might think could be biased towards positive or negative average welfare:

1. the set of sentient tools used in harsher conditions,

2. the set used in better conditions,

3. the set optimized for pleasure, and

4. the set optimized for pain.

Of course, there are many other sets of interest, and they aren't all mutually exclusive.

The expected value of the future could be extremely sensitive to beliefs about these sets (their sizes and average welfares). (And this could be a reason to prioritize moral circle expansion instead.)

Comment by michaelstjules on The expected value of extinction risk reduction is positive · 2018-12-18T21:05:13.242Z · score: 2 (2 votes) · EA · GW
Assuming that future agents are mostly indifferent towards the welfare of their “tools”, their actions would affect powerless beings only via (in expectation random) side-effects. It is thus relevant to know the “default” level of welfare of powerless beings.

By "in expectation random", do you mean 0 in expectation? I think there are reasons to expect the effect to be negative (individually), based on our treatment of nonhuman animals. Our indifference to chicken welfare has led to severe deprivation in confinement, more cannibalism in open but densely packed systems, the spread of diseases, artificial selection causing chronic pain and other health issues, and live boiling. I expect chickens' wild counterparts (red jungle fowls) to have greater expected utility, individually, and plausibly positive EU (from a classical hedonistic perspective, although I'm not sure either way). Optimization for productivity seems usually to come at the cost of individual welfare.

Even for digital sentience, if designed with the capacity to suffer -- regardless of our intentions and their "default" level of welfare, and especially if we mistakenly believe them not to be sentient -- we might expect their levels of welfare to decrease as we demand more from them, since there's not enough instrumental value for us to recalibrate their affective responses or redesign them with higher welfare. The conditions in which they are used may become significantly harsher than the conditions for which they were initially designed.

It's also very plausible that many of our digital sentiences will be designed through evolutionary/genetic algorithms or other search algorithms that optimize for some performance ("fitness") metric, and because of how expensive these approaches are computationally, we may be likely to reuse the digitial sentiences with only minor adjustments outside of the environments for which they were optimized. This is already being done for deep neural networks now.

Similarly, we might expect more human suffering (individually) from AGI with goals orthogonal to our welfare, an argument against positive expected human welfare.

Comment by michaelstjules on Existential risk as common cause · 2018-12-09T19:25:07.687Z · score: 6 (6 votes) · EA · GW

You can get similar value-independence in favour of extinction by using "bads" instead of "goods". Many of the values in Oesterheld's list have opposites which could reasonably be interpreted as "bads", and some of them are already "bads", e.g. suffering, pain and racism.

Comment by michaelstjules on Existential risk as common cause · 2018-12-09T19:04:01.825Z · score: 7 (4 votes) · EA · GW

Besides the person-affecting views and disvalue of life covered here, if an individual has an Epicurean view of life and death (another kind of person-affecting view), i.e. death is not bad, then improving wellbeing should probably take priority. And while Epicureanism assigns 0 disvalue to death (ignoring effects on others), one could assign values arbitrarily close to 0.

There are also issues with dealing with infinities that make utilitarianism non-action guiding (it doesn't tell us what to do in most practical cases); you could probably throw these in with nihilism. E.g. if the universe is unbounded ("infinite") in space or time, then we can't change the total sum of utility, and that number is not even well-defined (not even +infinity or -infinity) with the usual definitions of convergence in the real numbers. If you assign any nonzero probability to an infinite universe, you end up with the same problem, but it's actually pretty likely that the universe is spatially unbounded. There are several attempts at solutions, but all of them have pretty major flaws, AFAIK.

Some person-affecting views can help, i.e. using a Pareto principle, but then it's not clear how to deal with individuals whose exact identities depend on your decisions (or maybe we just ignore them; many won't like that solution), and there are still many cases that can't be handled. There's discussion in this podcast, with some links for more reading (ctrl-F "Pareto" after expanding the transcript): https://80000hours.org/podcast/episodes/amanda-askell-moral-empathy/

Rounding sufficiently small probabilities to 0 and considering only parts of the universe we're extremely confident we can affect can help, too. This proposed solution and a few others are discussed here: https://nickbostrom.com/ethics/infinite.pdf

You could also have a bounded vNM utility function, but this means assigning decreasing marginal value to saving lives, and how you divide decisions/events matters, e.g. "saving 1 life and then saving 1 life" > "saving 2 lives and then saving 0 lives".

For the unbounded time case (assuming we can handle or avoid issues with unbounded space, and people might prefer not to treat time and space differently): https://forum.effectivealtruism.org/posts/9D6zKRPfaALiBhnnN/problems-and-solutions-in-infinite-ethics

Comment by michaelstjules on A Happiness Manifesto: Why and How Effective Altruism Should Rethink its Approach to Maximising Human Welfare · 2018-11-04T06:05:08.670Z · score: 1 (1 votes) · EA · GW

Regarding interpersonal comparisons,

So long as these differences are randomly distributed, they will wash out as ‘noise’ across large numbers of people

I think this is a crucial assumption that may not hold when comparing groups, i.e. there could be group differences (which could involve the same people, but before and after some event) in interpretations of the scales, due to differences in experiences between the two groups, e.g. a disability.

That immigrants to Canada seem to use the scales similarly to Canadians doesn't mean they weren't using the scales differently before they came to Canada. I think we actually discussed scale issues with life satisfaction on Facebook before (prompted by you?), and differences after adjusting for item responses seem to suggest different interpretations of the scale (or the items), or different relationships between the items. Two examples (cited in one of the papers in your reading list, https://www.researchgate.net/publication/230603396_Theory_and_Validity_of_Life_Satisfaction_Scales):

https://www.researchgate.net/profile/Robert_Biswas-Diener/publication/225502271_The_Divergent_Meanings_of_Life_Satisfaction_Item_Response_Modeling_of_the_Satisfaction_with_Life_Scale_in_Greenland_and_Norway/links/570184a808aea6b7746a7df9.pdf?origin=publication_list

http://www.people.virginia.edu/~so5x/IRT%20JRP%20revision%202.pdf

But there's an obvious response here: we should use IRT to adjust for different scale interpretations.

Comment by michaelstjules on A Happiness Manifesto: Why and How Effective Altruism Should Rethink its Approach to Maximising Human Welfare · 2018-11-04T04:14:34.947Z · score: 1 (1 votes) · EA · GW

I think that the evidence you present in section 4, e.g. that people interpret scales as equal-interval and that immigrants have similar SWB, could be a good response to this paper, though, because it suggests that we can interpret the discrete life satisfaction scale as cardinal and just aggregate it instead.

Comment by michaelstjules on A Happiness Manifesto: Why and How Effective Altruism Should Rethink its Approach to Maximising Human Welfare · 2018-11-01T04:18:13.480Z · score: 1 (1 votes) · EA · GW

The theoretical results don't depend on the scale being 3-point. Their argument deals directly with the assumed underlying normal distributions and transforms them into log-normal distributions with the order of the expected values reversed, so it doesn't matter how you've estimated the parameters of the normal distributions or if you've even done it at all.

In the case of life satisfaction scales, is there any empirical evidence we could use to decide the form of the underlying continuous distribution?

They do suggest that you could "use objective measures to calibrate cardinalizations of happiness", e.g. with incidence of mental illness, or frequencies of moods, as the authors have done something similar here https://www.nber.org/papers/w19243.

Comment by michaelstjules on EA Hotel with free accommodation and board for two years · 2018-06-22T02:57:15.747Z · score: 3 (3 votes) · EA · GW

V*ganism shows very high 'recidivism' rates in the general population. Most people who try to stop eating meat/animal products usually end up returning to eat these things before long.

FWIW, based on Faunalytics surveys, the recidivism rate seems to be about 50% for vegans motivated by animal protection specifically:

https://www.facebook.com/groups/EffectiveAnimalAdvocacy/permalink/906516599519212/

Comment by michaelstjules on EA Hotel with free accommodation and board for two years · 2018-06-22T02:34:19.145Z · score: 3 (3 votes) · EA · GW

There are protected characteristics, like race and gender, and the only way I can see EA/non-EA being covered is through beliefs. This first link only says religion specifically, but the second includes philosophical beliefs more generally:

https://www.gov.uk/discrimination-your-rights

http://www.legislation.gov.uk/ukpga/2010/15/section/10

More here: http://www.legislation.gov.uk/ukpga/2010/15/contents

I would guess that nonprofits that only serve people of a certain protected characteristics can also be legal, e.g. women's shelters. Maybe it could fall under Services and public functions, Premises or Associations:

http://www.legislation.gov.uk/ukpga/2010/15/part/3

http://www.legislation.gov.uk/ukpga/2010/15/part/4

http://www.legislation.gov.uk/ukpga/2010/15/part/7

Comment by michaelstjules on How to improve EA Funds · 2018-04-05T03:34:00.243Z · score: 2 (2 votes) · EA · GW

allocate funds to the top charities in their cause area, and donate to those charities on a regular basis until the fund manager comes along and updates the allocation

Because of discount rates, wouldn't it then be better to do all of the disbursements between updates right after the update, instead of dragging them out?

Or, since people will continue donating between disbursements, disburse as funding becomes available, and save a chunk (everything after a certain date) for the next update because it will be better allocated.

Comment by michaelstjules on On funding medical research · 2018-02-26T02:39:17.152Z · score: 1 (1 votes) · EA · GW

I'm not sure I'd put it only on X-risk people. My understanding is that disease burden and DALYs are calculated using as a reference the highest life expectancy of any country by gender, which was previously Japanese women (now South Korean women?), and somewhere between 80 and 90 years. This means that deaths after this reference life expectancy simply don't count towards disease burden at all. I'd like to hypothesize that this and some of the downvotes may be due to what I suspect is a common intuition (perhaps not common in EA; I don't know): everyone ought to have an overall good life with a decent lifespan, i.e. "fair innings".

This "fair innings" might be part of why EAs are generally more concerned with global health and poverty than anti-aging. Maybe the stronger evidence for specific poverty/health interventions explains this better, though.

Mostly guesses on my part, of course.

Comment by michaelstjules on How to get a new cause into EA · 2018-01-11T17:57:23.501Z · score: 2 (2 votes) · EA · GW

One thing that's very useful about having separate cause areas is that it helps people decide what to study and research in depth, e.g. get a PhD in. This probably doesn't need to be illustrated, but I'll do it anyway:

If you consider two fields of study, A and B, such that A has only one promising intervention, and B has two, and all three interventions are roughly equal in expectation (or whatever other measures are important to you); then it would be better to study B, because if one of its two interventions don't pan out, you can more easily switch to the other; with A, you might have to move onto a new field entirely. Studying B actually has higher expected value than studying A, despite all three interventions being equal in expectation.

Comment by michaelstjules on Mental Health Shallow Review · 2017-11-22T01:39:50.399Z · score: 2 (2 votes) · EA · GW

Relevant recent systematic review:

Quality Assessment of Economic Evaluations of Suicide and Self-Harm Interventions: A Systematic Review.

http://psycnet.apa.org/record/2017-41357-001

PDF: http://psycnet.apa.org/fulltext/2017-41357-001.pdf

Comment by michaelstjules on Does Effective Altruism Lead to the Altruistic Repugnant Conclusion? · 2017-08-03T04:40:00.478Z · score: 1 (3 votes) · EA · GW

Thinking from the perspective of a beneficiary, I would rather get $100 than remove a 1/10,000,000 risk of death.

Would you also volunteer to be killed so that 10,000,000 people just like you could have $100 that they could only spend to counterfactually benefit themselves?

I think the probability here matters beyond just its effect on the expected utility, contrary, of course, to EU maximization. I'd take $100 at the cost of an additional 1/10,000,000 risk of eternal torture (or any outcome that is finitely but arbitrarily bad). On the other hand, consider the 5 following worlds:

A. Status quo with 10,000,000 people with finite lives and utilities. This world has finite utility.

B. 9,999,999 people get an extra $100 compared to world A, and the other person is tortured for eternity. This world definitely has a total utility of negative infinity.

C. The 10,000,000 people each decide to take $100 for an independent 1/10,000,000 risk of eternal torture. This world, with probability ~ 1-1/e ~ 0.63 (i.e. "probably") has a total utility of negative infinity.

D. The 10,000,000 people together decide to take $100 for a 1/10,000,000 risk that they all are tortured for eternity (i.e. none of them are tortured, or all of them are tortured together). This world, with probability 9,999,999/10,000,000 has finite utility.

E. Only one out of the 10,000,000 people decides to take $100 for a 1/0,000,000 risk of eternal torture. This world, with probability 9,999,999/10,000,000 has finite utility.

I would say D >> E > A >>>> C >> B, despite the fact that in expected total utility, A >>>> B=C=D=E. If I were convinced this world will be reproduced infinitely many times (or e.g. 10,000,000 times) independently, I'd choose A, consistently with expected utility.

So, when I take $100 for a 1/10,000,000 risk of death, it's not because I'm maximizing expected utility; it's because I don't care about any 1/10,000,000 risk. I'm only going to live once, so I'd have to take that trade (or similar such trades) hundreds of times for it to even start to matter to me. However, I also (probably) wouldn't commit to taking this trade a million times (or a single equivalent trade, with $100,000,000 for a ~0.1 probability of eternal torture; you can adjust the cash for diminishing marginal returns). Similarly, if hundreds of people took the trade (with independent risk), I'd start to be worried, and I'd (probably) want to prevent a million people from doing it.

Comment by michaelstjules on The Philanthropist’s Paradox · 2017-07-03T21:21:04.725Z · score: 1 (1 votes) · EA · GW

In this scenario you could (mathematically) save your wealth for an infinite number of periods and then donate it, generating infinite utility.

How is there anything (i.e. "and then") after an infinite amount of periods (taking altogether an infinite amount of time)? Are you introducing hyperreals or nonstandard analysis? Are you claiming this is just a possibility (from our ignorance about the nature of time) or a fact, conditional on the universe lasting forever?

I think it's extremely unlikely that time works this way, but if you're an EU maximizer and assign some positive probability to this possibility, then, sure, you can get an infinite return in EU. Most likely you'll get nothing. It's a lot like Pascal's wager.

Comment by michaelstjules on The Philanthropist’s Paradox · 2017-06-27T03:56:29.437Z · score: 1 (1 votes) · EA · GW

In recent discussion Patrick Kaczmarek informs me I’m absolutely mistaken to think it can problem with decision theory and helpfully suggested the issue might be the bridging principle between one’s axiology and one’s decisions theory.

The problem seems essentially the same as Parfit's Hitchhiker: you must pre-commit to win, but you know that when the time comes to pay/spend, you'll want to change your mind.

Comment by michaelstjules on The Philanthropist’s Paradox · 2017-06-27T03:40:15.368Z · score: 1 (1 votes) · EA · GW

We can make necessitarianism asymmetric: only people who will necessarily exist OR would have negative utility (or less than the average/median utility, etc.) count.

Some prioritarian views, which also introduce some kind of asymmetry between good and bad, might also work.