the quantity and quality of output is underwhelming given the amount of money and staff time invested.

Of Redwood’s published research, we were impressed by Redwood's interpretability in the wild paper, but would consider it to be no more impressive than progress measures for grokking via mechanistic interpretability, executed primarily by two independent researchers, or latent knowledge in language models without supervision, performed by two PhD students.^[4] These examples are cherry-picked to be amongst the best of academia and independent research, but we believe this is a valid comparison because we also picked what we consider the best of Redwood's research and Redwood's funding is very high relative to other labs.

I'm missing a lot of context here, but my impression is that this argument doesn't go through, or at least is missing some steps:

1. We think that the best Redwood research is of similar quality to work by [Neel Nanda, Tom Lieberum and others, mentored by Jacob Steinhardt]
2. Work by those others doesn't cost $20M
3. Therefore the work by Redwood shouldn't cost $20M

Instead, the argument which would go through would be:

1. Open Philanthropy spent $20M on Redwood Research
2. That $20M produced [such and such research]
3. This is how you could have spent $20M to produce [better research]
4. Therefore, Open Philanthropy shouldn't have spent $20M on Redwood Research, but instead on [alternatives]
   1. (or spent $20M on [alternatives] and on Redwood Research, if the value of Redwood Research is still above the bar)

But you haven't shown step 3, the tradeoff against the counterfactual. It seems likely that the situation is such that producing good AI safety research depends on somewhat idiosyncratic non-monetary factors. Sometimes you will find a talented independent researcher or a PhD student that will produce quality research for relatively small amounts of money, sometimes you will spend $20M to get an outcome of a similar quality. I could see that being the case if the bottleneck isn't money, which seems plausible.

Also note that building an institution is potentially much more scalable than funding one-off independent researchers.

As I said, I'm missing lots of context (i.e., I haven't read Redwood's research, seems within the normal range of possibility that it wouldn't be worth $20M), but I thought I'd give my two cents.

Answered here.

Thanks Vasco, these are great. Though, where are you getting the depression baseline from?

Thanks Nick.

"Neglected diseases" (as defined by the WHO)

Yeah, I was thinking more like rare genetic diseases. Edited to say rare rather than neglected

Thanks!

Neat post, and nice to see squiggle in the wild.

Some points

Suppose I have several point-estimates for the fuel efficiency of my car - I can easily take a weighted average of these to make an aggregate point estimate, but it’s not clear how I could turn them into an interval estimate without a heavy dose of personal bias.

You could create a mixture distribution, you could fit a lognormal whose x% confidence interval is the range expressed by the points you've already found, you could use your subjective judgment to come up with a distribution which could fit it, you could use kernel density estimation (https://en.wikipedia.org/wiki/Kernel_density_estimation). 

In your number of habitable planets estimate, you have a planetsPerHabitablePlanet estimate. This is an interesting decomposition. I would have looked at the fraction of planets which are habitable, and probably fit a beta distribution to it, given that we know that the fraction is between 0 and 1. This seems a bit like a matter of personal taste, though.

Yeah, you also see this with criticism, where for any given piece of criticism, you could put more effort into it and make it more effective. But having that as a standard (even as a personal one) means that it will happen less.

So I don't think we disagree on the fact that there is a demand curve? Maybe we disagree that I want to have more sapphires and less politeness, on the margin?

Oh nice. Maybe there could also be a setting for "do not show pinned post if you've already read it".

Also, you can turn off the sidebar and the intercom with something like

@-moz-document domain("forum.effectivealtruism.org") {
    .SingleColumnSection-root {
        /* To do: tweak for current redesign */
	  /* width: 1000px; */
        /*  margin-left: 60.5px;  */
        /* max-width: 1200px    */
    }
    .NavigationStandalone-sidebar {
        display: none;
    }
    .intercom-lightweight-app{
        display: none;
    }
}

in an extension like stylus.

I continue to dislike how much space pinned and highlighted posts occupy:

Compare for instance with hackernews:

In my particular case, I prefer to use an rss reader with a more dense presentation, in this case newsboat. Other users who prefer packed information might want to explore similar setups:

Also, personally I found it a shame that the community posts were removed from the general rss feed. Is there an rss feed for the community posts specifically?

Yeah, I agree they seem acceptable/good on an absolute level (though as mentioned I think that much better interventions exist).

This was great, thanks

Hey, thanks for writting this. You might want to check out the work by Just Impact, and the estimations by Open Philanthropy when they were working on this area.

I strongly, strongly, strongly disagree with this decision. 

Per my own values and style of communication, I think that welcoming people like sapphire or Sabs who a) are or can be intensely disagreeable, and b) have points worth sharing and processing, is strongly on the side of worth doing, even if c) they make other people uncomfortable, and d) even if they occasionally misfire, and even if they are wrong most of the time, as long as the expected value of the stuff they say remains high. 

In particular, I think that doing so is good for arriving at correct beliefs and for becoming stronger, which I value a whole lot. It is the kind of communication which we use on my forecasting group, where the goal is to arrive at correct beliefs.

I understand that the EA Forum moderators may have different values, and that they may want to make the forum a less spiky place. Know that this has the predictable consequence of losing a Nuño, and it is part of the reason why I've bothered to create a blog and added comments to it in a way which I expect to be fairly uncensorable^[1].

Separately, I do think it is the case that EA "simps" for tech billionaires^[2]. An answer I would have preferred to see would be a steelmanning of why that is good, or an argument of why this isn't the case.

1. ^{^}

   Uncensorable by others: I am hosting the blog on top of nja.la and the comments on my own servers. Not uncensorable by me; I can and will censor stuff that I think is low value by my own utilitarian/consequentialist lights.

2. ^{^}

   Less sure of AI companies, but you could also make the case, e.g., 80kh does recommend positions at OpenAI (<https://jobs.80000hours.org/?query=OpenAI>) 

Here is a squiggle model which addresses some of these concerns

p_student_donates = beta(2.355081, 49.726) // 0.01 to 0.1, i.e., 1% to 10%, from <https://nunosempere.com/blog/2023/03/15/fit-beta/>
amount_student_donates = 5 to 20 // dollars

p_parent_donates = beta(3.287, 17.7577) // 0.05 to 0.3, i.e., 5% to 30%
amount_parent_donates = 50 to 500 // dollars

num_parents = 5 to 20
num_students = 15 to 90

expected_impact = num_students * p_student_donates * amount_student_donates + num_parents * p_parent_donates * amount_parent_donates

To execute it, you can paste it here: <https://www.squiggle-language.com/playground>. 

Turns out that unpkg, an online service that Observable was relying on, was down yesterday (https://twitterc.om/unpkg/status/1636149026830843904), but this is fixed now. Sam Nolan thinks that it may have been cached by browsers, which is why it would show up broken for me but not for others, fixed now either way, though ¯\_(ツ)_/¯

I've been coming back to this model, because I've been doing some thinking on worldview diversification, and having a model like this is great for comparisons.

Two points:

1. The model is broken. Maybe worth pinning the observable imports to specific versions?
2. Thinking about this more, the counterfactual adjustments don't necessarily have to be a multiplication. i.e., it may not be that instead of dist, you have dist * 0.8 to 0.95, but rather you have mixture([dist, 0], [p, 1-p]). That is, when considering a counterfactual adjustment, it's not (only) that you have less impact because if you exist some worse option would replace you, it's that you have a chance at having no impact
   1. For GW's current estimates this doesn't matter, but I feel that for distributions, capturing the true shape of the uncertainty does matter.

it takes me several minutes with pen, paper and calculator to estimate the mean of the underlying lognormal distribution

You can arrive at a lognormal from the 95% c.i. in Squiggle, e.g., by writting 50 to 5000, and it will give you the mean automatically. You could also have a mixture of distributions, e.g., mx([dist_approach_1, dist_approach_2], [0.5, 0.5]). 

My preferred approach would be to look at the distributions produced by the two approaches, and then write a lognormal given what I think are plausible lower and upper bounds.

is there an easy way to turn discrete estimates into a aggregate estimate

Use processes which produce distributional estimates from the beginning :)

When is a point-estimate approach much worse than a distribution approach? For example, what is it about the Fermi paradox that leads to such differing results between the methods?

The general answer, for me, is that the true shape of the uncertainty is distributional, and so approaches which don't capture the true shape of the uncertainty will produce worse answers. 

The specific answer is that e.g., "probability of life per planet" doesn't capture our uncertainty about the number of planets with life, i.e., we aren't certain that there are exactly X planets, for a probability of X/num planets. Same applies to other factors. Answering  this quickly, lmk if it makes sense.

At the risk of repeating myself, I think that there is an important point here where most potential bet offers are going to be people ambiguously using the word "I'd bet" in a somewhat metaphorical world. And in that state of the world, I'm not going to go around trying to accept other people's potential bet offers, because most of the time people will chicken out. This makes the signal of using "I'd bet" much weaker, meaning that there aren't any downsides to using it metaphorically, which leads to a downward spiral where it loses the signal.

The alternative would be to have an unambiguous way of saying that you are willing to put money behind your belief. Having a norm of using "I'd bet" non-metaphorically would be one such way, but I'm open to others.

Nitpick, it is "Grilo", not "Grillo", but I like the Spanish version too!

Terribly sorry, fixed now.

I tend to agree. I also wonder which strategy does 80,000 Hours uses to come up with their list of top-recommended career paths. I understand factors like importance, tractability and neglectedness are part of it, but I wonder what is the specific procedure to come up with the rankings (e.g. maybe a weighted factor model).

I am not really sure!

Did you mean "underestimated" (instead of "overestimated")?

Yep, thanks, fixed.

Note also that the forum already has a section on Softer discussion norms and tips.

There are various levels of "policing", ranging from gentle suggestions (this post), occasional nudging, public shaming, coerced speech through some technical tweak on the EA forum, public flogging, or locking someone away and throwing away the key.

I think the lower levels are fine. The forum already has rules, some of them could refer to epistemics. "Be kind, stay on topic, be honest, offer an operationalization and stakes if you are offering a bet"

Well, you're right that signaling intelligence, creativity, wisdom, and moral virtues is sexually and romantically attractive

Geoffrey Miller fighting the good fight, upvoted.

I think it's ok if you would actually end up betting.

I agree that you are pointing out a downside, but I think that the tradeoff is small, and the benefits of increased betting larger.

Thanks Joel, really appreciate this comment

Why follow that policy, rather than "only make trades if their expected value is greater than the value of what you would otherwise have used your time with?"

"Trigger action pattern", a technique for adopting habits proposed by CFAR <https://www.lesswrong.com/posts/wJutA2czyFg6HbYoW/what-are-trigger-action-plans-taps>.

I might well be missing something, but better quality of care and more "convenient" treatment (meaning people get earlier treatment) both avert deaths and save DALYs, just like getting treatment vs. not getting at all does. So doesn't it all play into the same value proposition?

See, me missing context matters here. I was imagining that the most pessimistic scenario would be that:

• ODH provides treatment for malaria which is faster, nearer & more convenient
• but patients would have otherwise gotten the same treatment, just later, further away and paying more costs to get it
• So the value of ODH wouldn't be the value of the treatment, it would be the value of making it more convenient

But as you point out ("Quality of care is important - but perhaps even more important like @Ray_Kennedy  pointed out is how quickly people get the treatment. Malaria is an exponentially replicating parasite, and hours can make a differece.") you can't just neatly separate getting faster care from getting better care. There are some fun things you could do with distributions, i.e., explicitly model the benefit as a function of how fast you get treatment, and then estimate the counterfactual value as

∫∫ (Value of getting treatment in h hours - Chance of having otherwise gotten treatment in (h + x) hours instead × Value of getting treatment in (h + x) hours) dx dh

(where the double integral just means that you are explicitly estimating the value of each possible pair of x and h and then weighing them according to how likely they are)

But I think this would be overkill, and only worth coming back to do explicitly if/when ODH is spending a few million a year. Still they might add some clarity if we don't do the calculations. Anyways, best of luck.

1) Ok, so let me try to rephrase and then you can tell me whether this makes sense

Per here, for an individual malaria patient, you are calculating the impact as: 

Badness of malaria × Chance of correct diagnosis × Chance of treatment regime working^[1].

And then we are both thinking, well, this should really be something like: 

Badness of malaria × Chance of correct diagnosis × Chance of treatment regime working × Chance that the treatment was counterfactual.

But then you are pointing out that your term for "Badness of malaria" is actually the "Burden of disease of malaria", which is actually how bad malaria is, but given that some of the patients are already receiving treatment. So in the original estimate, you didn't have the counterfactual adjustment, but in exchange your "Badness of malaria factor" was too low.

So that's my paraphrasing this so far. Do you think it's mostly correct?

Then, what my intuition tells me one could do would be to try to: 

a) Make the "Badness of malaria" factor be "Badness of untreated malaria" and model the "Chance that the treatment was counterfactual" factor explicitly. But then this wouldn't capture all the benefits, so then maybe have some upwards adjustment for quality of care? 

b) If gettting estimates for the badness of malaria is somehow too difficult, keep the burden of disease number, but add the counterfactual adjustment, and note that this is explicitly an underestimate.

At this point though, the counterfactual adjustment might get really gnarly:

• perhaps the most severe cases would have counterfactually been treated more often? Unclear how much this could reduce your estimate of impact.
• One could also model downstream effects, e.g., maybe making it less urgent for the Ugandan gvmt to have more hospitals? But then freeing its capacity to do other things, etc.
• The more factors you start to consider, the less this would be an apples-to-apples comparison against, say, GiveWell's estimate of AMF. Like, they have some adjustment for what would happen if AMF didn't exist, but do I buy those estimates? Not sure.

One could also just have the non-counterfactually adjusted numbers, compare to the non-counterfactually adjusted AMF numbers, and leave it as an exercise to the reader to input the relative counterfactual adjustment^[2]. Idk, maybe this is too cute.

My biggest uncertainty would be to what extent the value of ODH comes from [counterfactually averting deaths/saving lots of DALYs], vs [providing better quality of care, or more convenient treatment]. Anyways, hope that these thoughts are midly useful, though obviously I'm missing lots of context and looking at this from a really abstracted perspective.

1. ^{^}

   Or, chance that the treatment works × magnitude of the improvement, if we are being punctillious.

2. ^{^}

   E.g., if the non-counterfactually-adjusted impact for ODH is X, and the non-counterfactually adjusted impact for AMF is Y, and you are choosing between the two, you don't actually have to calculate the counterfactual adjustments for both, you can just estimate the ratio R of "how much more counterfactual is ODH than AMF", and then see if X * R > Y. As I said, maybe too cute.

Hi, thanks for writing this, I thought this was great. I agree with the limitations section.

Some nitpicks:

- Can you say a bit more about how you adjust for counterfactual impact? 5km is like an hour of walking? Are you halving this?
- Out of curiosity, how difficult would it be to subsidize transport instead?

I like that you had the balls to scale 5-20x in a year, respect.

Yes, ctrl+F on "customize tags"

Coming back to this, yeah, 1-on-1s seem super effective. Enough that I'd be curious for sanity checks here.

Happy to report that it's now ~3s using the truncate function, and ~7s using the original code (though they have slightly different functionality, one crops the function and the other one moves all the points outside the range into the nearest point in the range).

More notes:

- Uncertainty for people under 5 is shockingly wide
- My uncertainty about the lack of counterfactualness adjustment would be wider and my estimate lower (i.e., less impact). If AMF didn't exist, some other org might step in (but then that org would not have capacity for their marginal project? unclear). 

One component of this post was the quantified estimate of AMF's impact. My estimate was extremely rough. There is now a merely very rough estimate  here: <https://observablehq.com/@tanaerao/malaria-nets-analysis> , which looks as follows:

(I'm taking the estimate of lives saved, and then dividing them by life expectancy to get something resembling QALYs. I'm getting the life expectancy figures from Our World In Data, and then adding a bit of uncertainty).

Overall this update doesn't change this analysis much.

I am actually a bit confused about why you bothered to answer. Like, no answer was fine, an answer saying that you hadn't read it but pointing to resources and pitfalls you'd expect me to fall into would have been welcome, but your answer is just weird to me.

I agree with a), and mention this somewhat prominently in the post, so that kind of sours my reaction to the rest of your comment, as it feels like you are answering to something I didn't say:

The second shortcut I am taking is to interpret Rethink Priorities’s estimates as estimates of the relative value of humans and each species of animal—that is, to take their estimates as saying “a human is X times more valuable than a pig/chicken/shrimp/etc”. But RP explicitly notes that they are not that, they are just estimates of the range that welfare can take, from the worst experience to the best experience. You’d still have to adjust according to what proportion of that range is experienced, e.g., according to how much suffering a chicken in a factory farm experiences as a proportion of its maximum suffering.

and then later:

Note that I am in fact abusing RP’s estimates, because they are welfare ranges, not relative values. So it should pop out that they are wrong, because I didn’t go to the trouble of interpreting them correctly.

In any case, thanks for the references re: b) and c)

Re: b), it would in fact surprise me if my prior was uncalibrated. I'd also say that I am fairly familiar with forecasting distributions. My sense is that if you wanted to make the argument that my estimates are uncalibrated, you can, but I'd expect it'd be tricky.

Re: c), this is if you take a moral realist stance. If you take a moral relativist stance, or if I am just trying to describe that I do value, you have surprisingly little surface to object  to.

Otherwise, you don't have any response to the person who uses the same strategy to explain why they assign very low value to other humans, even if the face of evidence that these humans matter just as much as they do.

Yes, that is part of the downside of the moral relativist position. On the other hand, if you take a moral realist position my strong impression is that  you still can't convince e.g., a white supremacist, or an egoist, that all lives are equal, so you still share that downside. I realize that this is a longer argument though.

Anyways, I didn't want to leave your comment unanswered but I will choose to end this conversation here (though feel free to reply on your end).

Like, I feel like with the same type of argument that is made in the post I could write a post saying "there are no voting impossibility theorems" and then go ahead and argue that the Arrow's Impossibility Theorem assumptions are not universally proven, and then accuse everyone who ever talked about voting impossibility theorems that they are making "an error" since "those things are not real theorems". And I think everyone working on voting-adjacent impossibility theorems would be pretty justifiedly annoyed by this.

I think that there is some sense in which the character in your example would be right, since:

• Arrow's theorem doesn't bind approval voting.
• Generalizations of Arrow's theorem don't bind probabilistic results, e.g., each candidate is chosen with some probability corresponding to the amount of votes he gets.

Like, if you had someone saying there was "a deep core of electoral process" which means that as they scale to important decisions means that you will necessarily get "highly defective electoral processes", as illustrated in the classic example of the "dangers of the first pass the post system". Well in that case it would be reasonable to wonder whether the assumptions of the theorem bind, or whether there is some system like approval voting which is much less shitty than the theorem provers were expecting, because the assumptions don't hold.

The analogy is imperfect, though, since approval voting is a known decent system, whereas for AI systems we don't have an example friendly AI.

(if this post is right)

The post does actually seem wrong though. 

Glad that I added the caveat.

Also, the title of "there are no coherence arguments" is just straightforwardly wrong. The theorems cited are of course real theorems, they are relevant to agents acting with a certain kind of coherence, and I don't really understand the semantic argument that is happening where it's trying to say that the cited theorems aren't talking about "coherence", when like, they clearly are.

Well, part of the semantic nuance is that we don't care as much about the coherence theorems that do exist if they will fail to apply to current and future machines

IMO completeness seems quite reasonable to me and the argument here seems very weak (and I would urge the author to create an actual concrete situation that doesn't seem very dumb in which a highly intelligence, powerful and economically useful system has non-complete preferences).

Here are some scenarios:

• Our highly intelligent system notices that to have complete preferences over all trades would be too computationally expensive, and thus is willing to accept some, even a large degree of incompleteness. 
• The highly intelligent system learns to mimic the values of human, which end up having non-complete preferences, which the agent mimics
• You train a powerful system to do some stuff, but also to detect when it is out of distribution and in that case do nothing. Assuming you can do that, their preference is incomplete, since when offered tradeoffs they always take the default option when out of distribution. 

The whole section at the end feels very confused to me. The author asserts that there is "an error" where people assert that "there are coherence theorems", but man, that just seems like such a weird thing to argue for. Of course there are theorems that are relevant to the question of agent coherence, all of these seem really quite relevant. They might not prove the things in-practice, as many theorems tend to do. 

Mmh, then it would be good to differentiate between:

• There are coherence theorems that talk about some agents with some properties
• There are coherence theorems that prove that AI systems as will soon exist in the future will be optimizing utility functions

You could also say a third thing, which would be: there are coherence theorems that strongly hint that AI systems as will soon exist in the future will be optimizing utility functions. They don't prove it, but they make it highly probable because of such and such. In which case having more detail on the such and such would deflate most of the arguments in this post, for me.

For instance:

“‘Coherence arguments’ mean that if you don’t maximize ‘expected utility’ (EU)—that is, if you don’t make every choice in accordance with what gets the highest average score, given consistent preferability scores that you assign to all outcomes—then you will make strictly worse choices by your own lights than if you followed some alternate EU-maximizing strategy (at least in some situations, though they may not arise). For instance, you’ll be vulnerable to ‘money-pumping’—being predictably parted from your money for nothing.

This is just false, because it is not taking into account the cost of doing expected value maximization, since giving consistent preferability scores is just very expensive and hard to do reliably. Like, when I poll people for their preferability scores, they give inconsistent estimates. Instead, they could be doing some expected utility maximization, but the evaluation steps are so expensive that I now basically don't bother to do some more hardcore approximation of expected value for individuals, but for large projects and organizations.  And even then, I'm still taking shortcuts and monkey-patches, and not doing pure expected value maximization.

“This post gets somewhat technical and mathematical, but the point can be summarised as:

• You are vulnerable to money pumps only to the extent to which you deviate from the von Neumann-Morgenstern axioms of expected utility.

In other words, using alternate decision theories is bad for your wealth.”

The "in other words" doesn't follow, since EV maximization can be more expensive than the shortcuts.

Then there are other parts that give the strong impression that this expected value maximization will be binding in practice:

“Rephrasing again: we have a wide variety of mathematical theorems all spotlighting, from different angles, the fact that a plan lacking in clumsiness, is possessing of coherence.”

“The overall message here is that there is a set of qualitative behaviors and as long you do not engage in these qualitatively destructive behaviors, you will be behaving as if you have a utility function.”

  “The view that utility maximizers are inevitable is supported by a number of coherence theories developed early on in game theory which show that any agent without a consistent utility function is exploitable in some sense.”

Here are some words I wrote that don't quite sit right but which I thought I'd still share: Like, part of the MIRI beat as I understand it is to hold that there is some shining guiding light, some deep nature of intelligence that models will instantiate and make them highly dangerous. But it's not clear to me whether you will in fact get models that instantiate that shining light. Like, you could imagine an alternative view of intelligence where it's just useful monkey patches all the way down, and as we train more powerful models, they get more of the monkey patches, but without the fundamentals. The view in between would be that there are some monkey patches, and there are some deep generalizations, but then I want to know whether the coherence systems will bind to those kinds of agents.

No need to respond/deeply engage, but I'd appreciate if you let me know if the above comments were too nitpicky.

I appreciate the whole post. But I personally really enjoyed the appendix. In particular, I found it informative that Yudkowsk can speak/write with that level of authoritativeness, confidence, and disdain for others who disagree, and still be wrong (if this post is right).

Seems like a good idea, but also a fair bit of work, so I'd rather wait until RP releases their value ratios over actually existing humans and animals, and update on those. But if you want to do that, my code is open source.

Hey, I thought I'd make a Bayesian adjustment to the results of this post. To do this, I am basically ignoring all nuance. But I thought that it might still be interesting. You can see it here: https://nunosempere.com/blog/2023/02/19/bayesian-adjustment-to-rethink-priorities-welfare-range-estimates/

I upvoted because I thought the argument was coherent, as in, it had a logical structure, even though I don't buy it.

Coming back to this, and comparing it to other entries, I also appreciate that it didn't bite more than it could chew.

So coming back and looking at this, one central mystery is: why is the multiplier so high? Some possible answers might be:

• Possible answer #1: Operations people are the engine of EA. They make shit happen, which is what ultimately results in impact. EA should raise operations salaries.
• Possible answer #2: Operations *positions* are valuable, but operations people are easy to find and easily replaceable, which is why salaries are generally lower than for other positions. Thus, although creating organizations which can deploy operations people is very valuable, operations people themselves are not that valuable.
• Possible answer #3: Something else.

I'm also confused about whether operations roles are all similar enough that they can be modelled the same way.

So if I was working more on this, I'd probably:

• put some thought about what is happening here, qualitatively, before continuing with the quantitative estimates.
• Also, because of simplicity, I'd probably stop at estimating the impact of ops as a multiplier, which could then be combined with specific estimates for specific  organizations, rather than combining multipliers with broad cause area factors. That would be more work, though.
•  

Nice, thanks

I think logits are usually log(1/(1-p)), but I think that those negative numbers are just log(p).

Fair point!