Weighted Pros / Cons as a Normpost by SeanEngelhart (seansre) · 2021-05-29T02:37:22.966Z · EA · GW · 6 comments
Takeaway Motivation Method Example Weights on Arguments For / Against the Topic at Hand Pros Cons Output None 6 comments
I think using weighted pros / cons (or more generally, arguments for / against) would be a useful norm to promote. For a summary of the reasons why, see the Example section.
Though maybe not an explicit norm, many people in EA endorse the idea of putting probabilities to statements in order to clarify one's credence in them. Doing so allows people to be much more precise and avoid the ambiguity of phrases like "almost certain" or "significant chance." It's also helpful for discussion as it can make it clearer how and to what degree people agree or disagree. It seems that many EA community members generally value "putting numbers to things." As an extension of this, I think it would be helpful for more people to weight their pros / cons or arguments for / against when discussing a given topic. I don't think this is currently done often and can't recall a time when I've seen it done firsthand.
I started thinking about this after reading the answers on my question about AIS harm. When reading the responses, I found myself unsure how much weight the author would give some of their considerations. This made it more difficult to determine how I should update some of my thinking on the subject. Further, it made additional discussion more difficult because I wasn't sure how much I agreed or disagreed with each response (I also unfortunately haven't had as much time to respond as I would've liked). I think that not having weights makes it much more difficult to digest arguments for / against something since the reader is left in the dark as to how each consideration should stack up against the others. I also think there are benefits for the writer, which I touch on in the example below. I'm not at all trying to pick on the answers given at that post (I very much appreciated them!). From what I've seen, this is how most answers on most forums are given. I'm also not claiming that weights should be added in all situations, but I do think they are often helpful, especially in cases where one is explicitly listing arguments for or against something. Like quantifying our credences with probabilities, this may be another useful norm to promote. As an example, below is a weighted pros / cons list of using weighted pros / cons (or putting weights to arguments for / against, more generally).
This can be done in a variety of ways, but I generally use a 1-5 scale. I typically don't feel I need higher resolution than this. 1 is something like "not at all important," 2 is "unimportant," 3 is in the middle, which could mean "just as important as unimportant," 4 is "important," 5 is "extremely important." Also, if one is pressed for time or the context is quicker, more informal, and doesn't need as much precision, you could just use qualitative weights, like "important," "very important," "not important," etc. I'm very open to other methods of doing this as well.
Scale: 1-5 as described above, indicated in bold parentheses.
Note: these weights are put down lightly as I don't have much direct experience comparing situations when they are used to ones when they aren't. I've mostly just seen cases when they aren't used and what the effects of that are.
Putting Weights on Arguments For / Against the Topic at Hand
- From R's perspective: clarifies their understanding of W's position on each of the considerations. (5)
- Allows for better discussion for the following reasons (5):
- R can better tell if they agree / disagree with the weights put on each point, which will very likely affect the ultimate conclusion drawn from the points. If they disagree with the weights, they may very well disagree with the conclusion.
- if R agrees / disagrees with one or more of the weights, R can immediately recognize this as the source of agreement / disagreement. This can otherwise be difficult to discern when faced with a large, complex set of considerations and a general feeling of agreement / disagreement.
- both of the above points allow R to better understand W's writing and put R in a better position to respond in a meaningful way.
- From W's perspective: clarifies their understanding of their position on each of the considerations. I didn't list this as a 5 since I would guess that W has a more solid idea of what W's weights would be than R does. (4)
- Allows one to sum the weights of the "con / against" side and subtract this from the sum of the "pros / for" side, yielding a single numerical representation of how the points stack up against each other. I certainly don't think this should be the final say on the issue, but it can be a useful input when exploring the topic at hand. (4)
- Could mislead some people into taking the sum mentioned above as being ultimately decisive when it should just be used as another input (we might treat the weight of this input as corresponding to its absolute value. That is, the farther the value is from zero, the more confident it is in the direction of for / against). This doesn't seem highly worrisome to me since I don't think most people's intuition tells them to blindly trust numbers without any further thought. Moreover, I think that many have probably seen 80K Hour's career decision-making processes, which echo this advice to take the number as another input. (4)
- Takes additional time to add weights. The time burden can be at least somewhat mitigated by using more broad qualitative weights if necessary, which is why I think this consideration isn't as important. (3)
(5 + 5 + 4 + 4) - (4 + 3) = 11
Note: this output number should not be taken very seriously since it's not clear in my scale that a 1 is really half the value of a 2, for example. Instead, it could possibly be used as a very rough indication of how strong your credence is in a given direction, but it doesn't seem like a good idea to read much into the exact number value. For better quantitative approaches, see the comments below.
Comments sorted by top scores.