Tweet
Hi all,
On occasion I have noticed debates or discussions stalling at a point in the discussion because the two viewpoints get stuck trying to come to some agreement over whether or not the evidence we have provides sufficient grounds to move on. This, of course, is a worthwhile focus at times, but there comes a point when its worth is questionable.
I'm going to present things here in a slightly abstracted form, because it is the notion of how we compare two competing views that I want to focus on, not any specific example. Partly, I wish to avoid the thread getting distracted by JtR specific topics, and focus on something that is applicable to much of what we do here.
If side A is presenting some explanation for events, and they bolster that by referencing something from the pool of information we have to work with. I’m using the word information here rather than evidence simply to avoid getting distracted by definitions. The information pool includes all of the things we have to examine, from court documents, police letters, newspapers, memoirs, and so forth. Now let’s say the side B argues “you cannot move on because there is some possibility that information is not correct”. How should we evaluate that argument? Is it sufficient to halt the discussion at that point in order to resolve that issue, or is side B’s argument insufficient?
Well, that really depends on how large that possibility of error is. I’m going to switch to use the term probability here, but only in the sense of percentages, and not in the sense that probability necessarily means “with high likelihood”. The probability, therefore, implied in Side B’s argument could be a value anywhere between and including 0% and 100%. I know side B’s argument doesn’t make any sense if the probability is 0%, but 0% is a value for probability so it technically gets included here.
Obviously, if we can demonstrate that Side B’s probability for error is 0% (or the converse, that Side A’s probability for being correct is 100%) we move on. Conversely, if side B can show the probability for error is 100%, then Side A has to drop their claim.
While the above constitutes the ideal, it is never achieved in the real world. This is why to secure a conviction the prosecution has to establish their case beyond reasonable doubt, and not beyond all doubt. Which is to say, the defense has to establish reasonable doubt, not just any doubt.
Too often arguments get bogged down at a point where one side or the other continues to point to the fact that this ideal has not been reached, that the probability for error has not been reduced to 0%, therefore their presentation must still be considered with as much regard as the other side’s. That, however, is not true.
To keep it simple, let’s say Side A is arguing for the idea A to be true (that could be something like the time at which a murder occurred, for example, or that Stride is a victim of JtR), and Side B is arguing that is not true (donated as ~A, which just reads as “not A”). Now, because things must be either A or ~A (that covers every possible condition), then the probability of A + ~A is 100%. Making the probability of ~A = 100 – the probability of A. Simply saying that the probability of ~A is > 0% is not a sufficient reason to drop the conclusion A.
What we do is compare the ratio of those probabilities, and create what is called an “odds ratio”. For example, if we knew the probability of A was 90% (so we had 90% confidence that A was true), side B is arguing “But there’s a 10% chance that ~A is the true state, so you cannot move on and my counter explanation is just as good as yours”. That might seem appealing, but notice, 90% is 9 times 10%, we have an odds ratio of 9:1 that side A is correct. That would be considered, in the context of JtR discussions at least, a safe bet. If, however, the evidence for A was weaker, and Side A only had a 50% chance of being correct, then one is no better off betting on Side A than Side B, we’re at a point where indeed, Side B has every reason to argue that moving on is a bad idea. Obviously, once Side A’s chance of being correct falls below 50%, then Side B’s case is the safer one to bet on, and increasingly so as the chance for Side falls lower and lower.
Now, in most situations, we can’t put actually numbers to each side. What we do is present ties to information. We debate the relative reliability of various sources of information, and so forth. And from that, we examine which side has more information in support than the other, and we would include in that examination the quality of the sources, and how direct the information is connected to the debated topic. If one side comes out with an overwhelming amount of information in its favour, and the other side only has the argument of “but it might be wrong”, with either little information to show it is wrong, or at best only tangential bits of information far removed from the debated point, that is a signal that the odds ratio fall strongly in favour of the side with the supporting information.
Does it mean that side is 100% absolutely incapable of being wrong? Of course not, we never can reach that absolute state of certainty (even DNA analyses in courts are presented as having some extremely small chance of being wrong, after all – do we toss that out? Or do we go with the odds of it being right vs wrong? Clearly, it is the odds that matter).
I think it is important, and would be helpful to many discussions, therefore, to keep in mind that it is not a valid argument against a point to simply say “that information might be wrong”. What one has to do is draw upon information to show that the odds of it being wrong are truly in favour of that over that of the other Side. If neither side has much information, we’re getting around that 50% point, but if one side has a lot of information, of a reasonable level of quality, and the other side has nothing but that fall back position, the side with the supporting information is considered to have the greater odds of being correct.
Yes, there is a lot of subjectivity in what I’m outlining here, though there may be times when it can be more objectively determined. What’s important, though, is that the argument “there’s a possibility the information is wrong” is not information that gets one great odds; it does not make your argument a safe bet, nor does it make the other side the unsafe bet.
On occasion I have noticed debates or discussions stalling at a point in the discussion because the two viewpoints get stuck trying to come to some agreement over whether or not the evidence we have provides sufficient grounds to move on. This, of course, is a worthwhile focus at times, but there comes a point when its worth is questionable.
I'm going to present things here in a slightly abstracted form, because it is the notion of how we compare two competing views that I want to focus on, not any specific example. Partly, I wish to avoid the thread getting distracted by JtR specific topics, and focus on something that is applicable to much of what we do here.
If side A is presenting some explanation for events, and they bolster that by referencing something from the pool of information we have to work with. I’m using the word information here rather than evidence simply to avoid getting distracted by definitions. The information pool includes all of the things we have to examine, from court documents, police letters, newspapers, memoirs, and so forth. Now let’s say the side B argues “you cannot move on because there is some possibility that information is not correct”. How should we evaluate that argument? Is it sufficient to halt the discussion at that point in order to resolve that issue, or is side B’s argument insufficient?
Well, that really depends on how large that possibility of error is. I’m going to switch to use the term probability here, but only in the sense of percentages, and not in the sense that probability necessarily means “with high likelihood”. The probability, therefore, implied in Side B’s argument could be a value anywhere between and including 0% and 100%. I know side B’s argument doesn’t make any sense if the probability is 0%, but 0% is a value for probability so it technically gets included here.
Obviously, if we can demonstrate that Side B’s probability for error is 0% (or the converse, that Side A’s probability for being correct is 100%) we move on. Conversely, if side B can show the probability for error is 100%, then Side A has to drop their claim.
While the above constitutes the ideal, it is never achieved in the real world. This is why to secure a conviction the prosecution has to establish their case beyond reasonable doubt, and not beyond all doubt. Which is to say, the defense has to establish reasonable doubt, not just any doubt.
Too often arguments get bogged down at a point where one side or the other continues to point to the fact that this ideal has not been reached, that the probability for error has not been reduced to 0%, therefore their presentation must still be considered with as much regard as the other side’s. That, however, is not true.
To keep it simple, let’s say Side A is arguing for the idea A to be true (that could be something like the time at which a murder occurred, for example, or that Stride is a victim of JtR), and Side B is arguing that is not true (donated as ~A, which just reads as “not A”). Now, because things must be either A or ~A (that covers every possible condition), then the probability of A + ~A is 100%. Making the probability of ~A = 100 – the probability of A. Simply saying that the probability of ~A is > 0% is not a sufficient reason to drop the conclusion A.
What we do is compare the ratio of those probabilities, and create what is called an “odds ratio”. For example, if we knew the probability of A was 90% (so we had 90% confidence that A was true), side B is arguing “But there’s a 10% chance that ~A is the true state, so you cannot move on and my counter explanation is just as good as yours”. That might seem appealing, but notice, 90% is 9 times 10%, we have an odds ratio of 9:1 that side A is correct. That would be considered, in the context of JtR discussions at least, a safe bet. If, however, the evidence for A was weaker, and Side A only had a 50% chance of being correct, then one is no better off betting on Side A than Side B, we’re at a point where indeed, Side B has every reason to argue that moving on is a bad idea. Obviously, once Side A’s chance of being correct falls below 50%, then Side B’s case is the safer one to bet on, and increasingly so as the chance for Side falls lower and lower.
Now, in most situations, we can’t put actually numbers to each side. What we do is present ties to information. We debate the relative reliability of various sources of information, and so forth. And from that, we examine which side has more information in support than the other, and we would include in that examination the quality of the sources, and how direct the information is connected to the debated topic. If one side comes out with an overwhelming amount of information in its favour, and the other side only has the argument of “but it might be wrong”, with either little information to show it is wrong, or at best only tangential bits of information far removed from the debated point, that is a signal that the odds ratio fall strongly in favour of the side with the supporting information.
Does it mean that side is 100% absolutely incapable of being wrong? Of course not, we never can reach that absolute state of certainty (even DNA analyses in courts are presented as having some extremely small chance of being wrong, after all – do we toss that out? Or do we go with the odds of it being right vs wrong? Clearly, it is the odds that matter).
I think it is important, and would be helpful to many discussions, therefore, to keep in mind that it is not a valid argument against a point to simply say “that information might be wrong”. What one has to do is draw upon information to show that the odds of it being wrong are truly in favour of that over that of the other Side. If neither side has much information, we’re getting around that 50% point, but if one side has a lot of information, of a reasonable level of quality, and the other side has nothing but that fall back position, the side with the supporting information is considered to have the greater odds of being correct.
Yes, there is a lot of subjectivity in what I’m outlining here, though there may be times when it can be more objectively determined. What’s important, though, is that the argument “there’s a possibility the information is wrong” is not information that gets one great odds; it does not make your argument a safe bet, nor does it make the other side the unsafe bet.
- Jeff
Comment