• We now have two 10k groups. The test group from which the sensitivity and specificity are learned ("the same test group you used to measure the sensitivity/specificity"). And a second 10k (which may or may not be the same as the first, but in your example they are). This second group is having the test applied to them. The important thing is that these are two different things which I think we both agree on.

    The first group is used to judge and measure the test. Sensitivity and specificity are learned from this group. Whatever you do with the test after this, sensitivity and specificity do not change (unless they were wrong to start).

    The second group is that which the test being used on. In this case, we can say that the second 10,000 people in your example are being used as an analogue for the general population, no? The results will depend on the makeup of this 10,000 people. They will not mimic reality (that is, the tests are not perfect).

    So even if you apply your own test to the same test group you used to measure the sensitivity and specificity of the test you find that a positive result is not as accurate as you expected.

    The tests are not 100% accurate, this was never a point of confusion.

    So again, the sensitivity and specificity which are learned from the first group is what I thought at the very beginning you were claiming shifts, depending on prevalence in the population (this is what I quoted in my reply to Chalfie, and if you remember, the first thing I said to you after you replied was: "Ah, sorry. So the raw number of false positives/false negatives will shift depending on how many true negatives/true positives there are. Okay - I mistook your "-ve" and "+ve" to be analogues for specificity/sensitivity.")

    The results in the second group (be it a sample or the general population) are dependent on the makeup of that population and the sensitivity/specificity of the test. I agree with this and always have.

    If I'm still misunderstanding you we can either take this off public chat or you can rest assured that you tried and I'm a moron.

  • I'm not sure talking about groups/populations is helpful. You just test an individual and there are 2 possibilities: a positive result or a negative. Of the positives approx. 50% are false results, which is a failure of the test. Of the negatives, 99% are accurate and 1% are false negatives. It is much more likely that the testee is negative so the false positives are fairly common and false negatives are an (unlikely) test failure on top of an (unlikely) result, meaning a very small proportion. Hence, negative results are more likely to be true than positive results.

    You are correct that the sensitivity and specificity never change. Just the likelihood of the result being true or false, which is what @Greenbank meant by "66% accurate" or whatever (i.e. 66% of these are true, the rest are test failures)

    I don't think you actually disagree with each other

  • I'm not sure talking about groups/populations is helpful.

    You're probably right, but because I misunderstood the point from the start (I think!), it was necessary to distinguish between the calculated sensitivity/specificity and the actual test results.

    You just test an individual and there are 2 possibilities: a positive result or a negative. Of the positives approx. 50% are false results, which is a failure of the test. Of the negatives, 99% are accurate and 1% are false negatives. It is much more likely that the testee is negative so the false positives are fairly common and false negatives are an (unlikely) test failure on top of an (unlikely) result, meaning a very small proportion. Hence, negative results are more likely to be true than positive results.

    I agree with this (and Greenbank's many examples of it using actual numbers). There is zero that is contentious about this.

    You are correct that the sensitivity and specificity never change. Just the likelihood of the result being true or false.

    This is what I assumed, but because of the initial confusion, I thought there may be something happening in medicine which I didn't understand or know about. Hence my original confusion/question.

    I don't think you actually disagree with each other

    Ha, all I can say is that I don't disagree with your take on it.

About

Avatar for frankenbike @frankenbike started