-
I assume in medical tests they know who does/does not have the virus via other methods. The test is then measured against these. Therefore, the accuracy of the results are related to this information, not any information about the general public.
If you have a test that has a specificity of one value (97%) and a sensitivity of another value (98%) then it will have a different accuracy if you give it all expected positive test inputs than if you give it all expected negative test inputs. Therefore the accuracy of a test depends on the prevalence of the disease in the population.
This page may help explain it: https://www.medcalc.org/calc/diagnostic_test.php
That page goes further and does 95% confidence intervals.
Fundamentally you're trying to combine two accuracy figures (one for getting it right with a positive outcome and one for getting it right with a negative outcome) into a single accuracy figure for the test. You can only do this if you know the prevalence population wide.
If you don't look at it population wide then you can't combine the two accuracy figures (sensitivity and specificity) into a single "accuracy" figure.
Greenbank-
Fundamentally you're trying to combine two accuracy figures (one for getting it right with a positive outcome and one for getting it right with a negative outcome) into a single accuracy figure for the test. You can only do this if you know the prevalence population wide.
If you don't look at it population wide then you can't combine the two accuracy figures (sensitivity and specificity) into a single "accuracy" figure.
Which is why you have a test/sample group. That's the population for which you know the number of false positives, false negatives, true positives, and true negatives. That's the population from which the accuracy of a test can be calculated.
Extrapolation to the world becomes more complicated. The prevalence in the population will impact raw results. But the accuracy at an individual level will remain the same (within reason/whatever p value).
As far as I understand! Not a doctor! Etc!
I'm really sorry if I'm being obtuse here, but maybe this is a discipline thing and I don't get how things are done in medicine (or it's me being a moron), but here:
you seem to be describing an F1 score. This would be calculated using test data - not population data - for which we know the features of the test population. Otherwise we wouldn't be able to give any measure of precision/recall (or sensitivity/specificity). So false/true positives/negatives can be reported with 100% accuracy. A test's accuracy will then be calculated based on how well it achieves results matching reality in every case.
I assume in medical tests they know who does/does not have the virus via other methods. The test is then measured against these. Therefore, the accuracy of the results are related to this information, not any information about the general public.
Just to reiterate, I do understand that the overall number of accurate results will depend on the prevalence of the disease in the population. But the accuracy of the test, in my understanding, should be independent of this.