-
I'm really sorry if I'm being obtuse here, but maybe this is a discipline thing and I don't get how things are done in medicine (or it's me being a moron), but here:
In order to combine two accuracy figures (sensitivity and specificity) into a single 'test accuracy' figure it's necessary to know what at what ratio those individual accuracies need to be combined and that depends on the prevalence of the condition within the population being tested.
you seem to be describing an F1 score. This would be calculated using test data - not population data - for which we know the features of the test population. Otherwise we wouldn't be able to give any measure of precision/recall (or sensitivity/specificity). So false/true positives/negatives can be reported with 100% accuracy. A test's accuracy will then be calculated based on how well it achieves results matching reality in every case.
I assume in medical tests they know who does/does not have the virus via other methods. The test is then measured against these. Therefore, the accuracy of the results are related to this information, not any information about the general public.
Just to reiterate, I do understand that the overall number of accurate results will depend on the prevalence of the disease in the population. But the accuracy of the test, in my understanding, should be independent of this.
-
I assume in medical tests they know who does/does not have the virus via other methods. The test is then measured against these. Therefore, the accuracy of the results are related to this information, not any information about the general public.
If you have a test that has a specificity of one value (97%) and a sensitivity of another value (98%) then it will have a different accuracy if you give it all expected positive test inputs than if you give it all expected negative test inputs. Therefore the accuracy of a test depends on the prevalence of the disease in the population.
This page may help explain it: https://www.medcalc.org/calc/diagnostic_test.php
That page goes further and does 95% confidence intervals.
Fundamentally you're trying to combine two accuracy figures (one for getting it right with a positive outcome and one for getting it right with a negative outcome) into a single accuracy figure for the test. You can only do this if you know the prevalence population wide.
If you don't look at it population wide then you can't combine the two accuracy figures (sensitivity and specificity) into a single "accuracy" figure.
Greenbank
In order to combine two accuracy figures (sensitivity and specificity) into a single 'test accuracy' figure it's necessary to know what at what ratio those individual accuracies need to be combined and that depends on the prevalence of the condition within the population being tested.
If 100% of people have the virus then a test that gives 2% false negatives is only going to be 98% accurate.
If 100% of the people don't have the virus then a test that gives 3% false positives is only going to be 97% accurate.
That's an extreme but I hope it shows how a single test with two different accuracy figures can produce different overall accuracy results if the testing population is made up differently.