Measuring the accuracy of tests
We help to measure the accuracy and reliability of diagnostic tests outside of the highly controlled research environment. This is crucial as no test is perfect and this gives rise to false positive and false negative results which have consequences for the patient. The performance of the test may change when it is used in the real world and in the hands of different users. Examining the characteristics of the test also pinpoints the scenarios where it could have the most beneficial impact.
The “diagnostic accuracy” of a new test is usually measured through benchmarking it against a highly reliable reference standard, which may be known as the “gold standard”. Two important features are examined when evaluating the accuracy of a test:
Sensitivity
The proportion of people with a condition who test positive. A test with sensitivity of 95% would mean that 5 in 100 people who have a condition would test negative (false negative). They have a condition, but the test says that they don’t.
Specificity
The proportion of people without a condition who test negative. A specificity of 95% means that 5 in 100 people who do not have a condition would test positive (false positive). They do not have a condition, but the test says that they do.
However, these numbers do not give a complete picture of a test’s reliability. The performance of a test in the real world can be quite different, depending on how common the condition is in the population that is tested (the prevalence).
High prevalence scenario (20% of the population has the condition)
In this scenario in a group of 100 people tested, 20 would have the condition. For a diagnostic test with 95% sensitivity and 95% specificity the diagnostic accuracy of the test would be as follows:
In the high prevalence scenario, of the 23 people who test positive, four of those do not have the condition (false positive). So, for a person who tests positive, there is an 83% chance they have the condition.
Low prevalence scenario (5% of the population has the condition)
In this scenario if 100 people are tested, the diagnostic accuracy of the test will look quite different when using the same test which still has 95% sensitivity and 95% specificity:
In the low prevalence scenario, of the 10 people who test positive, five of those do not have the condition (false positive). So, for a person who tests positive, there is a 50% chance they have the condition.
The way in which a test is used has important implications for the interpretation of the results. For example, the ideal features of a test to be used for confirming a clinical diagnosis of a patient in hospital with an infection where there is an outbreak are very different to those for a test intended for a mass screening programme. Therefore, the required accuracy of the test, including the acceptable sensitivity and specificity, needs to be considered alongside the potential consequences arising from positive and negative results.
We have expertise in care pathway analysis which helps to identify the optimal role and setting for medical tests based around its characteristics. To find out more about our care pathway work follow this link.
To find out more about how we are testing the reliability of COVID-19 tests follow this link.
