Measuring the diagnostic accuracy of tests
What is a diagnostic accuracy study?
A diagnostic accuracy study measures the reliability of diagnostic tests outside of the highly controlled research environment. This is crucial as no test is perfect and this gives rise to false positive and false negative results which have consequences for the patient. The performance of the test may change when it is used outside of the laboratory and in the hands of different people. Examining the characteristics of the test also pinpoints the scenarios where it could have the most beneficial impact.
When should diagnostic accuracy studies be done?
For this type of analysis we require the developers to have a CE marked or Research Use Only product and data to demonstrate the analytical performance of the test.
How is a diagnostic accuracy study conducted?
The diagnostic accuracy is usually measured through benchmarking the performance of the new test against a highly reliable reference standard. This may be known as the “gold standard”. Two important features are examined when evaluating the accuracy of a test:
Sensitivity
The proportion of people with a condition who test positive. A test with sensitivity of 95% would mean that 5 in 100 people who have a condition would test negative (false negative). They have a condition, but the test says that they don’t.
Specificity
The proportion of people without a condition who test negative. A specificity of 95% means that 5 in 100 people who do not have a condition would test positive (false positive). They do not have a condition, but the test says that they do.
However, these numbers do not give a complete picture of a test’s reliability. The performance of a test in the real world can be quite different, depending on how common the condition is in the population that is tested (the prevalence).
High prevalence scenario (20% of the population has the condition)
In this scenario in a group of 100 people tested, 20 would have the condition. For a diagnostic test with 95% sensitivity and 95% specificity the diagnostic accuracy of the test would be as follows:
In the high prevalence scenario, of the 23 people who test positive, four of those do not have the condition (false positive). So, for a person who tests positive, there is an 83% chance they have the condition.
Low prevalence scenario (5% of the population has the condition)
In this scenario if 100 people are tested, the diagnostic accuracy of the test will look quite different when using the same test which still has 95% sensitivity and 95% specificity:
In the low prevalence scenario, of the 10 people who test positive, five of those do not have the condition (false positive). So, for a person who tests positive, there is a 50% chance they have the condition.
The way in which a test is used has important implications for the interpretation of the results. For example, a test might be used in a hospital where there is an outbreak of an infection or as part of a mass screening programme in the community. The required accuracy of the test, including the acceptable sensitivity and specificity, may vary in these situations as the prevalence of the infection could be different.
What we do
We can help to:
- Design an appropriate diagnostic accuracy study
- Set up the diagnostic accuracy study
- Analyse and communicate the results from the diagnostic accuracy study
Look at the impact posts below for examples of our diagnostic accuracy work.
