Introduction to ROC Curves
|
Previous Section | Main Menu | Next Section |
The sensitivity and specificity of a diagnostic test depends on more
than just the "quality" of the test--they also depend on the definition
of what constitutes an abnormal test. Look at the the idealized graph
at right showing the number of patients with and without a disease arranged
according to the value of a diagnostic test. This distributions overlap--the test (like most)
does not distinguish normal from disease with 100% accuracy. The area of overlap indicates
where the test cannot distinguish normal from disease. In practice, we choose a cutpoint
(indicated by the vertical black line) above which we consider the test to be abnormal
and below which we consider the test to be normal. The position of the cutpoint will determine
the number of true positive, true negatives, false positives and false negatives. We may wish to
use different cutpoints for different clinical situations if we wish to minimize one of the erroneous
types of test results.
We can use the hypothyroidism
data from the likelihood ratio section to illustrate
how sensitivity and specificity change depending on the choice of T4 level
that defines hypothyroidism.
Recall the data on patients with suspected hypothyroidism reported by
Goldstein and Mushlin (J Gen Intern Med 1987;2:20-24.). The data on T4 values
in hypothyroid and euthyroid patients are shown graphically (below left) and in
a simplified tabular form (below right).
T4 value |
Hypothyroid |
Euthyroid |
5 or less |
18 |
1 |
5.1 - 7 |
7 |
17 |
7.1 - 9 |
4 |
36 |
9 or more |
3 |
39 |
Totals: |
32 |
93 |
Suppose that patients with T4 values of 5 or less are considered to be
hypothyroid. The data display then reduces to:
T4 value |
Hypothyroid |
Euthyroid |
5 or less |
18 |
1 |
> 5 |
14 |
92 |
Totals: |
32 |
93 |
You should be able to verify that the sensivity is 0.56 and the
specificity is 0.99.
Now, suppose we decide to make the definition of hypothyroidism less
stringent and now consider patients with T4 values of 7 or less to be hypothyroid.
The data display will now look like this:
T4 value |
Hypothyroid |
Euthyroid |
7 or less |
25 |
18 |
> 7 |
7 |
75 |
Totals: |
32 |
93 |
You should be able to verify that the sensivity is 0.78 and the
specificity is 0.81.
Lets move the cut point for hypothyroidism one more time:
T4 value |
Hypothyroid |
Euthyroid |
< 9 |
29 |
54 |
9 or more |
3 |
39 |
Totals: |
32 |
93 |
You should be able to verify that the sensivity is 0.91 and the
specificity is 0.42.
Now, take the sensitivity and specificity values above and put them
into a table:
Cutpoint |
Sensitivity |
Specificity |
5 |
0.56 |
0.99 |
7 |
0.78 |
0.81 |
9 |
0.91 |
0.42 |
Notice that you can improve the sensitivity by moving to cutpoint to a
higher T4 value--that is, you can make the criterion for a positive
test less strict. You can improve the specificity by moving
the cutpoint to a lower T4 value--that is, you can make the criterion
for a positive test more strict. Thus, there is a tradeoff
between sensitivity and specificity. You can change the definition of a
positive test to improve one but the other will decline.
The next section covers how to use the numbers we just calculated to
draw and interpret an ROC curve.
.
|
Previous Section | Main Menu | Next Section |