Sample size and power for the area under a ROC curve
This calculator sizes a study testing whether the area under a ROC curve (AUC) for a single diagnostic test or classifier differs from a reference value — typically 0.5 (chance-level discrimination), but any target AUC can be used. It uses the Hanley–McNeil (1982) approximation to AUC variance (the Q1/Q2 terms below are Hanley & McNeil's negative-exponential special case, not a general binormal-model result), with the critical value calibrated under the reference AUC and power evaluated under the anticipated AUC via a normal approximation — the same two-standard-error approach used elsewhere on this site (e.g. McNemar’s test). This is a large-sample approximation, not an exact finite-sample method: it assumes independent positive/diseased and negative/non-diseased samples, and its accuracy can degrade for very small groups, AUCs near 0 or 1, or score distributions with substantial ties. It tests one AUC against a fixed reference value — it does not compare two correlated ROC curves (e.g. two classifiers evaluated on the same subjects).
Required total sample size across plausible anticipated AUCs and negative:positive allocation ratios, using the current power, alpha, reference AUC, and test type.
| Anticipated AUC | Ratio 1:1 | Ratio 2:1 | Ratio 3:1 |
|---|
Sample size was calculated for a single ROC area under the curve (AUC) against a fixed reference AUC using the Hanley–McNeil negative-exponential approximation to AUC variance and a normal approximation for power, assuming independent positive/diseased and negative/non-diseased samples, the specified allocation ratio, alpha, power, and one- or two-sided hypothesis. Results are planning approximations and should be checked for small samples, AUCs near 0 or 1, substantial score ties, or correlated ROC curves. Citation: Hanley JA, McNeil BJ. The meaning and use of the area under a receiver operating characteristic (ROC) curve. Radiology. 1982;143:29–36.