Re: Evaluating a new diagnostic test: interim analysis possible?

David Jones wrote:
John Uebersax wrote:
A researcher is planning a study to evaluate a new diagnostic
test.

The test is estimated to have a true sensitivity of, say
Se = .9, based on theory and a pilot study. If the study
finds, say, Se = .80, it will be considered a success.

Let N be the sample size needed for the 90% CI of the
observed Se to be above .80, given a population Se of .90.
(We view Se as a simple proportion of disease-positive cases
that are also test-positive--so this is a CI for a proportion.)

Now the question: Is some form of interim analysis possible?
For example, if N is found to be 200, but if there is perfect
agreement on the first 50 cases, may the study be stopped?

My thinking is that a Bayesian would have no problem with this.
One could simply update the posterior pdf for population Se
after each new result. The study can be stopped when 90% of
the area of the posterior pdf exceeds .80.

However, I'd be interested in hearing other opinions.

There are existing approaches to doing this .... look under
"sequential methods", "sequential decision making", for example. In
the simplest case case you would decide whether to stop after each
observation, but there is really no reason not to allow decisions to
be made on an ad-hoc basis.

David Jones



The Bayesian approach is an excellent one. You can also use a frequentist approach without getting into complex calculations if you stop when the width of the confidence interval falls below a pre-specified number, ignoring its center.

Computation of sensitivity or specificity assume that the disease is a perfect dichotomy (it seldom is), that the test is a perfect dichotomy (it seldom is) and that one is interested in backward probabilities (patients certainly aren't - they want their pysicians to use test results and tell them the current probability of disease).

Frank Harrell
.