Re: Analysis of repeated measurements across different methods
- From: "Jeff Miller" <milleratotago@xxxxxxxxx>
- Date: 19 Apr 2006 18:34:47 -0700
artitj@xxxxxxxxx wrote:
If anybody could suggest an appropriate statistical analysis toI'm not sure I can supply that, but maybe some of these comments
perform, I would much appreciate it.
will help you get pointed in the right direction.
I am doing experiments to determine whether two automated methods, AExactly what do you mean when you ask whether automated A is
and B, are significantly different than 3 human measurements (that is,
3 different people). The reason for 3 human measurements is because the
"correct" measurement is not known, as this is measured in living
things.
different than the three human measurements? I presume the three
humans didn't all give identical measurements in all cases, or you
wouldn't have bothered to use three in the first place. So, are you
simply asking whether A is different from the _average_ of the three
humans? If so, you might consider whether it really gains you anything
to keep the three humans' scores separate, or whether you might
just as well use the average of these three scores for each frame.
Next, as you probably know, the question of statistical significance
basically asks whether the deviations of the data from some hypothesis
could be explained by random error. Well, what is randomly sampled in
your
example? Are you thinking of the humans as random representatives
from some larger sample? Or the videos? The frames? I'm not sure
which one(s) would be most appropriate from your question.
(If I had a gold standard, I would want to determine whichThis suggests to me that you might as well use the average across
method had a smaller error relative to the gold standard.)
the three humans (for each frame), as the closest you can get to
the gold standard. Distinguishing among the different humans's scores
tells you something about the accuracy of your "gold standard"
measurements, which is useful, but that info seems subsidiary to me.
I have about 10 video sequences, each with a varying # of frames
ranging from 30 to 300. For each method (2 automated methods, 3 human
observers), I measure the width of the blood vessel in each frame. The
width of the blood vessels are continuous integer data.
My data looks something like this:
I would think of it tabulated like this instead:
Video Frame # WidthMethodA WidthMethodB WidthMethodC ...
1 1 5 9
7 ...
1 2 6 12
9 ...
For starters, then, I think it might be useful to compare the
following two correlations (correlating across all frames):
o correlation of WidthMethodA scores with AvgHuman score
o correlation of WidthMethodB scores with AvgHuman score
Or, if there are substantial differences between videos, it might be
better to run correlations like this separately for each video,
correlating across frames.
I've been reading up on ANOVA, and I think maybe a two-factor ANOVAI think you are on the wrong track here. ANOVA is good for comparing
with repeated measurements (where factor 1 is width and factor 2 is
method) would work
means, but I suspect that isn't what truly interests you. Suppose,
for example, that Method A gives exactly the same mean as the
humans, on average across all videos and frames? Does that tell
you that method A is good? Not really, I would think. It may not
actually match the gold standard of humans very well on a frame by
frame basis, but instead its errors may just average out quite well
across frames to give you the "right" overall mean.
It sounds to me like you should be reading about measures of
correlation and regression, or interrater reliability, rather than
ANOVA,
but maybe I've got the wrong idea of what you are trying to do.
Hope that helps,
.
- Follow-Ups:
- References:
- Prev by Date: Multiple linear regression: How much deviation from normal is too much?
- Next by Date: Re: cox regression with 3 outcomes
- Previous by thread: Analysis of repeated measurements across different methods
- Next by thread: Re: Analysis of repeated measurements across different methods
- Index(es):
Relevant Pages
|
