Re: compare 2 odds ratios

Ray Koopman wrote:
arrayprofile@xxxxxxxxx wrote:
my apology for the ambuiguity. what I really meant is to compare the
results (i.e. odds ratios) of each diagnostic test in relation to the
actually clinical outcome (e.g. cancer vs. normal). Bruce is absolutely
right in his interpretation.

The interesting thing is that both tests were performed on the same set
of patients, so the data should look like:

outcome test1 test2
1 1 0
1 1 1
0 1 0
1 0 1
0 0 0
:
:

I always wonder if there is a statistical model than can handle the
data and answer my question. Bruce's suggestion is interesting, I just
feel it comes back down to the very simple t test type of thing, and
doesn't take advantage pf the fact that both tests were done one the
same patients.

Any more suggestions?

The data can be summarized as

O+ O-
2+ 2- 2+ 2-
1+ A B 1+ a b
1- C D 1- c d

(A+B)(c+d) (A+C)(b+d)
OR1 = ---------- OR2 = ----------
(a+b)(C+D) (a+c)(B+D)

OR1 (A+B)(c+d)(a+c)(B+D)
--- = -------------------- = Do you really want to look at this??
OR2 (a+b)(C+D)(A+C)(b+d)

How about the McNemar test on

2 2
Right Wrong
1 Right A+d B+c
1 Wrong C+b D+a

I'll tack on some additional comments to Ray's reply here.

I had seen the OP's follow up post today to the r-help list, where the sample data provided clarified some of the ambiguity now discussed here. That sample data clearly had three measures per subject, with the presumptive gold standard and the two comparative tests.

That being the case, it rules out the use of standard approaches that do not consider a dependent measures approach to this problem. In other words, the typical approaches normally represented by the CMH test (notwithstanding my post and Bruce's initial question), the Breslow-Day test of the homogeneity of OR's and log linear model approaches generally presume independence of measures.

That assumption is violated here and would require the use of (use your preferred term here) multi-level/hierarchical/repeated measures/mixed effects methods.

The per subject strata CMH approach as listed in Agresti and as I noted would be a possibility, but the McNemar test is easier to implement, yields essentially the same results and is available in R as the mcnemar.test() function in the default installation.

Upon seeing the post on r-help and after getting home this evening, I spent some time doing a Google and Medline search, looking for pointers to studies where a gold standard was being compared to two alternative tests, where the primary efficacy measure is dichotomous.

What appears to be "typical" in this situation, is the use of Cochran's Q Test, which is a generalization of the McNemar test to >2 strata. This test provides insight into whether or not there are differences across the multiple strata relative to marginal homogeneity.

Cochran's Q Test can then be followed by multiple pairwise comparisons performed with the standard McNemar test, giving due consideration to multiple comparisons correction approaches. Thus, in this case, performing GS vs T1, GS vs T2 and T1 vs T2.

A Cochran Q Test appears to be available in R, via the 'meta' package on CRAN in the metabin() function, though the calculations are in Sheskin below.

Subsequent research in Sheskin's Handbook of Parametric and Nonparametric Statistical Procedures (2004) brings one to Test 26: The Cochran Q Test (page 867). Extensive discussion of this approach and the subsequent pairwise comparison issues are covered and I would recommend that the OP secure a copy of the book to review this chapter. Good examples are provided as well.

A Medline search might not be a bad idea as well, if the OP needs to cite prior work for the selection of methodology.

Since a model based approach was mentioned and the OP appears to be using R, another option to consider here would be to use Doug Bates' lmer() function, which is in the lme4 CRAN package (technically it is in the Matrix package at the moment, but you actually install lme4 with dependencies). lmer() enables the specification of both fixed and random effects while allowing for a typical R glm()-like approach to model specifications and can use one of several standard glm model family distributions.

I attended Doug's half-day tutorial on lmer() in Vienna at the useR! 2006 meeting a couple of weeks ago and was quite impressed with the power and flexibility of the function. A caveat here however, is that Doug is in the midst of active development on the function and things are subject to change on a daily basis, as Doug duly noted in one example he used. Doug wrote an article for R News last year in Vol 5, Number 1 on Fitting Linear Mixed Models in R, which would be helpful if you want to pursue this approach.

HTH,

Marc Schwartz
.

Relevant Pages

  • Re: compare 2 odds ratios
    ... it rules out the use of standard approaches that do ... would be a possibility, but the McNemar test is easier to implement, ... lmer() enables the specification of both fixed and random ... Doug is in the midst of active development on the function and things ...
    (sci.stat.consult)
  • Re: Training...Ultra Hyperbole?
    ... performances in marathons...he literally says so. ... as doug advises.... ... standard is doug's words to the letter....which according to Doug says ... about tony & his running. ...
    (rec.running)
  • Re: upgrade to sbs2003?
    ... cents. ... >>I see that ISA Server doesn't come with the SBS2003 Standard Ed. Can I ...
    (microsoft.public.windows.server.sbs)
  • Re: future C standards
    ... be an unlimited amount of traffic if opened up to random ... discussion would have appreciable effect on Standard C ... would be little relevance for this newsgroup). ... In addition whether Doug realises it or not C has continued to develop with TRs and this is about the only place to discus those. ...
    (comp.std.c)