Re: Agreement for one subject with gold standard, ordinal data

I wonder whether standard correlation coefficients do the trick?

With this kind of exercise isn't the most critical information order
dependent? In other words an inversion at the beginning might be a more
critical error than one at the end?

Would some kind of weighted scoring be desirable?

Thom


"Bruce Weaver" <bweaver@xxxxxxxxxxxx> wrote in message
news:1123499643.891307dbb97bbec3bd5198b6ff3ab775@xxxxxxxxxxx
> Ray Koopman wrote:
>
>>
>>
>> The sum of the absolute differences is "Spearman's footrule". IIRC it
>> works ok for rankings that are close to one another but doesn't do so
>> well if they are very different. However, the sum of squared
>> differences, which is a linear function of the Pearson/Spearman
>> correlation of the two rankings, works well over the entire range of
>> agreement. I'd give more details, but my notes on this are in my
>> office.
>>
>> Kendall's W is a linear transformation of the average pairwise
>> correlation among k rankings, to map it from the interval [1/(k-1),1]
>> to the fixed interval [0,1]. For k = 2 -- the gold standard, and one
>> subject's ranking -- W = (1+r)/2, where r is the ordinary
>> Pearson/Spearman correlation of the two rankings.
>>
>> I would do the anova on W or r itself, not on some variance-stabilized
>> transform. I expect some people will object to this. My argument is
>> that transforming the dv to homogenize the variances distorts the
>> scale. It is appropriate only for testing the overall null and pairwise
>> contrasts in a one-way design; it can change the true values of more
>> complicated contrasts such as the main effects and interactions in a
>> two-way design.
>>
>
>
> Thanks Ray. I'll pass your comments on to the researcher.
>
>
> --
> Bruce Weaver
> bweaver@xxxxxxxxxxxx
> www.angelfire.com/wv/bwhomedir


.

Relevant Pages

  • Re: Is the Curt net a kind of decision tree?
    ... The two signals don't act as ... created by the network might have had something to do with the reward. ... If multiple pixels share a common correlation ... to the type of transformation these networks should be performing on the ...
    (comp.ai.philosophy)
  • Re: r^2 and log transformation
    ... appropriate transformation of regression variables’, ... from a set of alternative regression specifications involving different ... the correlation of the log-y values with the ... when you take logs.) ...
    (sci.stat.math)
  • Re: Problem related to a linear regression
    ... I think for finding the best correlation it's probably better not to ... separating the two operations correlation and transformation. ... either by minimising dy^2 or dx^2 or ODR. ...
    (sci.math)
  • Re: Rankig of Vtes Debate
    ... Even if there is some correlation between high ... >skill and high ratings. ... that, top to bottom, the rankings rank everyone accurately. ...
    (rec.games.trading-cards.jyhad)
  • Re: Rankig of Vtes Debate
    ... Even if there is some correlation between high ... >>skill and high ratings. ... > that, top to bottom, the rankings rank everyone accurately. ...
    (rec.games.trading-cards.jyhad)