sample size for kappa?

Hi there,

Can you tell me roughly how large of a sample I will need in order to
calculate Kappa with 2 raters? My outcome variable is yes, no, partial
and not applicable.


I am doing a study assessing an audit tool. I have 2 randomly selected
raters (out of 4 raters) and a audit tool/questionnaire that has 250
questions. I have been criticized for having a sample size that is too
small (I am looking at 4 different departments).


Can anyone tell me if my sample size indeed is too small, or if because

I have 250 questions that it is fine? We have selected 4 departments
because we are interested if there is a difference in kappa between the

departments, but this isn't our primary objective. We really want to
know if there is inter rater agreement between the 2 raters on the 250
questions, and which questions had poor agreement.


Thanks so much,

.

Relevant Pages

  • Re: sample size for kappa?
    ... Kappa is not a very good 'absolute' statistic beyond the 2x2 case. ... raters and a audit tool/questionnaire that has 250 ... know if there is inter rater agreement between the 2 raters on the 250 ... agreement in measuring a single 'dimension' that is shared ...
    (sci.stat.consult)
  • Re: comparing Kappa Statistics in case of dependence
    ... we would like to compare different kappa statistics. ... Here is the simple situation for 3 raters, ... Z disagrees with Y ...
    (sci.stat.math)
  • Re: Quadratic weighted Kappa and the Intraclass Correlation Coefficient
    ... kappa, using quadratic weights, asymptotically ... The Case 2 ICC assumes that the two raters compared are a random ... weighted kappa assumes that the two raters considered are the only ... random sample. ...
    (sci.stat.edu)
  • Re: kappa & ICC questions
    ... Can the data from more than one pairs of raters in an inter-rater ... reliability study ... There exists a creature called a multi-rater kappa. ... rated by each rater in the test-retest reliability study ranges from 9 ...
    (sci.stat.edu)
Loading