RTI uses cookies to offer you the best experience online. By clicking “accept” on this website, you opt in and you agree to the use of cookies. If you would like to know more about how RTI uses cookies and how to manage them please view our Privacy Policy here. You can “opt out” or change your mind by visiting: http://optout.aboutads.info/. Click “accept” to agree.
Identifying disagreement in ordinal ratings within a fixed panel of cytotechnologists
Holiday, D., & McLarty, JW. (1994). Identifying disagreement in ordinal ratings within a fixed panel of cytotechnologists. Analytical and Quantitative Cytopathology and Histopathology, 16(6), 381-392.
In many clinical settings, a fixed number of raters screen specimens with more than two ordinal outcomes (for example, mild, moderate, severe). A design is proposed that facilitates the measurement of both interrater and intrarater agreement and associated trends. Design deficiencies are discussed, as are the propriety and interpretation of some common indices of reliability and reproducibility. The concepts are illustrated with data from cytopathologic ratings for sputum light microscopy.