| HOME | HELP | FEEDBACK | SUBSCRIPTIONS | ARCHIVE | SEARCH | TABLE OF CONTENTS |
|
|
|
|||||||
|
|||||||||
Statistical Concepts Series |
1 From the Department of Radiology (H.L.K.) and MCP Hahnemann School of Public Health (M.P.), University of Pennsylvania Medical Center, 3600 Market St, Suite 370, Philadelphia, PA 19104. Received November 21, 2001; revision requested January 29, 2002; revision received March 4; accepted March 11. Supported by grant P01 CA 53141 from the National Cancer Institute, National Institutes of Health, U.S. Public Health Service, Department of Health and Human Services. Address correspondence to H.L.K. (e-mail: kundel@rad.upenn.edu).
Statistical measures are described that are used in diagnostic imaging for expressing observer agreement in regard to categorical data. The measures are used to characterize the reliability of imaging methods and the reproducibility of disease classifications and, occasionally with great care, as the surrogate for accuracy. The review concentrates on the chance-corrected indices, 
and weighted . Examples from the imaging literature illustrate the method of calculation and the effects of both disease prevalence and the number of rating categories. Other measures of agreement that are used less frequently, including multiple-rater , are referenced and described briefly.
© RSNA, 2003
Index terms: Diagnostic radiology, observer performance • Statistical analysis
This article has been cited by other articles:
|
|
 |
|
  H. Akiba, M. Tamakawa, H. Hyodoh, K. Hyodoh, N. Yama, T. Nonaka, Y. Minamida, M. Hashimoto, and M. Hareyama Assessment of Dural Arteriovenous Fistulas of the Cavernous Sinuses on 3D Dynamic MR Angiography AJNR Am. J. Neuroradiol., October 1, 2008; 29(9): 1652 - 1657. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 K. Tanitame, K. Sasaki, T. Sone, S. Uyama, M. Sumida, T. Ichiki, and K. Ito Anterior Chamber Configuration in Patients with Glaucoma: MR Gonioscopy Evaluation with Half-Fourier Single-Shot RARE Sequence and Microscopy Coil Radiology, October 1, 2008; 249(1): 294 - 300. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 S. Y. Kim, S. H. Park, E. K. Choi, S. S. Lee, K. H. Lee, J. C. Kim, C. S. Yu, H. C. Kim, A. Y. Kim, and H. K. Ha Automated Carbon Dioxide Insufflation for CT Colonography: Effectiveness of Colonic Distention in Cancer Patients with Severe Luminal Narrowing Am. J. Roentgenol., March 1, 2008; 190(3): 698 - 706. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 A. Taylor, E. V. Garcia, J. N. G. Binongo, A. Manatunga, R. Halkar, R. D. Folks, and E. Dubovsky Diagnostic Performance of an Expert System for Interpretation of 99mTc MAG3 Scans in Suspected Renal Obstruction J. Nucl. Med., February 1, 2008; 49(2): 216 - 224. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
D. S. Gierada, T. K. Pilgram, M. Ford, R. M. Fagerstrom, T. R. Church, H. Nath, K. Garg, and D. C. Strollo Lung Cancer: Interobserver Agreement on Interpretation of Pulmonary Findings at Low-Dose CT Screening Radiology, December 1, 2007; 246(1): 265 - 272. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
E. Just da Costa e Silva and G. Alves Pontes da Silva Eliminating Unenhanced CT When Evaluating Abdominal Neoplasms in Children Am. J. Roentgenol., November 1, 2007; 189(5): 1211 - 1214. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
S. K. Hofkes, B. J. Iskandar, P. A. Turski, L. R. Gentry, J. B. McCue, and V. M. Haughton Differentiation between Symptomatic Chiari I Malformation and Asymptomatic Tonsilar Ectopia by Using Cerebrospinal Fluid Flow Imaging: Initial Estimate of Imaging Accuracy Radiology, November 1, 2007; 245(2): 532 - 540. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 A. Suzuki, Y. Nakamoto, T. Terauchi, M. Kawamoto, Y. Okumura, Y. Suzuki, T. Sato, N. Takahashi, J. Lee, M. Senda, et al. Inter-observer Variations in FDG-PET Interpretation for Cancer Screening Jpn. J. Clin. Oncol., August 18, 2007; (2007) hym064v1. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 J. L. Alcazar, M. Garcia-Manero, and R. Galvan Three-Dimensional Sonographic Morphologic Assessment of Adnexal Masses: A Reproducibility Study J. Ultrasound Med., August 1, 2007; 26(8): 1007 - 1011. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
S. Suzuki, S. Furui, K. Okinaga, T. Sakamoto, J. Murata, A. Furukawa, and Y. Ohnaka Differentiation of Femoral Versus Inguinal Hernia: CT Findings Am. J. Roentgenol., August 1, 2007; 189(2): W78 - W83. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 R. KUMAR, R. A. BUMB, N. A. ANSARI, R. D. MEHTA, and P. SALOTRA CUTANEOUS LEISHMANIASIS CAUSED BY LEISHMANIA TROPICA IN BIKANER, INDIA: PARASITE IDENTIFICATION AND CHARACTERIZATION USING MOLECULAR AND IMMUNOLOGIC TOOLS Am J Trop Med Hyg, May 1, 2007; 76(5): 896 - 901. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
H.J. Cloft, T. Kaufmann, and D.F. Kallmes Observer Agreement in the Assessment of Endovascular Aneurysm Therapy and Aneurysm Recurrence AJNR Am. J. Neuroradiol., March 1, 2007; 28(3): 497 - 500. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 H. D. Stupak, T. H. M. Moulthrop, P. Wheatley, A. V. Tauman, and C. M. Johnson Jr Calcium Hydroxylapatite Gel (Radiesse) Injection for the Correction of Postrhinoplasty Contour Deficiencies and Asymmetries Arch Facial Plast Surg, March 1, 2007; 9(2): 130 - 136. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
Y. J. Kim, S. S. Raman, N. C. Yu, K. J. To'o, R. Jutabha, and D. S. K. Lu Esophageal Varices in Cirrhotic Patients: Evaluation with Liver CT Am. J. Roentgenol., January 1, 2007; 188(1): 139 - 144. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
T. A. Jaffe, L. C. Martin, C. M. Miller, K. M. Franklin, E. M. Merkle, W. M. Thompson, R. C. Nelson, D. M. DeLong, and E. K. Paulson Abdominal Pain: Coronal Reformations from Isotropic Voxels with 16-Section CT--Reader Lesion Detection and Interpretation Time Radiology, January 1, 2007; 242(1): 175 - 181. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
S. Kim, Y.-M. Huh, H.-T. Song, S.-A. Lee, J.-W. Lee, J. E. Lee, I. H. Chung, and J.-S. Suh Chronic Tibiofibular Syndesmosis Injury of Ankle: Evaluation with Contrast-enhanced Fat-suppressed 3D Fast Spoiled Gradient-recalled Acquisition in the Steady State MR Imaging Radiology, January 1, 2007; 242(1): 225 - 235. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
G. Andreisek, T. Pfammatter, K. Goepfert, D. Nanz, P. Hervo, R. Koppensteiner, and D. Weishaupt Peripheral Arteries in Diabetic Patients: Standard Bolus-Chase and Time-resolved MR Angiography Radiology, December 19, 2006; (2006) 2422051111. [Abstract] [Full Text]
|
|
|
|
 |
|
 R. de Silva, L. F. Gutierrez, A. N. Raval, E. R. McVeigh, C. Ozturk, and R. J. Lederman X-Ray Fused With Magnetic Resonance Imaging (XFM) to Target Endomyocardial Injections: Validation in a Swine Model of Myocardial Infarction Circulation, November 28, 2006; 114(22): 2342 - 2350. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 F Gilli, F Hoffmann, A Sala, F Marnetto, M Caldano, P Valentino, L Kappos, A Bertolotto, and R L. Lindberg Qualitative and quantitative analysis of antibody response against IFN{beta} in patients with multiple sclerosis Multiple Sclerosis, November 1, 2006; 12(6): 738 - 746. [Abstract] [PDF]
|
|
|
|
 |
|
 S. Widmalm, S. Brooks, T Sano, L. Upton, and D. McKay Limitation of the diagnostic value of MR images for diagnosing temporomandibular joint disorders. Dentomaxillofac. Radiol., September 1, 2006; 35(5): 334 - 338. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
K.-P. Jungius, M. R. Schmid, M. Zanetti, J. Hodler, P. Koch, and C. W. A. Pfirrmann Cartilaginous Defects of the Femorotibial Joint: Accuracy of Coronal Short Inversion Time Inversion-Recovery MR Sequence Radiology, August 1, 2006; 240(2): 482 - 488. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
F. H. Hetzer, G. Andreisek, C. Tsagari, U. Sahrbacher, and D. Weishaupt MR Defecography in Patients with Fecal Incontinence: Imaging Findings and Their Effect on Surgical Management Radiology, August 1, 2006; 240(2): 449 - 457. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
E. M. Remer, G. A. Motta-Ramirez, L. B. Shepardson, A. H. Hamrahian, and B. R. Herts CT histogram analysis in pathologically proven adrenal masses. Am. J. Roentgenol., July 1, 2006; 187(1): 191 - 196. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 N. G. Harness, D. Ring, D. Zurakowski, G. J. Harris, and J. B. Jupiter The Influence of Three-Dimensional Computed Tomography Reconstructions on the Characterization and Treatment of Distal Radial Fractures J. Bone Joint Surg. Am., June 1, 2006; 88(6): 1315 - 1323. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
E. Quaia, A. Palumbo, S. Rossi, F. Degobbis, S. Cernic, G. Tona, and M. Cova Comparison of visual and quantitative analysis for characterization of insonated liver tumors after microbubble contrast injection. Am. J. Roentgenol., June 1, 2006; 186(6): 1560 - 1570. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
R. M. Setser, J. K. Kim, Y. C. Chung, K. Chen, A. E. Stillman, R. Loeffler, O. P. Simonetti, J. A. Weaver, M. L. Lieber, and R. D. White Cine Delayed-Enhancement MR Imaging of the Heart: Initial Experience Radiology, June 1, 2006; 239(3): 856 - 862. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
H. Li, K. Sugimura, Y. Kaji, Y. Kitamura, M. Fujii, I. Hara, and M. Tachibana Conventional MRI capabilities in the diagnosis of prostate cancer in the transition zone. Am. J. Roentgenol., March 1, 2006; 186(3): 729 - 742. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
J. L. Alcazar, S. Ajossa, S. Floris, R. Bargellini, M. Gerada, and S. Guerriero Reproducibility of Endometrial Vascular Patterns in Endometrial Disease as Assessed by Transvaginal Power Doppler Sonography in Women With Postmenopausal Bleeding J. Ultrasound Med., February 1, 2006; 25(2): 159 - 163. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
E. J. Karimova, S. N. Rai, X. Deng, D. J. Ingle, A. C. Ralph, M. D. Neel, and S. C. Kaste MRI of Knee Osteonecrosis in Children with Leukemia and Lymphoma: Part 1, Observer Agreement Am. J. Roentgenol., February 1, 2006; 186(2): 470 - 476. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
E. J. Karimova, S. N. Rai, D. Ingle, A. C. Ralph, X. Deng, M. D. Neel, S. C. Howard, C.-H. Pui, and S. C. Kaste MRI of Knee Osteonecrosis in Children with Leukemia and Lymphoma: Part 2, Clinical and Imaging Patterns Am. J. Roentgenol., February 1, 2006; 186(2): 477 - 482. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
J. J. Hom, F. V. Coakley, J. P. Simko, A. Qayyum, Y. Lu, L. Schmitt, P. R. Carroll, and J. Kurhanewicz Prostate Cancer: Endorectal MR Imaging and MR Spectroscopic Imaging--Distinction of True-Positive Results from Chance-detected Lesions Radiology, January 1, 2006; 238(1): 192 - 199. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 L. Denny, L. Kuhn, M. De Souza, A. E. Pollack, W. Dupree, and T. C. Wright Jr Screen-and-Treat Approaches for Cervical Cancer Prevention in Low-Resource Settings: A Randomized Controlled Trial JAMA, November 2, 2005; 294(17): 2173 - 2181. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
G. Andreisek, M. Kilgus, D. Burg, N. Saupe, D. W. Crook, V. Meyer, B. Marincek, and D. Weishaupt MRI of the Intrinsic Muscles of the Hand: Spectrum of Imaging Findings and Clinical Correlation Am. J. Roentgenol., October 1, 2005; 185(4): 930 - 939. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
H. Arakawa, P. A. Gevenois, Y. Saito, H. Shida, V. De Maertelaer, H. Morikubo, and M. Fujioka Silicosis: Expiratory Thin-Section CT Assessment of Airway Obstruction Radiology, September 1, 2005; 236(3): 1059 - 1066. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 D. A. Lynch, J. D. Godwin, S. Safrin, K. M. Starko, P. Hormel, K. K. Brown, G. Raghu, T. E. King Jr., W. Z. Bradford, D. A. Schwartz, et al. High-Resolution Computed Tomography in Idiopathic Pulmonary Fibrosis: Diagnosis and Prognosis Am. J. Respir. Crit. Care Med., August 15, 2005; 172(4): 488 - 493. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
M. R. Schmid, C. W. A. Pfirrmann, P. Koch, M. Zanetti, B. Kuehn, and J. Hodler Imaging of Patellar Cartilage with a 2D Multiple-Echo Data Image Combination Sequence Am. J. Roentgenol., June 1, 2005; 184(6): 1744 - 1748. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 S. Yuan, M. L. Astion, J. Schapiro, and A. P. Limaye Clinical Impact Associated with Corrected Results in Clinical Microbiology Testing J. Clin. Microbiol., May 1, 2005; 43(5): 2188 - 2193. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
P. E. Crewson Reader Agreement Studies Am. J. Roentgenol., May 1, 2005; 184(5): 1391 - 1397. [Full Text] [PDF]
|
|
|
|
|
|
M. Wintermark, N. J. Fischbein, W. S. Smith, N. U. Ko, M. Quist, and W. P. Dillon Accuracy of Dynamic Perfusion CT with Deconvolution in Detecting Acute Hemispheric Stroke AJNR Am. J. Neuroradiol., January 1, 2005; 26(1): 104 - 112. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
M. Gyongyosi, C. Strehblow, M. Haumer, P. Wexberg, W. Sperker, S. Lehr, D. Glogar, G. Pasterkamp, and E. Minar Vascular Remodeling in Atherosclerotic Femoral Arteries: Three-dimensional US Analysis Radiology, November 1, 2004; 233(2): 366 - 375. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
M Schmitter, B Kress, S Hahnel, and P Rammelsberg The effect of quality of temporomandibular joint MR images on interrater agreement Dentomaxillofac. Radiol., July 1, 2004; 33(4): 253 - 258. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
H. K. F. Mak, K. K. W. Yau, and B. P. L. Chan Prevalence-adjusted Bias-adjusted {kappa} Values as Additional Indicators to Measure Observer Agreement [letter] Radiology, July 1, 2004; 232(1): 302 - 303. [Full Text] [PDF]
|
|
|
|
 |
|
 Z A Aziz, A U Wells, D M Hansell, G A Bain, S J Copley, S R Desai, S M Ellis, F V Gleeson, S Grubnic, A G Nicholson, et al. HRCT diagnosis of diffuse parenchymal lung disease: inter-observer variation Thorax, June 1, 2004; 59(6): 506 - 511. [Abstract] [Full Text] [PDF]
|
|
Read all eLetters
| HOME | HELP | FEEDBACK | SUBSCRIPTIONS | ARCHIVE | SEARCH | TABLE OF CONTENTS |
| RADIOLOGY | RADIOGRAPHICS | RSNA JOURNALS ONLINE |
