| HOME | HELP | FEEDBACK | SUBSCRIPTIONS | ARCHIVE | SEARCH | TABLE OF CONTENTS |
|
|
|
|||||||
|
|||||||||
Letters to the Editor |
Department of Radiology, University College Hospital, Level 2 Podium, 235 Euston Road, London NW1 2BU, United Kingdom
In the May 2005 issue of Radiology, Dr Park and colleagues (1) investigated the causes of false-negative findings at computed tomographic (CT) colonography. They found that 29 (46%) of 63 polyps were missed by observers and determined by means of logistic regression that flat morphology and small size were the only variables significantly associated with failure to detect polyps (1).
When attempting to determine the true rate of false-negative findings, it is useful to consider how such an observation might arise during a research study of CT colonography. Most obviously, a false-negative observation arises when a lesion is found with the reference test (in this case colonoscopy) but was "missed" with the new technology under investigation (in this case CT colonography). There are a multitude of well-recognized reasons why a polyp present in the colon may be missed at colonography, and researchers in the field are generally well aware of these reasons. For example, factors related to the patients and their preparation, data acquisition, data display, and observer experience and fatigue all play a part. Dr Park and colleagues (1) focused on factors related to patients—specifically, the size and morphology of their polyps.
All of the discussion thus far relates to the situation in which a polyp, for whatever reason, is not seen at CT colonography and, as a consequence, is classified as a false-negative observation when the polyp is detected by using the reference test. However, there are situations in which a polyp that is seen (and reported) with CT colonography may nevertheless still generate a false-negative observation with the reference test. The reasons underpinning this apparent contradiction are much less well appreciated and relate to the methodology that has been generally adopted in comparative studies of CT colonography. They are also well demonstrated in the study by Dr Park and colleagues (1).
Such studies have been focused on per-polyp (rather than per-patient) detection rates. Because patients may have more than one polyp, researchers have, quite understandably, felt the need to adopt prespecified criteria by which individual polyps detected at CT can be matched with those detected at subsequent colonoscopy. These criteria usually relate to size matching and location matching, since polyps can occur at different sites throughout the colon and can be a variety of sizes. For example, Dr Park and colleagues (1) state that when polyps found at CT colonography were compared with those detected at colonoscopy, "the two recorded sizes had to be the same within a 30% margin of error" and that polyps "had to be in the same colonic segment or an adjacent portion of an adjacent segment." This is all well and good if colonoscopy is a robust reference standard where polyp measurement and location are concerned. However, available evidence suggests otherwise. Authors of many studies have found colonoscopy-derived estimates of polyp diameter to be inaccurate (2,3), and location is similarly uncertain: A study that asked for endoscopists' assessments of the location of their instrument tip showed considerable disparity when the true position was revealed by means of electromagnetic imaging (4).
Given the above, consider what happens when CT and colonoscopy depict the same polyp in the same location, but the polyp is measured as 3 mm at colonography and 6 mm at endoscopy. If methodology similar to that of Dr Park and colleagues (1) is used, then the CT finding of a polyp becomes a false-positive observation because it does not correspond to the colonoscopic lesion according to size-matching stipulations. Simultaneously, a false-negative observation is also generated because CT apparently failed to depict the 6-mm polyp found at colonoscopy. The same set of circumstances applies to a polyp when there is discrepancy about location—a false-positive finding and a false-negative finding at CT will arise even if, in reality, CT and colonoscopy have depicted the same polyp.
Because the reference test is not robust with respect to either measurement or location (both areas where it could be argued that CT is superior), we may be unduly penalizing colonography when adopting this methodology for comparative studies. However, a further problem arises because, once having accepted stipulations for size and location matching, authors practically never describe in their results where these stipulations were violated. For example, Dr Park and colleagues (1) do not state if there were any instances where location-matched polyps were incompatible because of size discrepancy (according to their 30% diameter stipulation).
The purpose of this letter is not to criticize Dr Park and colleagues (1); many authors have adopted polyp-matching criteria, including myself. Rather, the aim is to describe less well appreciated ways in which false-negative observations can arise as a result of methodologic stipulations that are seemingly sensible at first inspection. It could be argued that we should completely abandon size and location matching criteria because of the problems described here and, also, because a per-patient analysis is more sensible from a pragmatic perspective (5). However, if matching criteria are adopted for a study, then data relating to false-positive and false-negative observations that are contingent on mismatching need to be reported fully.
 |
References |
|---|
 TOP References References |
|---|
Department of Radiology, University of Ulsan College of Medicine, Asan Medical Center, 388-1 Poongnap-Dong, Songpa-Gu, 138-040 Seoul, Korea
e-mail: seonghopark@paran.com
We thank Professor Halligan for his interest in our article (1). We totally agree with Professor Halligan on the opinion that some false-negative and false-positive results at CT colonography are caused by the imperfectness of the location- and size-matching algorithm that has been adopted by many authors, including us. Nevertheless, abandoning the location- and size-matching algorithm (ie, a per-lesion analysis) and using a per-patient analysis seem to have some problems.
Although colonoscopy may not be a robust reference standard with regard to the locations of colonic polyps, in many cases one can quite clearly determine whether the locations at CT colonography and colonoscopy match. An ambiguous situation occurs when the two locations differ slightly. Considering the limitation of using colonoscopy to determine the lesion location, many radiologists—including us—have used generous matching criteria such as "in the same colonic segment or in adjacent portion of adjacent segments" (1) or "within the same segment or in adjacent segments" (2,3). The limitation of the location-matching algorithm may be reasonably overcome, although not perfectly resolved, by using such generous criteria. We think that the more critical problem is the limitation of the size-matching algorithm, which is relevant to both a per-lesion analysis and a per-patient analysis. It stems from the difference in the measurement accuracy with CT colonography and colonoscopy and the fact that we are measuring a small object where just a millimeter of difference in measurement can have a substantial clinical impact. Another contributing factor to the limitation of the size-matching algorithm is that size measurement, especially measurement with colonoscopy, generally depends on the performer's skill and experience, as well as on the morphology of the lesions. Such limitations may tempt us to give up the per-lesion analysis. Although Professor Halligan stated that "a per-patient analysis is more sensible from a pragmatic perspective," we have a different opinion. If we were to take CT colonography as a tool in the screening of colon cancer to provide the binary qualitative information on whether a patient has a clinically important polyp or cancer, the same way in which the results of stool occult-blood tests are given, a per-patient analysis is appropriate. However, the role of CT colonography is not only to provide information on the presence or absence of colonic lesions but also to provide the information on the locations of the lesions, which will be very useful in many clinical situations. For example, if CT colonography can be used to accurately tell that lesions are present only in the rectosigmoid area, we can confidently recommend sigmoidoscopy as the appropriate next step rather than colonoscopic examination of the whole colon. Therefore, the accuracy of CT colonography in determining lesion location should also be addressed by using a per-lesion analysis in the evaluation of the performance. Given the limitations of the location- and size-matching algorithms, which Professor Halligan has clearly described, we agree that the effect of the matching algorithm should be considered in the evaluation of the performance of CT colonography. It would be interesting to investigate how different matching algorithms affect the diagnostic performance of CT colonography to determine the most appropriate matching algorithm both practically and scientifically.
|
References |
|---|
|
TOPReferencesReferences |
|---|
This article has been cited by other articles:
|
|
 |
|
  A. H. Dachman, K. B. Kelly, M. P. Zintsmaster, R. Rana, S. Khankari, J. D. Novak, A. N. Ali, A. Qalbani, and J. G. Fletcher Formative Evaluation of Standardized Training for CT Colonographic Image Interpretation by Novice Readers Radiology, October 1, 2008; 249(1): 167 - 177. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
S. H. Park, E. K. Choi, S. S. Lee, J.-S. Byeon, J.-Y. Jo, Y. H. Kim, K. H. Lee, H. K. Ha, and J. K. Han Polyp Measurement Reliability, Accuracy, and Discrepancy: Optical Colonoscopy versus CT Colonography with Pig Colonic Specimens Radiology, July 1, 2007; 244(1): 157 - 164. [Abstract] [Full Text] [PDF]
|
|
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| HOME | HELP | FEEDBACK | SUBSCRIPTIONS | ARCHIVE | SEARCH | TABLE OF CONTENTS |
| RADIOLOGY | RADIOGRAPHICS | RSNA JOURNALS ONLINE |
