| HOME | HELP | FEEDBACK | SUBSCRIPTIONS | ARCHIVE | SEARCH | TABLE OF CONTENTS |
|
|
|
|||||||
|
|||||||||
Editorial |
1 From the Department of Radiology, University of Arizona School of Medicine, 1609 N Warren, Bldg 211, Rm 112, PO Box 245067, Tucson, AZ 85724. Received November 19, 2003; accepted November 21. Address correspondence to the author (e-mail: krupinski@radiology.arizona.edu).
Index terms: Breast neoplasms, diagnosis, 00.30 • Cancer screening • Computers, diagnostic aid • Diagnostic radiology, observer performance • Editorials
There is little doubt that computer-aided detection (CAD) and computer-aided diagnosis schemes will soon be everyday tools in the radiologist’s arsenal of digital image processing and interpretation techniques. In general, radiologic image perception tasks are difficult. Interpretation varies because the images are two-dimensional representations in which three-dimensional solid structures are made transparent. This requires an understanding of projective geometry to determine true depth, location, and feature characterization of imaged structures. The perceptual task is complicated even further by the fact that lesions are typically embedded in a background of anatomic noise, thus making detection, discrimination, and integration of distinct lesion features difficult (1). Reader experience and the fact that visual search is not systematic but rather unique to each observer also contribute to the relatively high rates of interobserver and intraobserver variation in radiologic image interpretation.
The ultimate goal of use of CAD and other medical image processing techniques is therefore to improve radiologic interpretation by making lesions easier to detect and classify and to potentially identify lesions at an earlier stage so that treatment can be more effective (2). CAD will certainly prove to be useful in many image interpretation situations, but the most commonly discussed are those in which the radiologist’s perceptual and cognitive systems are put to the test: when lesion incidence is low (as in lung cancer screening), error rates are high (as in mammography), and image volume is high (as in chest computed tomography [CT]). All of these situations also tend to foster high interobserver and intraobserver variation, so a secondary goal of CAD use is to reduce observer variation of both kinds (3).
Although the most familiar CAD applications are the detection of pulmonary nodules on chest images (ie, radiographs and CT scans) and detection of masses and microcalcifications on breast images (ie, mammographic, ultrasonographic, and magnetic resonance images), there is a substantial amount of work being performed with a wide variety of applications and images obtained with all possible modalities (eg, CT colonography) (4–6).
However, up until about 10 years ago, CAD use and evaluation were limited to the laboratory setting, and most researchers reported the sensitivity and specificity of the various schemes independent of the performance of the radiologist. The goal was to achieve a high enough true-positive rate without increasing the false-positive rate.
Studies then shifted to comparisons of CAD performance with that of the radiologist, with mixed results. In some cases, performance of CAD was better than that of the radiologist, and in other cases, performance of CAD was worse; in still other cases, performance of CAD was about the same as that of the radiologist. Subsequently, the ultimate tests were comparisons of performance of the radiologist without and then with the addition of CAD input. Again, the results have been mixed in terms of whether or not CAD input improves the radiologist’s performance, although results in most studies suggest that even the most experienced readers may benefit on occasion from having CAD input during the image interpretation process (7–9).
Most of these studies, however, have been based in the laboratory, where experimental conditions can be well controlled. The cases, radiologists, and reading environment can be carefully selected and optimized. Although such studies are necessary to demonstrate the utility of CAD, they may not always reflect how well the CAD scheme will work in the true clinical environment, where conditions, readers, and cases are not especially well controlled. The article by Helvie et al (10) in this issue of Radiology is one of the few studies in which the authors report the use and evaluation of CAD in a true clinical reading environment. As such, this article illustrates a number of very interesting points.
The first point is that in the study a noncommercial CAD system was used with performance at a very high sensitivity of 91% and fairly low false-positive rates (maximum of three marks for masses marked per image and of one mark for microcalcification clusters marked, on average, per image). The fact that it was a noncommercial CAD system is not remarkable or even that important. What is important is that the study of Helvie et al demonstrates that there are approaches to CAD development and implementation, other than those that have been commercialized and approved by the Food and Drug Administration, that can be used to achieve performance that is at least as good as that achieved with commercial systems.
At the fundamental level, most CAD schemes do essentially the same types of basic operations: image segmentation, followed by feature extraction, and then application of various rules for discriminant classification of the features for identification as normal versus lesion, benign versus malignant, and so on. The ways to design and implement these operations, however, are numerous and varied. Just because a system has been commercialized does not imply that it carries out the selected detection and/or classification task in the optimal manner. Other approaches and techniques, such as the ones used in the CAD scheme by Helvie et al (10), may be better than those that have been commercialized, or some unique combination of independent approaches may be worth investigating to develop an even more powerful CAD tool.
With this study, a second important point is that in the clinical setting it may be very difficult to truly demonstrate the actual benefits of CAD for either an individual radiologist or the population of radiologists as a whole. The problem relates to disease incidence in the types of screening situations for which CAD is designed. In the study of Helvie et al (10), there were 2,389 subjects, and only 11 (0.46%) of them had mammographically detected nonpalpable cancers. CAD detected 10 (91%) of the cancers, and the radiologists detected 10 (the missed lesion was different for CAD and the radiologists). From a statistical point of view, one would hardly consider a sample size of 11 cases sufficient to yield enough statistical power to draw reliable or valid conclusions.
However, the CAD scheme had been tested previously in laboratory settings with case distributions that did have sufficient statistical power (ie, lesion incidence was much higher than in the clinical screening situation) (10). In the clinical environment, performance of CAD was actually better than was that of the laboratory tests in terms of sensitivity. In the clinical environment, it would take a huge number of patients and an extended period of time to have enough cancers for CAD to detect to yield sufficient statistical power. Yet, if one multiplies that one cancer marked by CAD that the radiologist missed by the millions of women who undergo screening annually, the potential for CAD to detect missed lesions and truly to influence patient care is clear.
It is also clear from the study of Helvie et al (10) that there is a moderately strong caveat that comes with the clinical use of CAD. It is important for every radiologist who uses a CAD scheme clinically to remember that CAD schemes do miss lesions. Sensitivity is rarely 100%, especially because of the wide variability in lesion and background appearance. It may be the case that the majority of CAD schemes may never be trained with enough cases to "see" all possible variations of a given target lesion. Even for those schemes that use artificial neural networks and continue to learn with each successive case they analyze, sensitivity of 100% may not be achieved.
In the study of Helvie et al (10), one lesion was missed by the CAD scheme that was apparently detected by the radiologist who reviewed the images in that case. However, if radiologists begin to rely too heavily on CAD prompts to detect and classify lesions, they may alter their normal search and decision-making processes (11). Instead of performing thorough searches of each image and actively considering each location as an area that is suspected of being malignant before receiving the CAD input, radiologists may develop the habit of doing only cursory searches without trying to render a decision with the information they gather themselves through independent visual search. An inadequate search combined with a less than 100% sensitivity for the CAD scheme could result in missed lesions that would otherwise have been detected.
Another important point may be considered in regard to the biopsy and recall results. In the study of Helvie et al (10), the biopsy rate increased by 8% and the callback rate increased by 1.4% with CAD. This was balanced out to some extent by a 9% overall improvement in cancer detection. The point is that CAD use in the clinic is bound to change the decision-making process; that is precisely what it is designed for. Ideally, one would like those changes to be only positive, but realistically this is unlikely to happen. CAD is simply a tool to help the radiologist make better diagnostic decisions. It is not perfect.
Even though CAD provides more information to radiologists than they likely gather on their own, some of the information may be flawed or incomplete (ie, there are false-positive marks and there are false-negative marks or missed lesions). The radiologist’s decision that is based on this flawed or incomplete information thus has the potential to be flawed as well. The use of CAD in the clinical environment involves a learning curve just like any other new task. The radiologist has to learn how reliable the CAD scheme is and to learn to recognize when CAD makes mistakes and, perhaps more importantly, why CAD makes mistakes. However, since each case and each lesion is unique, it may not be possible to recognize consistently all types of mistakes, even after prolonged experience.
Radiologists need to learn how to integrate CAD information into their own unique decision-making processes, thus balancing that information with the information they collect through their own visual search of the image. It is possible that if the radiologists in the study of Helvie et al had more experience with the CAD scheme and if they had seen more examples of the true- and false-positive marks, they would have learned to adjust their biopsy and recall rates to more closely match baseline levels.
At some point in the future, it is likely that CAD will be available for all types of images and lesions. Instead of having a single CAD scheme for a single specific lesion, the radiologist will be able to view any type of image and activate an "all-purpose" CAD scheme that will search the image for all possible types of lesions that can be found for that type of image. The CAD scheme will point out lesions, provide a classification and a probability of malignancy (computer-aided diagnosis), point out possible traces of the lesion on images obtained at previous examinations or track progress of the lesion over time, and even provide examples of similar lesions from other cases (12).
In the study of Helvie et al (10), the CAD scheme had marked within 2 cm the site of two (40%) of five cancers that developed within the next year. The five locations were not reported by the radiologists, even though CAD had marked them. This brings up an interesting conundrum. If a CAD scheme can detect enough features of an early or subtle lesion to put a mark on the image, but the radiologist cannot perceive those features and thus interprets the mark as false-positive, is CAD really being used to its fullest potential? In retrospect, the five lesions were deemed nonactionable findings, but this will not always be the case.
If the early or subtle lesion can somehow be made more perceptible to the radiologist, suitable actions may be initiated. Perhaps this is where the future of CAD development lies. Instead of simply putting a mark on potential lesion locations and possibly providing a probability score for benign-versus-malignant status, the CAD schemes will also perform tailored image processing to enhance the visibility of the lesion for the radiologist so that it can be readily perceived and interpreted. CAD use will become a much more interactive process than it is in its current form as a prompting device. Perhaps the radiologist will also be able to provide information for input to the CAD system that he or she has gathered from years of viewing tens of thousands of images and lesions in order to refine the CAD output even further.
FOOTNOTES
See also the article by Helvie et al in this issue.
REFERENCES
Related Article
This article has been cited by other articles:
|
|
 |
|
  M. Gromet Comparison of Computer-Aided Detection to Double Reading of Screening Mammograms: Review of 231,221 Mammograms Am. J. Roentgenol., April 1, 2008; 190(4): 854 - 859. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 S. D. O'Connor, J. Yao, and R. M. Summers Lytic Metastases in Thoracolumbar Spine: Computer-aided Detection at CT--Preliminary Study Radiology, March 1, 2007; 242(3): 811 - 816. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
F. J. Gilbert, S. M. Astley, M. A. McGee, M. G. C. Gillan, C. R. M. Boggis, P. M. Griffiths, and S. W. Duffy Single Reading with Computer-aided Detection and Double Reading of Screening Mammograms in the United Kingdom National Breast Screening Program Radiology, October 1, 2006; 241(1): 47 - 53. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
E. N. C. Milne, E. A. Krupinski, M. A. Helvie, L. Hadjiiski, and H.-P. Chan Computer-aided Detection of Breast Cancer * Dr Krupinski responds: * Dr Helvie and colleagues respond: Radiology, November 1, 2004; 233(2): 615 - 617. [Full Text] [PDF]
|
|
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| HOME | HELP | FEEDBACK | SUBSCRIPTIONS | ARCHIVE | SEARCH | TABLE OF CONTENTS |
| RADIOLOGY | RADIOGRAPHICS | RSNA JOURNALS ONLINE |
