| HOME | HELP | FEEDBACK | SUBSCRIPTIONS | ARCHIVE | SEARCH | TABLE OF CONTENTS |
|
|
|
|||||||
|
|||||||||
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Breast Imaging |
1 From the Dept of Radiology (W.A.B.) and Greenebaum Cancer Ctr (W.A.B.), Univ of Maryland, 419 W Redwood St, Ste 110, Baltimore, MD 21201; Dept of Radiology, Univ of Massachusetts, Worcester, Mass (C.J.D.); Dept of Radiology, Indiana Univ School of Medicine, Indianapolis (V.P.J.); Dept of Radiological Sciences, UCLA School of Medicine, Los Angeles, Calif (L.W.B.); Dept of Radiology, Medical College of Wisconsin, Milwaukee (C.A.B.); and American College of Radiology, Reston, Va (R.S.L., P.E.C.). From the 2000 RSNA scientific assembly. Received Oct 5, 2001; revision requested Nov 1; revision received Feb 6, 2002; accepted Feb 28. Supported by the Maryland Chapter of the Susan G. Komen Breast Cancer Foundation and the American College of Radiology Technology Assessment Studies Assistance Program. Address correspondence to W.A.B. (e-mail: waberg@umaryland.edu).
 |
ABSTRACT |
|---|
 TOP ABSTRACT INTRODUCTIONMATERIALS AND METHODS RESULTS DISCUSSIONREFERENCES |
|---|
MATERIALS AND METHODS: A test set of mammograms was developed, with 54 proven lesions consisting of 28 masses (nine [32%] malignancies) and 26 microcalcifications (10 [38%] malignancies). Three experienced breast imagers reviewed cases independently and by means of consensus. Twenty-three practicing mammogram-interpreting physicians reviewed mammograms before and after a day’s lectures on BI-RADS. Observer performance before and after training was measured by means of agreement (
) with consensus description and assessments, rate of biopsy of malignant and benign lesions, and areas under receiver operating characteristic (ROC) curves. Performance was also measured for 11 participants 2–3 months after training.
RESULTS: Improved agreement with consensus feature analysis was found for mass margins and/or asymmetries, with a pretraining generalized value of 0.36 and a posttraining generalized value of 0.41. Similar improvement was seen for description of calcification morphology (pretraining value of 0.36 improving to 0.44 after training). No improvement was seen in describing calcification distribution. Final assessments were more consistent after training, with a pretraining value of 0.31, as compared with 0.45 after training. The mean biopsy rate for malignant lesions improved from 73% (range, 53%–89%) before training to 88% (range, 74%–100%) after training, with minimal increase in mean biopsy rate of benign lesions (43% [range, 26%–60%] before to 51% [range, 31%–63%] after training), and no net change in area under the ROC curve, as compared with histopathologic findings. For the subset of participants with delayed follow-up, no significant decline in posttraining results was seen.
CONCLUSION: BI-RADS training resulted in improved agreement with the consensus of experienced breast imagers for feature analysis and final assessment. It is important that trainees showed improved rates of recommending biopsy for malignant lesions. This effect was maintained over 2–3 months.
© RSNA, 2002
Index terms: Breast radiography • Diagnostic radiology, observer performance
|
INTRODUCTION |
|---|
|
TOPABSTRACTINTRODUCTIONMATERIALS AND METHODS RESULTS DISCUSSIONREFERENCES |
|---|
The Breast Imaging Reporting and Data System (BI-RADS) (3) was developed through the American College of Radiology, or ACR, to help standardize feature analysis and final management of mammographic findings. It was also hoped that a large national database of outcomes would ensue that would provide greater rationale and validation for management when certain features or combinations of features were present. Indeed, the National Mammography Database is currently set up through the ACR to receive such information (4).
The purpose of our study was to determine whether training in BI-RADS feature analysis would improve observer agreement with experienced breast imagers in mammographic lesion description and final assessments (management recommendations). We further sought to assess whether such training would improve the biopsy rate of malignant lesions, without loss of specificity.
|
MATERIALS AND METHODS |
|---|
|
TOPABSTRACTINTRODUCTIONMATERIALS AND METHODS RESULTS DISCUSSIONREFERENCES |
|---|
Photographic slides were prepared that were centered on the area of interest (indicated with arrows when needed), with the entire mammographic image also provided when a broader view was needed (eg, asymmetric densities and diffuse and/or bilateral findings). The University of Maryland institutional review board, with which we consulted, did not require its approval or informed consent for this study, as all patient identifiers were removed from the images and database. Magnification views were provided for 35 cases. When magnification views were not available, collimated compression views were used (n = 11), or the lesion was marked on the slides after photographic magnification (n = 8). Lesions were selected to be pure examples of individual features of the BI-RADS lexicon in that the 28 masses and/or asymmetric densities did not have associated calcifications, and the 26 lesions manifest as calcifications were chosen to be of one main morphologic type, without associated density or mass. The case mix was designed to have at least three lesions from each of the major lesion descriptor categories.
For masses, lesion descriptors supplied included the following: lymph node, circumscribed round or oval, circumscribed lobulated, obscured, microlobulated, indistinct, or spiculated mass; or focal asymmetric density. Shape and margins were combined to simplify the number of variables evaluated. While circumscribed round or oval and circumscribed lobulated descriptors were initially separated, combining them (as "circumscribed") did not substantially alter any agreement or outcome analyses. To the greatest extent possible, descriptors were listed in order of increasing risk of malignancy (3,5–8). No logical ordering could be inferred for asymmetric density, and it was arbitrarily placed last on the list; participants were so instructed.
For calcification morphology, descriptors included the following: typically benign, punctate, amorphous and/or indistinct, pleomorphic, and fine linear and/or branching. Participants were instructed that the typically benign category included coarse, secretory, oil cyst, milk of calcium, popcorn, vascular, and dermal calcifications. For calcification distribution, descriptors available were the following: diffuse, clustered, regional, multiple groups, linear, and segmental. Participants were instructed that segmental meant one or multiple ducts and corresponding branches. Multiple groups described the distribution of several groupings of similar morphology (3,7). Final assessments were category 2, benign; 3, probably benign; 4, suspicious abnormality; and 5, highly suggestive of malignancy. While BI-RADS category 5 is intended for lesions that may proceed directly to definitive surgery without initial tissue diagnosis, both categories 4 and 5 indicate the need for biopsy, which is the end point of interest (9). Categories 4 and 5 were combined for evaluation of interrater agreement.
Three experienced breast imagers (C.J.D., V.P.J., L.W.B.) participated in the training. Each of these individuals interprets at least 4,000 mammograms per year and had at least 17 years of experience (range, 17–30 years) in mammography at the time this article was written. These experienced imagers were shown the quiz cases on slides the evening before the course (see below) and were asked to independently record their responses on worksheets. The cases were then reviewed and consensus opinion achieved as to both appropriate feature description and final assessment.
Twenty-seven radiologists participated in a day-long course, "Applying BI-RADS to Mammography Practice," in Baltimore, Md, in September 1999. The training and experience of the participants was recorded, including fellowship training in mammography, the number of mammograms reviewed per week, and the number of years in practice. Complete data sets for lesion description were obtained from 23 participants. Audience response keypads (Audience Response Systems, Evanston, Ill) allowed immediate data entry, with questions repeated when incomplete participation occurred. Individual participants were given unique identifiers required for keypad entry. Slides showing the previously mentioned quiz cases were presented, immediately followed by questions where participants were asked to provide a description and a BI-RADS final assessment with its linked recommendation (benign, routine screening; probably benign, 6-month follow-up; suspicious, biopsy; or highly suggestive of malignancy, biopsy) for each lesion. Quiz cases were shown at the start of the day and again after a day’s lectures in BI-RADS. The lectures consisted of 3
hours of didactic material, including mammographic and pathologic correlation of individual BI-RADS features for calcifications, masses, and asymmetric densities, and another half-hour of questions and answers. Another 2 hours were devoted to examples, with audience response for both feature analysis using BI-RADS terms and final assessments, with time for discussion. There was no overlap of case material presented in the didactic sessions with that in the quiz. For the quiz, participants were asked to assume that the findings were noted on the baseline screening mammogram and that the patient had no known risk factors for breast cancer. Participants were asked to use the single most worrisome applicable feature descriptor in each category. During the afternoon quiz, after participants recorded their description and final assessment for a particular case and prior to the next case, they were immediately given the consensus opinion of the experienced breast imagers with regard to both feature analysis and final assessment for that case. Twenty-three participants completed data sets on lesion description, and 21 participants completed data sets on management for all 54 lesions before and after training. A subgroup of 11 observers was shown the quiz cases again in random order 2–3 months after completing the training course. Immediate consensus feedback was given after each case in the delayed follow-up evaluation.
Control Group
A group of six senior radiology residents who had not participated in the course were shown the quiz slides on two occasions separated by 1 week, with the consensus of the experienced breast imagers supplied after each case after recording of their impressions on the second occasion. The residents were instructed to not read any mammography-related materials during the intervening week. All residents had completed at least 1 month of training in mammography.
Measures of Performance
Observer performance was considered in terms of agreement with the consensus opinion and, separately, agreement with histopathologic truth. A final assessment of BI-RADS category 4, suspicious abnormality, was considered as agreement with an assessment of category 5, highly suggestive of malignancy. values were used as measures of agreement between each participant and the consensus beyond that expected by chance (10,11). Fleiss (11) has suggested that > 0.75 indicates excellent agreement; 0.40 
0.75, fair to good agreement; and < 0.40, poor agreement. Multireader generalized values (10,12) were used to describe agreement among participant and expert readers for each feature descriptor. This statistic measures the degree to which interpretation variability arises from differences between cases relative to differences among readers interpreting the same case. It is analogous to the intraclass correlation used in assessment of laboratory measurements. Positive values indicate that interpretation variability is governed more by case-related differences than by between-reader disagreements. Negative values indicate the opposite. A zero value occurs whenever the two sources of variability are equal. We report generalized values from the individual consensus ratings as an agreement benchmark.
Accuracy was estimated by calculating sensitivity, false-positive rate (1 - specificity), and area under the receiver operating characteristic (ROC) curve (Az) (13). Sensitivity for malignancy was calculated as the proportion of malignancies given final assessments of BI-RADS category 4, suspicious abnormality, or category 5, highly suggestive of malignancy. Similarly, benign lesions so categorized were considered false-positive findings. A separate analysis considered the consensus assessment as the reference standard: In addition to known malignancies, some benign lesions are appropriately recommended for biopsy. Az in the latter analysis enabled measurement of concordance between reader classification and expert consensus.
Training effectiveness was measured by means of the paired difference between post- and pretraining values (individual reader vs consensus), sensitivity, false-positive rates, and Az value. To identify statistically significant improvements in agreement (), sensitivity, false-positive rate, and Az value, lower 95% CIs were used, analogous to a one-tailed hypothesis test with a 5% significance level. If the lower CI bounds contained zero, we concluded that there was no statistically significant improvement. If the lower CI bounds did not contain zero, we concluded that there was a statistically significant improvement. The latter is reported as a P value less than or equal to .05. Multiple regression analysis was used to model the effect of individual reader characteristics on (a) pretraining Az value and (b) relative change in Az value after training. For the regression model variables, a two-tailed 95% CI was used, since there was no a priori hypothesis regarding the direction of the parametric relationships. Performance measures were assessed as a function of experience in breast imaging.
Lower bounds and two-tailed CIs of paired differences (before vs after training) were estimated by using jackknifed standard errors to provide the appropriate inference given the correlated nature of the data. Caution is recommended in relying on estimates of statistical significance and use of CIs with these data. Our distribution of images is not representative of that in clinical practice. The lesions used in the current study include an enriched malignant case mix along with a deliberate selection of lesions close to the threshold for intervention. In addition to limited generalizability, this lesion selection can be expected to result in greater variance in assessments among both lesions and study participants. This is especially true for value estimates, since both paired and generalized values can vary dramatically, depending on the distribution of participant interpretations. All analyses were conducted by using SAS software (SAS Institute, Cary, NC).
|
RESULTS |
|---|
|
TOPABSTRACTINTRODUCTIONMATERIALS AND METHODS RESULTS DISCUSSIONREFERENCES |
|---|
|
value of 0.57 (range, 0.43–0.66) for mass margins and asymmetries. The least agreement was seen for use of the term obscured, although the value was still 0.43, indicating fair to good agreement. For participants before training, the generalized value for all mass descriptors was 0.36 (range, 0.07–0.61), with the least agreement seen for lymph node ( = 0.07; Table 2) and indistinct masses ( = 0.24), which were more likely to be considered focal asymmetric densities by another reader. After 1 day of training, the mean generalized value improved to 0.41 (range, 0.06–0.59) (mean change, 0.05), with the least agreement seen for lymph node masses ( = 0.06). Focal asymmetric density showed the greatest improvement in agreement with training. Of the 23 participants evaluated, 12 (52%) showed improvement after training in agreement with consensus description of mass margins and asymmetries; six (26%) showed no change; and five (22%) showed less agreement with the consensus after training.
|
value among the experienced breast imagers was 0.44 (range, 0.06–0.75 for individual descriptors; Table 1). The greatest disagreement among experienced breast imagers was seen for punctate morphology ( = 0.06); such calcifications were more likely described as pleomorphic or amorphous by another experienced breast imager. The generalized value for participants for description of microcalcification morphology prior to training was 0.36 (range, 0.17–0.65), with the greatest disagreement for pleomorphic calcifications ( = 0.17, Table 2). After training, the generalized value for participants improved to 0.44 (range, 0.31–0.60; mean change, 0.08), and the term pleomorphic remained the most variably used ( = 0.31). With training, 11 (48%) of the 23 participants showed improved agreement with the consensus on description of calcification morphology, nine (39%) showed no change, and three (13%) showed less agreement.
The generalized value for experienced breast imagers for all categories of microcalcification distribution was 0.44 (range, -0.03 to 0.62). Use of descriptors diffuse and/or scattered and multiple groups showed more disagreement than expected by chance (Table 1). Prior to training, the generalized value for participants was 0.35 (range, 0.10–0.50), with segmental and regional showing poor agreement with the consensus ( = 0.10 and 0.11, respectively; Table 2). With training, the generalized value improved to 0.36 (range, 0.17–0.52; change, 0.01), and descriptors segmental and regional remained the most problematic ( = 0.17 for each).
The generalized value for the three experienced breast imagers for final assessment BI-RADS category 2 (benign) was 0.51; for category 3 (probably benign), 0.04; and for categories 4 and 5 combined (suspicious abnormality and highly suggestive of malignancy, respectively), 0.50. Participants showed improved agreement with the consensus after training in each of the final assessment groupings (Table 2). The overall generalized value improved from 0.31 before training to 0.45 after training (change, 0.15). With training, 15 (65%) of the 23 (65%) participants showed improved agreement; five (22%), no change; and three (13%), decreased agreement with the consensus.
Accuracy Measures
While individual experienced imagers recommended biopsy for 15–17 (79%– 89%) of the 19 cancers in the quiz set, the group consensus was to perform biopsy on all malignancies (sensitivity, 100%). Similarly, individual experienced imagers recommended biopsy for 10–18 (29%–51%) of the 35 benign lesions, and the consensus was to perform biopsy in 15 (43%) of the 35.
When using the consensus assessments as the reference standard, we found significant improvement in the biopsy rate of malignant lesions after training (Table 3). Prior to training, for the group of 21 participants with complete data sets both before and after training, biopsy rates for malignant lesions were 50%–91% (mean, 71%; median, 71%). After training, biopsy rates for malignant lesions improved to 71%–97% (mean, 86%; median, 88%) (P < .05). Improvement was 3%–32% (mean, 16%; median, 14%). Prior to training, participants’ biopsy rates of lesions judged benign or probably benign (false-positive rate) by means of consensus of the experienced imagers were 10%–45% (mean, 25%; median, 25%). It is important to note that after training, no significant change was seen in biopsy rates for benign lesions, which were 10%–40% (mean, 26%; median, 25%) after training, with a mean change of 0.01. Az values showed significant improvement, with a mean change of 0.04 (P < .05) (Fig 1, Table 3), as compared with the consensus reference standard. Of the 21 participants evaluable, 17 (81%) showed improvement, and two (10%) showed no change. The two (10%) observers who showed a decrease in Az value were the two with the highest initial pretraining Az values. By using histopathologic truth as the reference standard, a significant increase was seen in both sensitivity and biopsy rate of benign lesions (false-positive rate), with no significant change in Az value (Table 3). In this analysis, participants remained clustered around an Az value of 0.71 after training, which is less than the prestudy experienced breast imager consensus Az value of 0.81.
|
|
Training Retention
When a subgroup of 11 participants was shown the quiz cases again in random order 2–3 months after training, improvement in value, as compared with pretraining values, was retained for nine (82%) participants with regard to mass and/or asymmetry description. The value was 0.53 (as compared with the subgroup’s pretraining value of 0.35 and immediate posttraining value of 0.43, P = not significant; Table 4). Improved values were maintained for eight (73%) of the 11 participants with regard to calcification morphology, as compared with pretraining values. Less success was maintained for calcification distribution, with four (36%) of the 11 observers showing persistent improved agreement with the consensus. For final assessments, nine (82%) of the 11 participants retained significant improvement, as compared with pretraining values, although all but two participants showed slight decreases in values, as compared with immediate posttraining values. For the subset of participants with delayed follow-up, the mean value for final assessment was 0.37 before training, 0.53 immediately after training, and 0.49 at delayed follow-up (Table 4). The changes in mean value at delayed follow-up, as compared with those immediately after training, were not significant for any category. These 11 readers showed increases in agreement immediately after training for mass margins and/or asymmetries, microcalcification morphology, and final assessment, and the results shown in Table 4 suggest that this improvement was retained.
|
|
Cases with Greatest Variance
Calcifications proved more problematic than masses in achievement of consistency in management. The experienced breast imagers all recommended biopsy for all nine malignant masses, whereas at least one experienced imager rated four (40%) of the 10 malignant calcifications as benign or probably benign. When rank ordered according to variance across participants after training, of the 20 cases with the least variance, only six (30%) were calcifications, whereas 15 (75%) of the 20 cases with the greatest variance were calcifications. Three cancers were among the 20 with the greatest variance, including two clusters of pleomorphic calcifications (Figs 2, 3) and one obscured mass (Fig 4). There were two cases with no variance (spiculated masses) prior to training. After training, there were 13 cases with no variance (four spiculated masses, three indistinctly marginated masses, one obscured and one microlobulated mass each, two pleomorphic calcifications, and two fine linear and/or branching calcifications). Five of 10 cases with greatest variance after training were typically benign calcifications, according to the consensus of the experienced imagers. Participants had particular difficulty recognizing secretory calcifications as benign.
|
|
|
|
|
|
|
DISCUSSION |
|---|
|
TOPABSTRACTINTRODUCTIONMATERIALS AND METHODS RESULTS DISCUSSIONREFERENCES |
|---|
Linver et al (22) have shown that dedicated training in mammography through continuing medical education courses improves the performance of radiologists both by increasing sensitivity and by decreasing the size of cancers detected, without a loss of specificity. Pure experience in reviewing mammograms has been shown to contribute to improved performance, with the best accuracies seen after cumulative experience of reviewing 10,000 mammograms (23). Kan et al (24) have shown improved reader performance when a minimum of 2,500 mammograms are interpreted per year. Ciatto et al (25), in a study of 117 Italian radiologists, showed increasing accuracy in a mammographic proficiency test correlated with increasing number of years in practice, number of mammograms reviewed, and total number of mammograms reviewed per year. Indeed, many European countries require that 5,000 mammograms be interpreted annually to maintain qualifications as a screening reader (26). We did not demonstrate significant differences in Az values with varying experience levels, although we showed that readers performing worse initially were more likely to benefit from training.
Improvement of reader classification of mammographic findings has received relatively little attention. D’Orsi et al (27) have demonstrated that mammographic interpretation was more accurate when features were entered into a computer and the risk of malignancy was provided to the reader. Recently, Chan et al (28) showed improved Az values when lesion features were entered into artificial neural networks and compared with a database of known benign and malignant lesions, with an output risk of malignancy provided to the radiologist. Performance is further enhanced when patient risk factors are also considered (29,30).
Wherever possible, we listed descriptors in order of increasing risk of malignancy. This parallels the presentation of the terminology in the third edition of the BI-RADS lexicon (3). This may have introduced bias in linking final assessments with feature analysis. We were trying to teach the relationship between the two, and such bias, if any, was considered desirable. It is not clear that this actually biased the results. For example, we were surprised to note that 15% of calcifications described as "coarse, typically benign" were still assessed as suspicious or highly suggestive of malignancy, with biopsy recommended.
Use of the BI-RADS lexicon in practice has not yet, to our knowledge, been optimized. Until 1998, with publication of the Illustrated BI-RADS (3), little illustration was available for the average practitioner to master the use of standardized terms for feature analysis. Baker et al (31) had five radiologists, who came from the same practice and trained in the same program, apply the BI-RADS terminology to 60 proven cases. values were used as measures of agreement, with values of 0.63 for mass margins, 0.50 for microcalcification morphology, and 0.77 for microcalcification distribution (31). Berg et al (32) evaluated the use of BI-RADS by five separately trained radiologists reviewing 103 proven cases and determined values of 0.40 for mass margins, 0.36 for microcalcification morphology, 0.47 for microcalcification distribution, and 0.37 for final assessment. Particular terms were more variably used, especially obscured ( = 0.1), amorphous ( = 0.25), and diffuse ( = 0.08) descriptors and the probably benign and suspicious assessment categories ( = 0.17 and 0.16, respectively) (32). This suggests that lesions near the threshold for biopsy were the most problematic and intuitively likely accounted for the greatest variability in actual practice. Indeed, D’Orsi and Swets (33), in analyzing the results of Elmore et al (1), have shown that the 10 radiologists were in fact operating on the same ROC curve but used different thresholds for intervention. On the basis of these data, our case mix deliberately included lesions close to the threshold for intervention (ie, punctate, amorphous, and pleomorphic calcifications, microlobulated and indistinct masses, and focal asymmetric densities), to attempt to clarify the areas of greatest confusion in practice.
As has been observed previously (32), we found lower pretraining values for such intermediately suspicious features than for those more typically benign (ie, circumscribed) or highly suspicious (ie, spiculated masses and linear or branching calcifications). It is important to note that we were able to improve the consistency of lesion description for most of these intermediate-suspicion features, including microlobulated and indistinct mass margins, focal asymmetric densities, and amorphous and pleomorphic calcifications.
Punctate (round, <0.5-mm) calcifications remain problematic and were, indeed, problematic for our experienced breast imagers to consistently describe, with a generalized value of only 0.06 for the experts. While punctate calcifications have been variably included in the probably benign classification (5), it is clear that even punctate calcifications in a linear or segmental distribution are suggestive of cancer. Indeed, in an abstract by Arnoldus and Berg (34), of 11 foci of punctate calcifications on which biopsy was to be performed because of suspicious distribution or increase or because of a strong family history of breast cancer, three (27%) proved malignant. Similarly, in the study by Liberman et al (8), one (11%) of nine clustered punctate calcifications on which biopsy was to be performed proved malignant. While training participants to consistently describe and manage punctate calcifications may remain problematic, a majority of isolated, scattered, and diffuse bilateral, punctate, and round calcifications are indeed benign and can be dismissed (35).
What may be more problematic is inconsistent recognition of a segmental (duct and its branches) distribution of calcifications. Liberman et al (8) found that 17 (74%) of 23 such calcifications on which biopsy was to be performed were malignant, similar to the three (75%) of four malignancy rate of these lesions in this study’s case mix. While our experienced breast imagers were consistent in their use of the term segmental ( = 0.62), participants were not, with a pretraining value of 0.10 and a posttraining value of 0.17. We found that participants tended to consider secretory calcifications to be linear and branching in a segmental distribution and to be suspicious, although they were instructed to consider them to be typically benign. Training in calcification distribution was less successful than in the other features analyzed.
We demonstrated an improvement in lesion management that was maintained at delayed follow-up. It is important to note that the sensitivity of participants improved without a significant increase in false-positive results, as measured against the consensus of the experienced breast imagers. Indeed, a majority of participants achieved performance similar to that of the experts after training.
Whether or not such a quiz set of cases can be used to accurately measure performance in practice has been questioned. Rutter and Taplin (36) recently found that actual clinical practice exceeded test performance. Inclusion of a large number of malignancies in the test set introduces "context bias" that tends to increase sensitivity and decrease specificity (37). Nevertheless, we were most concerned with demonstrating that we could achieve improved agreement with consensus of experienced breast imagers rather than with any absolute measure of performance. This was accomplished.
In summary, training in BI-RADS feature analysis and assessment resulted in improved consistency in lesion description, particularly for mass margins and calcification morphology, but in less improvement for calcification distribution; with this training, participants uniformly improved in their recognition of suspicious features and recommended biopsy on more cancers. The benefits of such training were retained after 2–3 months in a subgroup of participants.
|
ACKNOWLEDGMENTS |
|---|
|
FOOTNOTES |
|---|
3 Current address: Moffitt Cancer Center, University of South Florida, Tampa.
4 Current address: Health Services Research and Development Service, Department of Veterans Affairs, Washington, DC.
Abbreviations: Az = area under the ROC curve, BI-RADS = Breast Imaging Reporting and Data System, ROC = receiver operating characteristic
Author contributions: Guarantor of integrity of entire study, W.A.B.; study concepts, W.A.B., C.J.D., P.E.C.; study design, W.A.B., P.E.C.; literature research, W.A.B.; clinical studies, W.A.B., C.J.D., V.P.J., L.W.B.; data acquisition, W.A.B.; data analysis/interpretation, W.A.B., R.S.L., P.E.C., C.A.B.; statistical analysis, R.S.L., P.E.C., C.A.B.; manuscript preparation and definition of intellectual content, W.A.B.; manuscript editing, revision/review, and final version approval, all authors.
|
REFERENCES |
|---|
|
TOPABSTRACTINTRODUCTIONMATERIALS AND METHODS RESULTS DISCUSSIONREFERENCES |
|---|
This article has been cited by other articles:
|
|
 |
|
  C. A. Coursey, C. L. Hollingsworth, A. M. Gaca, C. Maxfield, D. DeLong, and G. Bisset III Radiologists' Agreement When Using a 10-Point Scale to Report Abdominal Radiographic Findings of Necrotizing Enterocolitis in Neonates and Infants Am. J. Roentgenol., July 1, 2008; 191(1): 190 - 197. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
E.-K. Kim, K. H. Ko, K. K. Oh, J. Y. Kwak, J. K. You, M. J. Kim, and B.-W. Park Clinical Application of the BI-RADS Final Assessment to Breast Sonography in Conjunction with Mammography Am. J. Roentgenol., May 1, 2008; 190(5): 1209 - 1215. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 A. Suzuki, Y. Nakamoto, T. Terauchi, M. Kawamoto, Y. Okumura, Y. Suzuki, T. Sato, N. Takahashi, J. Lee, M. Senda, et al. Inter-observer Variations in FDG-PET Interpretation for Cancer Screening Jpn. J. Clin. Oncol., August 18, 2007; (2007) hym064v1. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
G. Schueller, E. Kaindl, W. K. Matzek, F. Semturs, C. Schueller-Weidekamm, and T. H. Helbich Image Quality of a Wet Laser Printer Versus a Paper Printer for Full-Field Digital Mammograms Am. J. Roentgenol., January 1, 2006; 186(1): 38 - 43. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
P. E. Crewson Reader Agreement Studies Am. J. Roentgenol., May 1, 2005; 184(5): 1391 - 1397. [Full Text] [PDF]
|
|
|
|
 |
|
 D. Gur, L. P. Wallace, A. H. Klym, L. A. Hardesty, G. S. Abrams, R. Shah, and J. H. Sumkin Trends in Recall, Biopsy, and Positive Biopsy Rates for Screening Mammography in an Academic Practice Radiology, May 1, 2005; 235(2): 396 - 401. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 D. Saslow, J. Hannan, J. Osuch, M. H. Alciati, C. Baines, M. Barton, J. K. Bobo, C. Coleman, M. Dolan, G. Gaumer, et al. Clinical Breast Examination: Practical Recommendations for Optimizing Performance and Reporting CA Cancer J Clin, November 1, 2004; 54(6): 327 - 344. [Abstract] [Full Text] [PDF]
|
|
| ||||||||
), as compared with the consensus reference standard, before and after 1 day of BI-RADS training. The line is one of equivalence. Note that two participants performed worse after training, and two showed no change. The remaining 17 showed at least some improvement.
