| HOME | HELP | FEEDBACK | SUBSCRIPTIONS | ARCHIVE | SEARCH | TABLE OF CONTENTS |
|
|
|
|||||||
|
|||||||||
Perspectives |
1 From the Division of Medical Education, University of Southern California School of Medicine, KAM 211, 1975 Zonal Ave, Los Angeles, CA 90033. Received February 17, 1999; accepted March 3. Address reprint requests to the author.
Index terms: Decision theory • Education • Perspectives
Medicine is a science of uncertainty and an art of probability. Sir William Osler (1)
Important components of the art of medicine are skills in repeatedly making decisions, formulating appropriate judgments, and being comfortable with risk and uncertainty. Medical training, with its heavy emphasis on factual learning, often assigns a lesser priority to the study of decision making. Our own history of medicine contributes to dismissive attitudes about decision making. Before the latter part of the 19th century, medical treatment was largely a matter of tradition, spurred on by a physician's need to do something for the patient. The ineffectiveness of most treatments made accurate diagnosis less critical than it has become in more modern times.
At the same time that technology and methods of diagnosis began to advance, it was noted that some patients improved by themselves, while others did not, and attention turned to similarities and dissimilarities among diseases and observations of the outcomes of diseases. With this new direction, physicians began to learn of the natural courses of diseases (2).
As Sir William Osler took medical teaching to the bedside, he introduced medical decision making as an exercise in problem solving between the student, teacher, and patient (3). Supported by the introduction of immunizations and early therapy with antibiotics in the 1930s, accurate diagnostic decisions became important, because some patients could be cured, and some diseases could be prevented. No longer were physicians merely predictors of outcome; now, they could effectively influence the outcome (4). Lusted (4) quoted Dr Richard Cabot, an early proponent of the clinicopathologic conference, who called differential diagnosis "a very dangerous topic—dangerous to the reputation of physicians for wisdom. It is, I suppose, owing to this danger that so little has been written on differential diagnosis and so much on diagnosis." Even today, we must guard carefully that this admonition does not remain true. There still is little agreement about whether medical diagnosis is an art or a science and about how to define the diagnostic process.
Despite a dramatic maturation of the physician's role in medicine, the emphasis in medical education still is focused on the accumulation of a large store of facts, with testing focused on factual knowledge. This practice exists despite limited understanding of how experts access and use information to solve problems. Inadequate time and effort are expended to explore and teach the art of the scientific method as it is used in the evaluation of information and the application of analytic processes to the diagnosis, prevention, and management of diseases. Our curricula often ignore the uncertain nature of clinical data, the lack of validity of clinical information, and potential errors in the decision-making process. What follows is a discussion of hypothesis formation, the use of probability in hypothesis testing, and sources of error that influence decisions.
Logical medical reasoning is grounded in the development and evaluation of a hypothesis of disease and the comparison of the hypothesis with alternative hypotheses. The alternative hypotheses are other diagnoses or "no disease." Radiologists, experts within a perceptual specialty, develop hypotheses on the basis of disease as it is revealed in observed signals or complexes of signals and differentiated from "noise." Noise is a form of artifact: a technical or variant observation or a feature that is unrelated to the positive signal and is judged to be unimportant to hypothesis formation. The inherent meaning of signals or patterns of signals is derived by the expert from previous, tested experience (2). The expert's strength is in the selection and coding of relevant variables; the expert's weakness is in combining them (5).
Signals are important because they affect the probability of a diagnostic hypothesis. Not merely the observation, but the sequence and visual environment in which the observation is made and the certainty of the observer that this is the same signal and condition, add to diagnostic probability. The expert decides on the probability that the signal is true-positive, with specific meaning, and not a false-positive or a negative observation. Definition of the signal leads to a search for a fit within established expert schemata of disease or abnormal conditions. Although experts do not all approach the act of diagnosing in the same way, this is the general method by which perceptual specialists proceed to build a diagnostic hypothesis. The expert accesses and reasons through a network of elaborate connections between abstract concepts and specific experiences. The most frequently accessed concepts or those that have been in memory the longest are the most readily available (6).
Observation of a finding (signal) and true-positive assignment represent the determination of a personal probability with which the radiologist expresses certainty about an event (observation, symptom, sign, diagnosis), with the degree of certainty being conditioned by his or her existing knowledge of the signal. (Remember: We see only what we know.) Personal probabilities are learned from experience and should constantly be tested and appropriately altered by the conscientious physician.
A theorem often used in explanations of medical reasoning is the Bayes theorem, which permits use of personal probabilities to combine observed evidence with existent prior information to reach a differential diagnosis. Lusted (4) explained three basic Bayesian points of view: (a) Probabilities are orderly opinions, (b) statistics (or any kind of information processing) are concerned with the revision of opinion in the light of new information, and (c) the Bayes theorem of probability theory is a formally optimal rule about how such revisions should be made.
Probabilities apparently are quantitative, in that they may be assigned a numeric score between 0 and 1. A personal probability indicates the degree to which the physician believes a statement or observation to be true. The nearer the probability is to 1, the stronger the probability. Because probabilities are additive, seemingly unrelated events may be combined to produce a high probability that a sign or signal is true. Personal probabilities usually are consistent within the individual and are opinions related to meaning or inferences about data. As such, personal probabilities usually are considered to be objective. One individual's personal probabilities may diverge widely from the those of another, even though both individuals may reach similar solutions to a problem. Subjective probabilities are choices and decisions reached after heavy influence by outside circumstances. Subjective probabilities are direct—the decision maker applies a number from 0 to 1 that corresponds to the probability of an outcome—and indirect—a series of related judgments or choices between possibilities is required (2).
A diagnosis is achieved when a hypothesis has been satisfactorily supported. The diagnosis is made after searching an array of data obtained in the form of a history, physical examination findings, and diagnostic test results. The hypothesis is formed on the basis of experience, and plans are created to obtain additional data to supplement existing information. If it is possible to validate a hypothesis, the patient's condition can be classified into a disease category, so that the physician can proceed to decisions about management and expected outcomes. Further testing is chosen judiciously to support the hypothesis to the degree considered essential and to provide adequate evidence for the exclusion of other hypotheses.
Costs, in terms of financial output, usefulness of testing, and consideration of patient need and comfort, are carefully considered at this point. However, the conscientious diagnostician must remain skeptical about his or her hypotheses, testing frequently until the results of the analysis are persuasive for a specific decision (7,8). The physician must decide on the degree of uncertainty that is tolerable, to establish the difference between true-positive and false-positive data and the relative weight of the predictors that are chosen to test the data. More closely related hypotheses require more testing than do widely divergent ones.
The usefulness of a test will vary depending on its predictive power and on when it is performed in the sequence of tests. As the chances of a test making a difference decrease, the usefulness begins to decrease as well. Keep in mind that the reliability of a diagnostic test often is not very strong.
Bayesian thinking invokes the comparison of a hypothesis with one or more alternative hypotheses (the differential diagnoses), always with the inclusion of a hypothesis of "no disease." An observation, finding, or test should add value to the diagnostic outcome decision. The sensitivity of a test (given a specific decision criterion) is the influence of the test or procedure on the probability of a chosen diagnosis (9). An increase in the sensitivity (true-positive contribution) results in an increase in the probability score, which moves the diagnostician closer to 100% certainty that the hypothesized illness is the correct diagnosis. Negative results of procedures are just as important for support or rejection of hypotheses.
Those procedures that exert a statistically significant effect on probability, are said to have high "diagnosticity." The "diagnosticity" of a test is directly related to its sensitivity. The relative influence of a test could be determined by calculating the degree to which every possible test outcome would influence the probability of a hypothesis (9).
Errors in decision making by inexperienced physicians usually take one of two forms: "pseudodiagnosticity" or a premature diagnostic conclusion. Pseudodiagnosticity describes a situation in which physicians interpret observations solely on the basis of a single hypothesis, believing that a high true-positive rate of supportive evidence is diagnostic in itself and ignoring false-positive results. Students and novice residents seem to prefer information on the many signs and symptoms that apply to a single disease instead of broad data that could help rule out competing diagnostic hypotheses (8–11). Pseudodiagnosticity occurs because students gather a large number of positive results that point toward one diagnosis, although the same results do not exclude alternative diagnoses.
Another pitfall of the novice is the premature diagnostic conclusion. A premature conclusion is another diagnostic error, similar to pseudodiagnosticity, in which diagnostic clues are assembled and diagnostic procedures and therapies are begun, with the focus on a single diagnosis, although hypotheses should rightly be multiple and test results may be ambiguous for the diagnosis assumed (12).
These two types of diagnostic error are important ones that emphasize that the teacher or mentor must caution students to maintain a skeptical attitude about diagnoses, testing them until it is reasonable to recognize fully justifiable conclusions. Sometimes, an independent review of diagnostic conclusions helps illustrate these mistakes.
Experienced diagnosticians may face a similar trap by using a strategy known as "heuristics" (2). This method is a shortcut for the generation and support of hypotheses established on the basis of adaptation of patterns from previous experience to solve similar problems. These "rules of thumb" are based on stereotypes of disease and usually work in uncomplicated diagnostic situations. This method lends efficiency to the diagnostic process but may suffer from inaccuracies when situations are wrongly interpreted as analogous, and stereotypical strategies are used by physicians for decision making.
Important sources of error that affect probability are problems of interpretation of positive test results as they are related to the likelihood of disease and errors in subjective probability estimates. The former situation refers to a common misunderstanding about conditional probability, as discussed by Eddy (13), who cited instances in which probability is confused with the likelihood of a test result. He compared the probability of the disease (eg, cancer) being present given a positive test result (Probability[Cancer/+Test]) with the probability of a positive diagnostic test result in a patient with the disease (Probability[+Test/Cancer]). If the probability is high, the test is considered to be diagnostic. Although the physician wishes to know the value of Probability(Cancer/+Test), the information available to physicians is the value of Probability(+Test/Cancer). Because these clearly are not the same, the physician is left with a dilemma with regard to decisions about a patient with a positive test result, especially if the cancer diagnosis was not a strong hypothesis.
Subjective probability estimates are not statistical and are highly prone to error. These subjective decisions are related to the ease with which events similar to those under consideration are retrieved from memory. Investigators (14,15) have shown that events that are easier to recall seem more likely to us. Media coverage has a strong effect on the recall of probability, which is why natural disasters and sensational events are perceived as being responsible for many deaths, whereas diabetes, asthma, emphysema, and coronary artery disease are perceived as having a lower probability for causing deaths (which is not true) (14,15). Publication in a medical journal, whatever the context and content, results in overestimates of the occurrence and lethality of a condition (16). The results of studies (17) on the availability of a medical history, even an unrelated history, have shown that the history had an effect on the diagnosis of bronchiolitis on the basis of chest radiographic findings in children, even resulting in altered consideration of the initial observations.
A related effect on decision making is the primacy effect, as reported by Wallsten (18). The initial hypothesis influences the way in which later evidence is evaluated. New data that conflict with an earlier hypothesis might be ignored. Both medical students and graduate physicians distorted evidence produced in the latter part of a work-up, to support opinions formed earlier in the diagnostic process (18). If it helped support an early hypothesis, neutral data were interpreted as positive. Results of a study (19) with physicians showed that more weight was given to data that supported an initial hypothesis than to data that refuted it.
The value of diagnostic test results is in the ability of those results to influence the physician's judgment and course of action. If test results do not change probabilities, there is no reason to perform the test, yet physicians will make predictions on the basis of information they recognize as having little diagnostic value. If a diagnostic procedure yields results that appear to be representative of the prediction, the predictive accuracy of the procedure is ignored, and the "illusion of validity" is invoked for the procedure (20). Because such an illusion is dangerous, feedback about the accuracy of any diagnostic test should be emphasized to the decision makers.
Does this mean that computer-assisted diagnosis is more accurate than human hypothesis testing? In some respects, computers are able to perform better, because computers will rapidly and thoroughly search a larger bank of information and deliver and deduce the facts with statistical accuracy. Computers learn rapidly but may have much more difficulty in separating signal from noise. Expert systems should serve as models with intelligent behavior in cognitive and perceptual realms and skills to solve problems thought to require human intelligence. Expert systems will serve as excellent consultants for decision making. People are better at clarifying a problem, suggesting kinds of procedures to follow, judging the reliability of facts, and deciding if a solution is reasonable (21). The problem solver must know how to use knowledge and see patterns in the signals presented.
The task of solving medical problems requires the formulation of logical hypotheses on the basis of signals from the patient (signs, symptoms, observations). The validity of these hypotheses must be regarded with skepticism as tests are chosen and performed to test each. The decision maker must bear in mind the inaccuracies of tests and the errors and bias that occur in the interpretation of test results. It is not so much errors due to lack of knowledge or omission that lead to diagnostic failure as it is errors in judgment or interpretation applied to the hypotheses we create.
References
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| HOME | HELP | FEEDBACK | SUBSCRIPTIONS | ARCHIVE | SEARCH | TABLE OF CONTENTS |
| RADIOLOGY | RADIOGRAPHICS | RSNA JOURNALS ONLINE |
