| HOME | HELP | FEEDBACK | SUBSCRIPTIONS | ARCHIVE | SEARCH | TABLE OF CONTENTS |
|
|
|
|||||||
|
|||||||||
Computer Applications |
1 Department of Radiology, Mental Health Clinical Research Center, the University of Iowa Hospitals and Clinics, 2911 JPP, 200 Hawkins Dr, Iowa City, IA 52242. Received July 10, 1998; revision requested August 13; revision received September 21; accepted November 13. Supported in part by National Institutes of Mental Health grants MH31593, MH40856, and MHCRC43271 and Research Scientist award MH00625, and an Established Investigator Award from the National Alliance for Research on Schizophrenia and Depression. Address reprint requests to V.A.M.
 |
Abstract |
|---|
 TOP Abstract IntroductionMATERIALS AND METHODSRESULTSDISCUSSIONReferences |
|---|
MATERIALS AND METHODS: An ANN was designed that learned from experience to define the corpus callosum, whole brain, caudate, and putamen. Manual segmentation was used as a training set for the ANN. The ANN was trained on two-thirds of the manually segmented images and was tested on the remaining one-third. The reliability of the ANN was compared against manual segmentations by two technicians.
RESULTS: The ANN was able to identify the brain structures as readily and as well as did the two technicians. Reliability of the ANN compared with the technicians was 0.96 for the corpus callosum, 0.95 for the whole brain, 0.86 (right) and 0.93 (left) for the caudate, and 0.71 (right) and 0.88 (left) for the putamen.
CONCLUSION: The ANN was able to identify the structures used in this study as well as did the two technicians. The ANN could do this much more rapidly and without rater drift. Several other cortical and subcortical structures could also be readily identified with this method.
Index terms: Brain, anatomy, 10.92 • Brain, MR, 10.121411, 10.121412 • Computers, neural network, 10.92 • Images, analysis, 10.92 • Images, processing, 10.121411, 10.121412, 10.92
|
Introduction |
|---|
|
TOPAbstractIntroductionMATERIALS AND METHODSRESULTSDISCUSSIONReferences |
|---|
An artificial neural network (ANN) offers an appealing alternative to the existing methods for structure identification, which have been primarily landmark based (24), elastic deformations to a canonical template (20–22), or knowledge based (35). Modeled on the circuitry of the brain, ANNs have been "taught" to mimic the decision-making processes needed to perform a variety of tasks that require prior knowledge and expertise, including pattern recognition processes (36–39).
An ANN, like the central nervous system, is a massive construct of many parallel functioning subunits whose components are linked in a hierarchy of layers. Data from the outside world are presented at the input level and are arithmetically processed before being turned over to the next level. The propagation of information through the upward chain is effected and stored by the system by means of minor adjustments made to the connections between subunits. Most of the commonly used ANNs consist of three groups (or layers) of units: A layer of input units is connected to a layer of hidden units, which is in turn connected to a layer of output units. Raw data are fed in at the input level. The activity of the "hidden layer" (so named because, unlike the input and output layers, it does not have direct contact with the outside world) is determined by the weight applied by each interconnection to the data values sent across it. The resultant output is also dependent on the weight applied to data in the connections between the hidden and output layers. "Supervised" learning occurs when the system is trained by entering data (input vector) that go through a series of processing steps (nodes), which results in output. The ANN is then provided with the correct "answer," which it uses to modify the steps between data input and output. The weights are the parameters that the ANN adjusts in an attempt to obtain output that is closer to the answer desired by its "trainer," which constitutes the lesson learned by the ANN in its attempt to "do better the next time around." Each weight needs to be changed by an amount proportional to the error between the output of the ANN and the correct answer.
ANNs have important potential applications in radiology as a means to identify structures and conduct measurements. Some recent applications have included the identification of areas of tumor and edema (40,41) and the measurement of gray matter, white matter, and cerebrospinal fluid (42,43).
The purpose of this study was to apply an ANN method to the challenging problem of identifying and measuring specific anatomic structures or regions within the brain. Our ANN method was applied to structures that have relatively clear boundaries: the whole brain, the corpus callosum, the caudate nucleus, and the putamen. The corpus callosum demonstrates a two-dimensional (2D) application of the ANN, whereas the other structures show the ability to define three-dimensional (3D) structures. This method appears sufficiently promising that it could be used to measure several of the subcortical and cortical structures in individual brains, thereby providing rapid automated measurements that can be used to address a variety of scientific and clinical questions.
|
MATERIALS AND METHODS |
|---|
|
TOPAbstractIntroductionMATERIALS AND METHODSRESULTSDISCUSSIONReferences |
|---|
|
All images were rated for overall quality and for movement artifacts by using a scale of 0–4 (4, excellent; 0, very poor). This scale includes degradation due to motion artifacts, susceptibility artifacts, loss of portions of the image due to the presence of metal, whole-brain coverage, and signal-to-noise ratio. Images with a quality rating of 4 would be free of artifacts of any sort; 3, minor artifacts appear in the image, such as slight motion artifacts or a slight signal intensity increase around the sinus area; 2, prominent motion artifacts or loss of small portions of the image due to the presence of metal; and 1 or 0, unusable in any further analysis owing to artifacts that dominate the image in general. Three separate ratings were acquired for each image for the anterior, middle, and posterior portions of the image sets. Only images with an average rating of 2.5 or greater were used in this study. The images were rated by postdoctoral fellows (with MD and PhD degrees) who had been trained for reliability.
Postacquisition Processing
Immediately after acquisition, the data were transferred to our image processing laboratory to undergo postacquisition processing on workstations (Silicon Graphics, Mountain View, Calif) by using locally developed software (BRAINS; Iowa Mental Health Clinical Research Center, Iowa City) (44,45). The T1-weighted images were spatially normalized and resampled to 1.0-mm3 voxels so the anteroposterior axis of the brain was realigned parallel to the anterior commissure–posterior commissure line, and the interhemispheric fissure was aligned on the other two axes. A six-point linear transformation was used to place images into the standard spaces illustrated in the atlas by Talairach and Tournoux (46). The T2- and intermediate-weighted images were then aligned to the spatially normalized T1-weighted image by means of an automated image registration program (47,48). The data sets were then segmented by means of a Bayesian classifier based on discriminant analysis to reduce the variability in signal intensity across individual image sets and to correct for partial volume effects (23,24). For the corpus callosum, only T1-weighted images were used in the segmentation. For all of the other structures, segmentation was accomplished with T1-, T2-, and intermediate-weighted images. With our multispectral data, our discriminant analysis segmentation method permits identification of the range of voxel signal intensity values that characterize gray matter, white matter, and cerebrospinal fluid (10–70 for cerebrospinal fluid, 70–190 for gray matter, and 190–250 for white matter). Each voxel is assigned an intensity value that is based on the weights assigned by the discriminant function and that reflects the relative combinations of gray matter, white matter, and cerebrospinal fluid in a given voxel. The segmented images were used as the signal intensity data input into the ANN. This provided a well-normalized signal intensity that did not vary considerably from brain to brain.
Training Data
The ANN identification and measurement procedure begins with the generation of a training or testing data set. For the corpus callosum, which was a 2D application of the ANN, a midsagittal section was chosen from the segmented images. Seventy-one images were manually traced by two experienced technicians. Forty-eight images were used to train the ANN. The remaining 23 images were used to test the ANN. The whole brain, caudate, and putamen were used to show the ability of the ANN to define a 3D structure. The whole brain was traced by two experienced technicians on 45 images. Thirty images were used for the training set, and the remaining 15 images were used to test the reliability of the ANN. Finally, the caudate and putamen segmentations were performed by two experienced technicians on 30 images each to produce the training data set for the ANN. Data from 20 images were used for training, and the remaining 10 were used to check the result.
In all cases, the traces of only one of the technicians were used to train the ANN. This allowed variability between technicians to be eliminated in the training set. Traces were obtained by two technicians to allow measurement of reliability between the two technicians and between the automated method and a manual method. The experienced technicians ranged from research assistants to postdoctoral fellows who had been trained for reliability based on the tracing guidelines for each structure.
For the results reported herein, the corpus callosum, the whole brain, the caudate, and the putamen were manually segmented according to the following rules.
Corpus callosum.—The corpus callosum was segmented on the center section of a brain normalized on the basis of the atlas by Talairach and Tournoux (46). Segmentation was started by locating the border between the white matter of the corpus and the callosal sulcus at the superior aspect of the genu. By taking partial volume effects into consideration, only those pixels on the superior border of the corpus that indicate more white matter than cerebrospinal fluid were included. Tracing was performed counterclockwise, following the border of the genu posteroinferiorly, and then along the rostrum as it curves posterosuperiorly. The inferior limit of the body of the corpus callosum did not include the pellucid septum or fornix. The border of the corpus was followed along the splenium corporis callosi as it curves superiorly, continuing along the superior border of the body.
Whole brain.—The whole brain was traced anteroposteriorly on coronal sections. The tracing was performed approximately one pixel outside the brain matter and inside the border of the dura. The following structures were excluded from the traces: cranial nerves; the dural venous sinuses, including the superior sagittal, transverse, straight, and sigmoid sinuses; and blood vessels visible on sulci and fissures. Tracing of the brain stem was stopped when the cerebral arteries first appeared posteriorly.
Caudate nucleus.—The caudate was traced anteroposteriorly on coronal sections. Care was taken not to include cerebrospinal fluid or meningeal artifacts on the medial aspect of each caudate. The nucleus accumbens septi (which abuts on the ventral aspect of the caudate and forms a "bridge" of tissue between the former and the lenticular nucleus and is first visualized after the appearance of the putamen) was carefully excluded by attempting to differentiate between it and the caudate, by comparing the different pixel intensities of the two structures. Appearing in the position formerly occupied by the nucleus accumbens septi (but in caudal traces, when the nucleus accumbens septi is no longer visible), the stria terminalis and terminal vein were noted and excluded in the same manner as described previously. After the medial and ventral delineations of the caudate were identified, the lateral delineation (the anterior and, more caudally, the posterior limbs of the internal capsule) was somewhat easier to define owing to its appearance as distinctly lighter in color. Lateral projections of caudate tissue (projecting toward and sometimes abutting on the putamen) were included in the traces. The dorsal aspect of the caudate, surrounded by white matter, has no "adjoining" structures that may be confused with it. The "tail" of the caudate was judged to terminate when it could no longer be clearly seen and not when a "cutoff point" was reached, as determined by the appearance of another structure (eg, the velum interpositum cerebri or the posterior commissure).
Putamen.—Segmentation was started when the putamen appeared as an oval abutting on the ventrolateral surface of the caudate, and the medial aspect of the putamen was defined by following the lateral aspect of the caudate. As the structure is followed caudally, the anterior limb of the internal capsule separates it from the caudate. Then the nucleus accumbens septi followed by the globus pallidus become its most medial "boundary." Again, by following the shape of the structure from section to section, a cutoff point was established between the nucleus accumbens septi and the putamen. The external medullary lamina of the globus pallidus provides such a delineation at the medial boundary of the putamen.
The ventral boundary of the putamen is first noted as the anterior commissure and later as part of the anterior perforated substance. Similarly, care was taken not to include the perforated substance, which is closest in appearance to the ventral putamen tissue. We also decided to "define" the ventral borders at the same level and continue the line marking the sharp separation between the globus pallidus and its ventral border in areas where the anterior perforated substance was present. Since a small amount of white matter separates the lateral aspect of the putamen from the claustrum, recognition of the presence and shape of this structure became important to avoid counting it as putamen. Dorsally, the putamen is well separated from the caudate by part of the anterior and posterior limbs of the internal capsule. Any "projections" or tissue "bridges" between the caudate and putamen were included as part of the caudate. The putamen was noted to gradually change shape, progressing caudally. We actively sought and excluded the optic fibers running directly ventral to the putamen in its caudal views. Tracing was stopped when the putamen could no longer be accurately visualized.
ANN Architecture
A brief search of the possible ANN architectures provided insight into the type of architecture that would work for this application. Several architectures were tried and rated for their ability to generalize, once trained. For this study, a fully connected feed-forward, three-layer ANN was used (Fig 1). The output node was used as a "fuzzy value" to indicate whether or not the voxel was in the region of interest (ROI). An experiment was performed with two output nodes (one for "yes" and the other for "no"), but they reverted into perfect complements of one another, and the second node was redundant.
|
|
The input vector for the ANN consisted of the signal intensity for the voxel being considered. To provide a computationally efficient survey of the neighborhood of the voxel, signal intensity was sampled as far away as three voxels. A circular region was used for the corpus callosum and a spherical region for the 3D applications. Only voxels diagonal and orthogonal to the location of the voxel were used, which provided information from 20 and 42 surrounding voxels, respectively. Three additional nodes were used for the 3D location of the voxel on the basis of the atlas by Talairach and Tournoux (46). The final input layer node value was the frequency with which the location was found in the search space for the training set.
The training set consisted of a number of input and target output vector pairs based on the ROIs manually segmented by one technician. The ROIs of only one technician were used because the traces were more consistent from data set to data set and allowed the ANN to develop rules more easily. The number of training ROIs may vary from one type of structure to another, but the number should reflect a set of basic principles. The number should be sufficient to contain a spectrum of deviances from the "typical" structure of interest. It should be large enough to give a good sample of the different voxel neighborhoods in the search area but small enough to be computationally efficient.
The training set consisted of ROIs from 20 subjects each for the caudate and putamen, 47 for the corpus callosum, and 30 for the whole brain. The differences in the training sets were due in part to the time required to segment the structures by hand. The corpus callosum was defined on a single sagittal section, whereas the caudate, putamen, and whole brain were defined as 3D structures. Because the exact location of borders is difficult to visualize precisely and segment accurately, border voxels were weighted less than were interior voxels. To allow a larger number of data sets to be used in the training, only a subset of the possible input vectors were used from each brain. Vectors were chosen at random from the data sets. For the corpus callosum, 164,827 input vectors were used from the data set; for the whole brain, 1,999,980; and for the caudate and putamen, 800,000 each.
The vector pairs were then fed into the back propagation algorithm, and the mean squared error of the response of the ANN, as compared with the target vector from the training set, was tracked over time. The squared error usually drops off quickly in the early part of the training and then gradually flattens out. Training was halted when an asymptote was reached (Fig 2).
|
2subject/2total, where 2total = 2subject + 2technician + 2error (49). Variance components were found by means of restricted maximum likelihood estimation (50). For the sensitivity, specificity, and overlap, one of the traces was considered the standard of reference, B, and the other as the measurement, A. The reliability measurements then become
Platform and Software
The software to execute this ANN measurement was locally developed as a component of our BRAINS software package, a group of related programs that perform a variety of image analysis tasks for neuroscience research, such as tissue classification, generation of the cortical surface, 3D surface rendering, automated measurement of sulcal and gyral anatomy, image registration, or statistical analysis of MR and positron emission tomographic data (44,45,47). The software was written in C to run on Silicon Graphics workstations, but it can be adapted to other platforms. The run time needed to train the ANN was approximately 1 day for the 3D structures and was on the order of hours for the corpus callosum. Once the ANN was trained, the time required to automatically segment these structures was less than 1 minute for the corpus callosum and was approximately 1–3 minutes for the 3D data sets.
|
RESULTS |
|---|
|
TOPAbstractIntroductionMATERIALS AND METHODSRESULTSDISCUSSIONReferences |
|---|
Face Validity: Visualization
Face validity is determined by comparing the 2D and 3D visual images of each structure as identified by the ANN with their known shape and location as seen on MR images. A series of 2D sections, with the ANN-generated boundaries superimposed, is shown in Figure 3 (corpus callosum), Figure 4 (whole brain), Figure 5 (caudate), and Figure 6 (putamen) for four representative subjects with images in the testing set (not in the training set). These images indicate that the trained ANN is able to perform shape and pattern detection at a level that approximates that of the human eye and brain. A 3D reconstruction of the left and right caudate from two subjects is shown in Figure 7, which demonstrates that the ANN detects its characteristic c-shaped curved volume and its arching extent back toward the amygdala. The full caudate (ie, including the tail) is not visualized because it is not normally detected by the human eye on MR images since it is smaller than the voxels of the images.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Speed and Efficiency
A technician trained by a neuroanatomist to conduct volumetric measurements on MR images can complete an outline of the caudate on multiple sections in 30 minutes. The corpus callosum, a simpler structure that is normally measured on only a midsagittal section, requires 5 minutes of human time. The ANN can complete the same measurements in 1–3 minutes. In addition, it does so without being subject to fatigue or inattentiveness, which can affect human measurements and diminish reliability in studies of large samples over a long time.
|
DISCUSSION |
|---|
|
TOPAbstractIntroductionMATERIALS AND METHODSRESULTSDISCUSSIONReferences |
|---|
Structure identification by means of an ANN has clear superiority over that with manual segmentation. If ANNs are carefully trained initially with a data set that is anatomically accurate, they use a stable standard that remains constant over time. Not only are manual approaches slow and time-consuming, but they are also potentially subject to other problems. Human technicians require extensive anatomic training to reliably identify the boundaries of structures, and the measurements of each new technician must be calibrated with those of his or her predecessor if measurements are to have adequate reliability. If studies are conducted over a long time, measurements of technicians must be checked periodically for "rater drift," or errors will occur that will be propagated over time. Since ANNs are trained with a stable anatomically accurate data set (as assessed by means of the benchmarks of reliability, sensitivity, and specificity) (8), they are less prone to error propagation than are human technicians. ANNs offer an automated and efficient approach to quantification of brain morphology and detection of morphologic abnormalities that eliminates inter- and intrarater variabilities and ensures reproducibility of results over time.
In addition to improved efficiency and reliability, ANNs have additional advantages. ANNs have the ability to mimic the way that the human brain recognizes patterns and locations on visual images, and they make judgments about how to discriminate among their components (36,38). More information can be assimilated and used by ANNs than can be put into other automated techniques, and they can approach complex decisions with a flexibility similar to that in human decision making. Once they have "learned" from their training set, ANNs are able to generalize well to other image sets and to make knowledge-based decisions even when confronted by novel or ambiguous data. For example, we were able to check the ability of our ANN to generalize and make stable classifications by demonstrating that it measured comparable volumes of structures with use of two different imagers. Decisions regarding voxel classification (ie, whether a voxel is or is not part of the structure in question) may be made with an ANN more expeditiously than with other methods because the ANN recognizes reference points such as the appearance of the structure in neighboring sections to define its extent in the present section. This occurs because an ANN is trained to rely on "nearest neighbor" voxels for information and to use them in classification decisions, just as the human brain does when it processes complex visual information. In fact, certain patterns inherent in structure identification or recognition of a pathologic condition may be even more apparent to an ANN since it processes data on a voxel-by-voxel basis, unencumbered by the limits of human eyesight (36–38).
Adaptation of an ANN approach to structure identification is a challenging task, however, and it is not without inherent limitations. ANNs can perform well only with structures for which adequate training sets can be developed. This may limit their value with inherently difficult structures that human beings have difficulty delineating reliably, such as the thalamus.
Further, ANNs must be well designed, and different types of ANNs may require specific training data set development, depending on the structure-identification task. Though much of the facility of the ANN lies in its inherent ability to be trained, much of the dilemma regarding the use of ANNs arises from the fact that the ANN needs to retain its ability to calculate output based on input rather than to merely memorize the path toward the right answer. An ANN should be able to adapt to variations in the data presented at the input level if it is to be used as a deductive reasoning tool rather than as a rote learning tool. Therefore, the size of the training set is paramount in determining the neural capacity of the ANN to reason. With a training set that is too restrictive, the ANN may be unable to account for minor variations in the input. With a training set with too broad a definition, the ANN will not be able to construct sufficiently definable criteria with which to exclude those areas foreign to the desired area of interest. Further, the architecture of the ANN itself may need to be modified, depending on the task.
Although input data sampling of the segmented image provided a well-normalized signal intensity and a nice border that can be easily visualized on the segmented images, some information in the original T1-, T2-, or intermediate-weighted data sets may be lost by using only the segmented images. Therefore, our future plans include the introduction of other data sets such as a combination of T1-, T2-, and intermediate-weighted images. Other structures may require a larger signal intensity neighborhood to perform adequately in the segmentation of certain structures. Also, as automated methods become better and better at replicating or producing traces that have the same or less variability as those produced by human technicians, our goal is to determine an adequate standard of reference with which to compare methods.
In our future work, we will apply this approach to other more difficult structures, with the ultimate goal of developing a highly automated and accurate way of measuring other structures or ROIs on individual MR images (eg, ventricles, other components of the basal ganglia [globus pallidus, nucleus accumbens septi], hippocampus, amygdala, and cerebellum [midline cerebellar vermis, including anterior, superior, and inferior posterior lobes]). All these structures are of interest in the study of structure-function relationships such as memory or various disease states such as schizophrenia or Alzheimer disease (20,22,25,27,28). Currently, the major limitation on our capacity to study these structures in large informative samples has been our inability to measure them rapidly, efficiently, and accurately. ANNs may offer a useful solution to this problem.
|
Footnotes |
|---|
Author contributions: Guarantor of integrity of entire study, V.A.M.; study concepts, D.H., T.C.; study design, N.C.A.; definition of intellectual content, N.C.A.; literature research, P.W.C.; experimental studies, V.A.M.; data acquisition, J.C.E., W.T.C.Y.; data analysis, V.A.M.; statistical analysis, T.C.; manuscript preparation, V.A.M., P.W.C.; manuscript editing, P.W.C., N.C.A.; manuscript review, all authors.
|
References |
|---|
|
TOPAbstractIntroductionMATERIALS AND METHODSRESULTSDISCUSSIONReferences |
|---|
This article has been cited by other articles:
|
|
 |
|
  N. C. Andreasen, C. A. Calage, and D. S. O'Leary Theory of Mind and Schizophrenia: A Positron Emission Tomography Study of Medication-Free Patients Schizophr Bull, July 1, 2008; 34(4): 708 - 719. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 R. E. Jorge, D. J. Moser, L. Acion, and R. G. Robinson Treatment of Vascular Depression Using Repetitive Transcranial Magnetic Stimulation Arch Gen Psychiatry, March 1, 2008; 65(3): 268 - 276. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 B. Hermann, J. Jones, K. Dabbs, C. A. Allen, R. Sheth, J. Fine, A. McMillan, and M. Seidenberg The frequency, complications and aetiology of ADHD in new onset paediatric epilepsy Brain, December 1, 2007; 130(12): 3135 - 3148. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 P. Nopoulos, D. R. Langbehn, J. Canady, V. Magnotta, and L. Richman Abnormal Brain Structure in Children With Isolated Clefts of the Lip or Palate Arch Pediatr Adolesc Med, August 1, 2007; 161(8): 753 - 758. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
B. Hermann, J. Jones, R. Sheth, C. Dow, M. Koehn, and M. Seidenberg Children with new-onset epilepsy: neuropsychological status and brain structure Brain, October 1, 2006; 129(10): 2609 - 2619. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 X. Chen, V. A. Magnotta, K. Duff, L. L. Boles Ponto, and S. K. Schultz Donepezil Effects on Cerebral Blood Flow in Older Adults With Mild Cognitive Deficits J Neuropsychiatry Clin Neurosci, May 1, 2006; 18(2): 178 - 185. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 C. Tamagaki, G. C. Sedvall, E. G. Jonsson, G. Okugawa, H. Hall, S. Pauli, and I. Agartz Altered White Matter/Gray Matter Proportions in the Striatum of Patients With Schizophrenia: A Volumetric MRI Study Am J Psychiatry, December 1, 2005; 162(12): 2315 - 2321. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 N. H. Pfaendner, G. Reuner, J. Pietz, G. Jost, D. Rating, V. A. Magnotta, A. Mohr, B. Kress, K. Sartor, and S. Hahnel MR Imaging-Based Volumetry in Patients with |
