Shannon C. Agner, MS, Rutgers University; Anant Madabhushi, PhD; Mark Rosen, MD, PhD; Sarah Englander, PhD; Kathleen Thomas; Mitchell Schnall, MD, PhD

Content-based image retrieval (CBIR) would be a particularly useful tool for examining breast DCE-MRI. Despite the fact that the Breast Imaging Reporting and Data System (BIRADS) lexicon for x-ray mammography has been standardized and validated, the BIRADS system for DCE-MRI is still relatively new, and the associated lexicon is used sporadically by radiologists in interpreting breast MRIs. Creating quantifiable and validated metrics associated with image characteristics will provide insight into image characteristics for research purposes. It would also be useful in training residents and fellows to foster consistent and accurate evaluation of breast cancer using DCE-MRI.

Challenges in creating a breast lesion CBIR system:

CBIR as an application in breast DCE-MRI is a relatively new concept. Lessman et al. have previously proposed methods for quantifying DCE curves for use in a CBIR system but little work has been done to evaluate parameters specific to the problem of breast DCE-MRI.[1] Signal enhancement and lesion morphology are characteristics that radiologists often point to as having the ability to distinguish benign from malignant lesions.[2] However, quantification of the performance of these feature types has not been performed in the setting of a CBIR system. In addition, the Euclidean distance is typically the similarity metric used in CBIR systems, yet there is little evidence to suggest it is the optimal similarity metric for this particular problem.

Novel contributions:

In this paper, we evaluate a content-based image retrieval system on the basis of precision and recall, in order to determine the ability of morphological and signal intensity characteristics to classify benign and malignant lesions. We also compare ten different similarity metrics to determine if a similarity metric other than Euclidean distance, generally assumed to be the default metric, ought to be used in a CBIR system for breast DCE-MRI.

A. Data description
31 breast lesions (24 malignant and 7 benign) that were imaged using DCE-MRI were collected at the University of Pennsylvania in accordance with IRB approval. Each dataset had 1 image acquisition before gadolinium contrast injection and between 3 and 8 post-contrast image acquisitions that were acquired at 90 second intervals starting from contrast injection. Lesions were segmented by an expert radiologist.

B. Feature extraction
Within the lesion volume, 7 morphological features are calculated, and 4 signal intensity coefficients are determined. 3D morphological features are calculated based on area and the centroid computed using Green’s theorem (Area Overlap Ratio, Distance Ratio, Standard Deviation of Distance Ratio, Variance of Distance Ratio, Perimeter Ratio, Compactness, Smoothness). For signal enhancement characteristics, maximum signal intensity within each lesion's volume across all timepoints was evaluated. Then, coefficients (a0, a1, a2, a3) of a third order polynomial are obtained from least-squares curve fitting. These coefficients comprise the signal enhancement features set. In addition to the morphological and signal enhancement features sets described above, we also combined the feature sets to create a combined feature set.

C. Dimensionality reduction
The fourth feature set is created by performing graph embedding, a nonlinear dimensionality reduction technique, on the combined feature set to reduce the data to a 3-dimensional feature space that preserves object adjacencies that exist in the 11-dimensional feature space. Graph embedding is different from linear dimensionality reduction methods such as principal component analysis, in that it does not assume linear relationships between data when embedding the data in the reduced feature space. The three vectors created in the embedding space comprise the fourth feature set that we term the reduced combined feature set.

D. Precision Recall
In a CBIR system, the goal is to select images in a database that are of the same class as the query image. Using the feature sets as vector representations of the images, we can calculate the similarity between two images using one of 10 distance measures: Euclidean, Bray, Curtis, Chebychev, Chi-squared, Kullbak-Leibler, Manhattan, Minkowski, Squared Chi-squared, and Squared Chord. Each of these is used to construct a symmetric d-by-d similarity matrix, where d = the number of images in the dataset (d=31 in this study). By removing any randomly selected query image from the database, we can sort the remaining images in ascending order of similarity, allowing us to retrieve an increasing number of images from the sorted list, moving from most similar to most dissimilar. By iteratively moving down the list, we can construct a precision-recall curve for each feature set and for each distance measure to identify which are more useful in describing the content of the images. Additionally, we can calculate the mean average precision (MAP) across all retrieved images: a MAP of 1.0 signifies a perfect retrieval, where all images from the relevant class are retrieved first (i.e. no out-of-class images are retrieved ahead of in-class ones).

We evaluate ten distance metrics (Euclidean, Bray, Curtis, Chebychev, Chi-squared, Kullbak- Leibler, Manhattan, Minkowski, Squared Chi-squared, and Squared Chord) and four feature sets (Signal Enhancement, Morphology, Combined features, and Reduced combined features) in their ability to distinguish malignant from benign images. Overall, malignant lesion images had a higher MAP than benign lesion images.

Feature classes;

Signal intensity features had higher MAP values than morphological features (see Figure 1). In addition, the combined feature set performed slightly better than either signal enhancement or morphology features alone. Figure 2 shows the precision-recall curves for 4 of the top-performing distance measures, as well as for Euclidean distance for signal enhancement features and morphology features.

Distance metrics;
The Canberra distance and Chi squared distance both performed consistently better than Euclidean distance in classifying the malignant lesions (see Figures 1 and 2).

We have examined both feature types and distance metrics as specific components of a CBIR system for breast DCE-MRI. Our data show that signal enhancement features and morphological features alone are weak classifiers of breast lesions. Malignant lesions that first retrieved out-of-class (i.e., benign) lesions often had more round, smooth lesion contours, which are typically characteristic of benign lesions (see Figure 3). When compared to a benign lesion (see Figure 3a), one can see how a quantified morphologic metric might classify these lesions as similar. This also points to the conclusion that morphology is likely not the single best feature set for breast lesions, and other features types must be further explored.

Although Euclidean distance is usually the putative choice for distance metric, our evaluation showed that Euclidean distance is not the obvious preferred metric. Previous work from Doyle et al. has shown that biomedical data often require a nonlinear distance metric in order to appreciate nonlinear relationships between data, and this is also reflected in our study.[4] This issue must be further explored in the context of breast DCE-MRI, but it is an important issue to keep in mind when choosing a distance metric for a CBIR system.

Although a limited dataset was used in this evaluation, it addresses various components of designing a CBIR system; it is important to evaluate and optimize features used for lesion discrimination as well as distance metrics used to create similarity matrices.