Contrast-detail (CD) analysis is a common way for assessing the visibility of various details provided by an imaging system. Several phantoms have been designed for estimating CD results. CDMAM phantom was specifically developed for mammography systems. Such contrast detail measurements rely on a large number of observer readings. This procedure suffers from two main disadvantages. One advantage is the presence of significant inter-observer error. The other disadvantage is that using human observers is time consuming and a very tedious task. It would, therefore, be convenient to have an automatic method of obtaining threshold contrast data when testing digital mammography systems. A possible solution to these drawbacks is using a software which allows automatic reading of CDMAM images. CDCOM is a well-know and freely downloadable software developed by the Radiology Department of the University Medical Centre in Nijmegen. It is very useful for automatically evaluating images of the CDMAM 3.4 phantom.[1,2]

The aim of this paper is to present a comparison between human and software reading of CDMAM images coming from different FFDM systems. We already achieved a complete CD analysis of the investigated systems by human observers.[3] Here, we also consider automatic analysis of the same data, in order to achieve a comparison between human and automatic evaluation. CDCOM software was used for assessing CD results in an automatic way.

We acquired CDMAM images from three different FFDM clinical units.[3] The first system is a CR unit FCR 5000-MA manufactured by Fuji (Tokio, Japan), the second is an indirect FPD Senographe 2000D, by GE Medical Systems (Milwaukee, WI, USA), and the last is a direct FPD Giotto Image-MD, by IMS (Bologna, Italy). For all the measurements, a common setup was chosen. All measurements were obtained with no compressor, no anti-scatter grid, and a fixed tube voltage equal to 28 kVp (Mo-Mo). A low scatter condition is simulated by putting a 2 mm aluminum foil attached to the X-ray generator head. To assess the contrast-detail characteristic, we used the well known CDMAM 3.4 phantom developed in Nijmegen. It consists of a matrix of squares, each one containing two identical gold disks of given thickness and diameter. One disk is placed in the center and the second in a randomly chosen corner. The observer has to indicate the corner where the eccentric disk is located. The object thicknesses range between 0.03 and 2 microns of gold, resulting in a radiation contrast range of about 0.5-30% at standard mammography exposure conditions. CDMAM reading are obtained both by human observers and by the developed ad-hoc software CDCOM.[1,2]

Human observers evaluated images on two dedicated high-resolution monitors (Barco MGD521, 2048×2560 matrix, 8 bit, max luminance: 500 cd/m2). The operating conditions were chosen so that all three systems would be very near to the best possible observer’s performance. The images were presented on the monitor with the room light off, using suitable zoom factor, brightness, and contrast. The contrast and the brightness have been fixed at the same value for the three systems. There was no restriction on viewing time for each decision trial, and observers were given feedback about whether their decision was correct. Five experienced operators evaluated the entire image set. The three systems were analyzed at two different exposures (70 uGy and 140 uGy). Six images were acquired for each exposure. We used an apposite software, in order to facilitate human reading of the CDMAM images. Figure 1 shows the main panel of this software. Here, each cell is cropped from original images and presented, randomly rotated, to observers. Further, crops are randomly chosen from one of the six images acquired for each exposure. The operator has to indicate in the corner in which the eccentric disk is most likely to be located. The software stores results for the final analysis, where the contrast detail curve is calculated by fitting data with a psychometric curve.

Figure 1

We also achieved CD results by means of the CDCOM software. The main difference between the human reading and the software reading is that here the location of the eccentric disk is found by the software itself. In this way, the subjective variability of humans is dramatically reduced, and the time required for analyzing several images is very short. Once software has completed the reading, the interpretation of the results is achieved in the same way used for the human case.

The evaluation of the radiographic image quality by CDCOM is done in several steps: First, the algorithm determines the border of the phantom and resolves its position and the centre of its contrast-detail cells by means of the Hough transform. After that, average pixel value within four ROIs are computed for each cell. ROIs are located on the four corners of the cell. ROI with the highest average value is supposed to be the one which includes the eccentric disk. Average pixel values and standard deviation are also evaluated on ROIs located over the central disk and in the background. The program tests with a statistical method, if the average contrast-detail is greater than the average background, to consider the disk as detected. CDCOM provides results in the form of two different matrixes, one for the eccentric disks, and another one for the details positioned in the center of each cell. Results from the six images acquired for each exposure are averaged. These results are then fitted with the same psychometric curve used with human observers. For each investigated condition, a contrast detail curve is obtained, together with two different figures: Correct Observation Ratio (COR) defined as the ratio between the correct observations and the total sqares, and Image Quality Figure (IQF) defined as the summation over all contrast columns of the product between threshold in contrast and the diameter.

Figure 2 illustrates a direct comparison between human and automatic CD curves for the three systems at exposures equal to 70 uGy. As already noted, CDCOM results are significantly better than human ones over the entire range of details diameter. These curves show that the gap between human and automatic analysis is almost constant for all the details of Fuji and GE systems. Conversely, IMS system presents a larger gap for small details, whereas the gap decreases for large details.

Figure 2

Table 1 summarizes results of human and automatic analysis of the CDMAM images for the three FFDM systems and all the investigated conditions. COR and IQF values are shown for the two exposures (70 uGy and 140 uGy). First, we can note that CDCDOM software is able to provide results significantly better than those estimated by human observers. This is confirmed both by COR and IQF data. Besides, CDCOM confirms the very similar response of Fuji and GE systems, for both the investigated exposures. On the other hand, IMS provides worse results, with respect to other two systems.

Table 1

1. Fletcher-Heath L, Van Metter R. “Quantifying the performance of human and software CDMAM phantom image observers for the qualification of digital mammography systems.” SPIE Proc. 2006;6142.
2. Young KC, Cook JH, Oduko JM, Bosmans H. “Comparison of software and human observers in reading images of the CDMAM test object to assess digital mammography systems.” SPIE Proc. 2006;6142.
3. Rivetti S, Lanconelli N, Campanini R, et al. “Comparison of different commercial FFDM units by means of physical characterization and contrast-detail analysis.” Med. Phys. 2006;33:(11):4198-4209.