Eliot L. Siegel, MD, FSIIM, University of Maryland School of Medicine, VA Maryland Health Care System; Joseph J. Chen, MD; Daniel Kunaprayoon, MD; Tara A. Morgan, MD; Bruce I. Reiner, MD; Naomi J. Saenz, MD

The ability to subjectively and objectively assess image quality has been important, both as a means to optimize the efficacy of imaging studies and as a metric to advance the state-of-the-art. For example, the use of image phantoms in CT has resulted in the development of techniques to assess differences in image detectors, image reconstruction techniques, and to optimize radiation dose. Despite the Transforming the Radiology Interpretation Process (TRIP) initiative, we are not aware of any previous attempts to create phantoms for subjective and objective assessment of image visualization techniques, such as volumetric rendering techniques for both static and dynamic images. This has, in the opinion of the authors, made it more difficult to compare different techniques and may have slowed the rate of innovation and new technologies in this area. We have created three types of phantom images, in order to study the relative performance of different vendors’ algorithms/approaches to visualization. These include medical images such as CT angiography of the circle of Willis, dynamic contrast enhanced images of the abdomen and pelvis; a “run-off” angiogram, as well as non-medical objects, such as a high resolution CT of a violin and other objects; and finally using mathematically defined datasets with defined patterns of pixels with varying Hounsfield units designed to test the ability of the rendering engines to display these objects and characteristics such as linearity, geometric distortion, degradation of image quality with motion, dependence on direction of motion, lighting, and artifacts associated with rendering. These dataset are available for free public unrestricted download on the Internet and are designed to serve as an initial model for use by the medical imaging community, including comparison of various vendors’ rendering systems, improvement in volume rendering algorithms, testing of the impact of various bandwidth connections on rendering quality, and education about the impact of different approaches to image quality especially for dynamically rendering images.

Seven different vendors’ volume-rendering systems were tested using the phantom datasets. Approaches by these vendors included a classification/interpolation approach, in which a zoomed 12 bit image is converted to an 8 bit image, and a classification/interpolation approach, in which an original 12-bit image is converted to an 8 bit image. The resulting image is then zoomed. Additional techniques utilized by the vendors included texture mapping and volumetric ray tracing/casting. Synthesized datasets included ones that test low opacity objects, high opacity objects, indented surfaces to test dynamic rendering and precision of rendering of small details and gradient lighting and geometric calibration using non-isotropic slightly skewed objects to assess precision of 3D measurements. Medical images that had previously been de-identified were utilized for evaluation of the texture of body organs and blood vessels and bones and complex textured physical objects, such as a classical violin, were also utilized as an additional test of the rendering engines’ performance and accuracy. All of these actual physical objects were scanned, utilizing a 64 channel scanner at a collimation of 0.75 mm and a reconstructed slice thickness of 0.75 mm. The impact of varying degrees of reconstruction overlap (e.g., .75 mm thick slices reconstructed every 0.5 mm) was also studied. Additionally, any image artifacts were noted and correlated with the basic approach of the various rendering hardware/software. Substantial variation in the quality of the rendering for these various types of datasets and in the types of artifacts encountered was identified during the evaluation of the phantom datasets and rendering systems.

Despite the widespread and increasing use of advanced visualization systems for specialized applications such as CT angiography, virtual colonoscopy, and quantitative analysis, as well as for routine workflow for all studies, surprisingly little attention has been paid to a prospective study of the relative quality and speed of various systems. We are, additionally, not aware of any previously published attempts at creating an evaluation set to compare the performance and artifacts of various approaches, nor are we aware of any other publically accessible datasets for this purpose. We believe that our initial early work should be supplemented by a combination of the vendor and academic communities to enhance and refine our initial efforts to create reference datasets. We plan to create a website and utilize an existing site to solicit feedback and suggestions and to accept additional candidates for objects for CT and, in the future, MR, PET/CT, and other modalities.