Kevin M. Schartz, PhD, The University of Iowa; Kevin S. Berbaum, PhD; Robert T. Caldwell; Mark T. Madsen, PhD

Reducing diagnostic errors in radiology will improve the quality of healthcare. In particular, it is important to avoid false negative errors in which people do not receive treatment because illness is missed. There are two ways to reduce diagnostic errors in radiology: improving imaging technology and improving the process of interpreting the radiological images.

Imaging technology is improved as a result of technology evaluation studies. These studies compare the diagnostic accuracy of new imaging systems and procedures to systems and procedures currently used. For example, the American College of Radiology (ACRIN) National Lung Screen Trial (NLST) screened over 18,000 participants for lung cancer with radiography and computed tomography. The study compared different models of radiography equipment.[1]

The process of interpreting radiological images is improved as a result of medical image perception experiments. These experiments discover how human observers detect abnormalities in radiological images and make judgments about the significance of the abnormalities. Through experiments, researchers determine ways to reduce observer errors, such as ways to avoid the failure to detect an abnormality or the failure to judge that the abnormality is indicative of a condition requiring treatment.

Berbaum (2006) has stated that experiments in diagnostic radiology would benefit by having larger patient samples and by having more observers (readers).[2] An easy way to increase the patient sample size and the number of readers is for investigators at one institution to partner with investigators at one or more other institutions. However, a possible problem is that these institutions may use different commercial imaging systems. An additional problem is that these commercial systems are intended for clinical use and may not be optimal for research purposes. For example, commercial clinical systems are not designed to collect observer reports of abnormalities and their locations, or detailed information on the interaction of the observer, with the image workstation display such as window/level and magnification settings, inspection time for each image.

The Medical Image Perception Laboratory at The University of Iowa has developed image presentation software that mimics the functionality available in the clinic, records time-stamped, observer-display interactions, and is readily deployable on diverse workstations.[3] The software records observer reports on abnormalities and their locations, and collects data on inspection time until report, inspection time for each computed radiograph, and for each slice of tomographic studies, window/level, and magnification settings used by the observer. The software, called WorkstationJ, is a modified version of the open source ImageJ software available from the National Institutes of Health.[4]

The original version of WorkstationJ was created through extensive modification of several of ImageJ package classes and the addition of custom classes. ImageJ is typically updated one or twice per year. However, this frequent updating of WorkstationJ is hindered by the extensive modifications necessary for each ImageJ update. Therefore, the 2008 Product Support Development Grant from SIIM provided funding to convert it into plugins and adding additional classes to the standard ImageJ package, rather than by modifying the ImageJ classes themselves, making WorkstationJ an open-source product that could be more easily updated.

WorkstationJ can display CT and computed radiography (CR) images. Thumbnail views of all images of the patient are initially shown on the left monitor of a dual-monitor system (see Figure 1). Individual examinations can be maximized onto the right monitor. If the CT images are the three registered, orthogonal volumetric data sets (axial, sagittal, and coronal views), then they are maximized as a group by clicking on a location in one of the views causing the other two views to be updated automatically (see Figure 2). Crosshairs can be displayed to indicate a point in the image volume.

Figure 1: This shows a typical setup for a medical image perception study with two medical flat panels. Initially, all images for a case are displayed as thumbnails on the left monitor.

Figure 2: The screen layout showing the sagittal, coronal, and axial views of the CT voxel files are maximized on the right monitor. The crosshairs indicate current position in the registered, orthogonal volumetric data sets.

When maximized, each image frame provides preset buttons for window/level adjustment (two bone presets, a lung preset, and a mediastinum preset), as well as slider controls for finer adjustment of window/level.

WorkstationJ recording of observer-display interactions is event driven in that whenever the observer initiates an action, such as changing the window/level, changing the magnification, or navigating to another slice in a CT study, an event is generated. A time-stamped, structured, textual description of the event is written to the log file. Event log files can be analyzed to produce summaries of observer behavior. Thus, WorkstationJ allows detailed information on observer responses and observer interactions with the display to be collected without using eye-tracking or videotaping the observer’s session. These logs can be used to construct summaries of observer behavior.

Researchers write simple text-based script files that are read by the WorkstationJ-specific plugins to conduct their studies. These script files provide the names of the image files and control aspects of their display, such as screen location and order of presentation. To improve performance when loading and displaying computed tomography (CT) images, the DICOM series of files is first converted to a voxel (three-dimensional) file prior to the start of the study.

WorkstationJ provides an efficient means for conducting medical image perception and technology evaluation studies within and among institutions. By deploying the WorkstationJ software together with experiment-specific script files that administer experimental procedures and image file handling, multi-institutional studies can be conducted with every reader reading every case. Thus, WorkstationJ makes it possible to add experimental conditions or increase reader sample sizes by collecting data at multiple locations without purchasing commercial systems for each location. Each location needs only a computer, a monitor or panel capable of displaying the needed resolution and image quality, and our software. The image files and the experiment-specific script files can be sent to each location, and the log files of observer behavior returned. The software is packaged as Java class files and can be used on Windows, Linux, or Mac systems.

1. Cagnon CH, Cody DD, McNitt-Gray MF, Seibert JA, Judy PF, Aberle DR. Description and implementation of a quality control program in an imaging-based clinical trial. Academic Radiology. 2006;13:1431-1441.

2. Berbaum KS. God, like the devil, is in the details. Academic Radiology. 2006;13,1311-1316.

3. Rasband WS. ImageJ. U.S. National Institutes of Health, Bethesda, Maryland. USA. 1997. Available at: http://rsb.info.nih.gov/ij.

4. Schartz KM, Berbaum KS, Caldwell RT, Madsen MT. WorkstationJ: Workstation emulation software for medical image perception and technology evaluation research. Proceedings of the International Society for Optical Engineering. 2007;6515:1-11.