David S. Channin, MD, Northwestern University; Pat Mongkolwat, PhD; Vladimir Kleper; Daniel Rubin, MD, MS

Images, in particular medical and scientific images, contain vast amounts of information. While this information may include meta-data about the image, such as how, when, or where the image was acquired, the majority of image information is encoded in the pixel data. Observational or computational descriptions of image features (including their spatial coordinates), can be attached to an image, though there is no standard mechanism for doing so in healthcare. Worse, the majority of the human observed image feature descriptions are captured only as free text. This free text is often not associated with the spatial location of the feature, making it difficult to relate image observations to their corresponding pixel locations. Free text is also difficult, in both the lay and technical sense, to index, query, and search, to retrieve images or their features based on those free text descriptions. This limits the value of image data and its interpretation for clinical, research, and teaching purposes.

The mission of the National Institutes of Health’s (NIH) National Cancer Institute’s (NCI) Cancer Bioinformatics Grid (caBIG™) is to provide infrastructure for creating, communicating, and sharing bioinformatics tools, data, and research results, using shared data standards and shared data models. Imaging, critical to cancer research, lies at an almost unique juncture in the translational spectrum between research and clinical practice. Image and image annotation information obtained, for example, in cancer clinical trial research, is collected in a clinical setting of commercial information systems. Image annotations, in particular, need to be available in a standard format that is both syntactically and semantically interoperable with the infrastructure of caBIG™, while supporting widespread clinical healthcare standards, such as DICOM and HL7.

Selecting a single standard format to store image annotations will streamline software development and enable the work to focus on providing rich annotation features and functionality. Designing the tools to be compatible with other standards will enable a high degree of interoperability and allow the incorporation of the annotation standard into commercial, clinical information systems. This has the potential to open many existing resources and databases of cancer-related image data and metadata for exploitation not only by caBIG™, but by the broader research and clinical radiology community.

The base use case was that of the image based clinical trial:

Prerequisites:
1. One series of DICOM images from one study of one patient at a first time point.
2. One series of DICOM images from one study of same patient at a second time point.

Use Case:
1. Observer 1 annotates observations in images from first time point.
2. Observer 1 measures each observation.
3. Software application assigns unique identifier to each image annotation.
4. Observer 1 assigns human readable name to each image annotation.
5. For each named image annotation from Observer 1, Observer 2 (blinded to Observer 1’s result or not) annotates the same observation in the images from the second time point.
6. Observer 2 measures each observation.
7. Researcher retrieves annotations from Observer 1 and Observer 2.
8. Researcher performs calculations on measurements from Observers.
9. Researcher creates annotation of annotations that documents change.

Variations on this use case can meet a lot of different needs. There can be many observers at each of many time points. Adjudicators can be introduced to reconcile truth between different observers at different time points. Image processing and analysis software can replace human observers. Degenerate use cases include a clinician documenting observations in routine clinical work or an instructor documenting observations in a teaching case.

The AIM project provides a UML model of the information components in an image annotation and markup. Of primary importance are the Anatomic Entities, the Imaging Observations, and the Imaging Observation Characteristics. The AIM software library uses an ontology (Protégé, Stanford University) to deliver context sensitive controlled terminology from RadLex® for these classes of information. Imaging Observations are “things” seen in images, for example, a mass or a hyperechoic focus. Imaging Observation Characteristics further describe the Imaging Observation. For example, “speculated” is a type of margin which is a characteristic of the observation, “mass.”

The model also encompasses patient, observer, equipment, image reference, two and three dimensional coordinates of defined geometric shapes, markup and text annotation, and comments, as well as defined and arbitrary (defined by an application) calculations.

The AIM model is caBIG silver compliant. This means that the concepts used in the model, the value domains, and their meaning have all been harmonized with those of other projects in caBIG™ and the NCI. The AIM project models, in both UML and XML, can be downloaded for use. Software libraries that create AIM instances and translate instances between their different formats are available. Currently, AIM instances can be serialized as XML or DICOM SR.

The AIM project provides both a model and software to create standardized image annotations. Once adopted, in both clinical and research systems, AIM will allow the consistent tagging of image features by both humans and machines. It will then be possible to search collections of AIM annotations to find images that contain specific image features in order to explore the relationships between image observations and disease.