James Y. Chen, MD, University of Maryland School of Medicine/University of Pennsylvania; Paul Nagy, PhD; Eliot L. Siegel, MD, FSIIM; Nabile Safdar, MD; F. Jacob Seagull, PhD; Paras C. Lankhani, MD; Elias R. Melhem, MD, PhD

Manual segmentation of simulated lesions with a graphics tablet is at least as accurate, easy, and fast as that performed using an optical mouse.

Lesion segmentation involves outlining the contour of an abnormality on an image to distinguish boundaries between normal and abnormal tissue. Segmentation is essential to track malignant and benign disease in medical imaging for clinical and research purposes. Large trials, such as the ACRIN National Lung Screening Trial, as well as individual clinical cases, currently rely on accurate and repeatable methods of lesion measurement and segmentation. The creation of automated segmentation tools relies on testing against the gold standard of manually segmented lesions. This gold standard, however, may vary in accuracy, depending on the input method used for measurement. To our knowledge, nearly all clinical and research users use the mouse as the input device for manual lesion segmentation.

The QUERTY keyboard and mouse are the de facto standard configuration in computer input devices. Although these work well for standard user interface interactions, many graphic designers have chosen to replace or augment these devices with graphics tablets that more closely mimic conventional pen and paper. Applications that require manual lesion segmentation bear similarities to the tasks of graphic designers, chiefly a need for finer motor control. Most people performing segmentation, however, still rely mainly on the mouse interface. The mouse interface responds to larger movements, predominantly those associated with arm motion; whereas finer movements, such as wrist flexion, determine the accuracy of pen strokes. We hypothesize that the pen-and-tablet interface should be empirically and subjectively at least as accurate, easy, and fast as an optical mouse for lesion segmentation.

This is a prospective study comparing 2 different computer input devices for manual segmentation of simulated lesions, including a post-experiment survey of all participants.

Two sets of images were created with simulated reference lesions and then combined into a single experimental set. Each image contained only a single lesion, to minimize the cognitive task and time required to identify the lesion. Lesions were created with high-contrast, black-on-white backgrounds and hard edges. Lesion contours were created to simulate expected clinical lesions, including ovoid, lobulated, and spiculated forms. To account for differences in lesion size, the first set of 3 images was resized 50% in each dimension using bicubic interpolation to create a second set of smaller size. The 2 sets were then merged as a single experimental set.

Optical mice (Logitech International S.A.; Switzerland) used the default driver (Microsoft Corporation, version 5.1.2600.5512) and were set to default movement parameters, including variable gain/acceleration. The graphics tablet (Wacom Corporation; Japan) was also set to default parameters within the driver software (Wacom Corporation, version 6.00-5).

Five radiologists performed manual lesion segmentation using commercially available image editing software (Adobe Systems, USA). Subjects were randomly assigned to perform the first segmentation with either the optical mouse or graphics tablet. Participants were allowed a single image on which to practice segmentation, up to 2 times with each input device, prior to segmenting each experimental set. Lesion segmentation time was recorded manually for individual lesions.

Undersegmented areas were defined as areas of reference lesion that was not included in manual segmentation (false exclusion). Oversegmented areas were defined as areas of segmented lesion that did not correlate to the reference lesion (false inclusion). The segmented lesions were subtracted from the reference lesion to obtain areas of under-segmentation. The reference lesion was then subtracted from the segmented lesion to obtain the area of oversegmentation. The areas of over- and undersegmentation were combined to assess the total mis-sampled area, representing the Root Mean Square Error. Differences were compared with a 2-tailed, paired Student t-test.

After segmentation of both lesion sets, each individual completed a survey including number of years of experience with a mouse or tablet; perceived ease, speed, and accuracy of segmentation; and preference for segmenting lesions with the mouse or graphics tablet.

Average lesion volume was 16,041 pixels. Segmentation with the tablet was more accurate than with a mouse with average RMS error of 1,350 pixels (8.4% of average lesion volume) versus 739 pixels (4.6% of average lesion volume) (P = 0.00046).

Time for segmentation per lesion did not vary with the 2 devices, averaging 16.0 and 16.1 sec with mouse and tablet, respectively (P = 0.98, R2 = 0.93).

All radiologists reported more experience using a mouse than a tablet, with 100% having >5 y of experience. For the graphics tablet, 40% reported no experience, 40% reported up to 1 y of experience, and 20% reported 3–5 y of experience.

Sixty percent of participants reported the perception that segmentation with a mouse was slower for segmentation, 20% believed that the mouse was faster, and 20% believed that the speed with the 2 devices was equal.

All (100%) participants reported segmentation with a mouse to be more difficult as well as less accurate. All (100%) participants also reported a preference for using a graphic tablet when segmenting lesions.

A significant decrease in error was seen when manually segmenting lesions with a graphics tablet, compared with the same task performed with a mouse. Despite a general lack of experience with a graphics tablet, no significant difference in time to segment lesions was noted between the graphics tablet and mouse. All operators reported the graphics tablet to be (subjectively) more accurate and easier, and all preferred the tablet and perceived it as easier for task performance. This suggests that with the same amount of time expenditure, lesions can be segmented more accurately and with less effort than with a mouse, a finding that could have significant implications for research or clinical work involving large numbers of lesions.

The perceived increased ease of lesion segmentation may magnify differences between the mouse and graphics tablet when large volumes of lesions are segmented within and across patients. Operator fatigue and frustration with the input method during segmentation can lead to less accurate segmentation when the task is performed repeatedly. If segmentation with the tablet is truly easier, the graphics tablet could potentially decrease the amount of, and effects of, fatigue. This could result in creating more accurate datasets with large numbers of segmented lesions against which to evaluate automated methods of segmentation as they are developed.

We speculate that additional experience with the tablet beyond that held by the participants in this study might increase the speed of lesion segmentation, but only minimally affect accuracy––an area that remains for future evaluation.

Choice of input device for manual lesion segmentation was found to significantly affect segmentation accuracy. These findings suggest that for both clinical and research purposes, the graphics tablet is superior to the mouse for manual segmentation with no cost in speed.

1. Cao L, Li X, Zhan J, Chen W. Automated lung segmentation algorithm for CAD system of thoracic CT. Journal of Medical Colleges of PLA. 2008;23:215-22.
2. De Xivry JO, Janssens G, Bosmans G, et al. Tumor delineation and cumulative dose computation in radiotherapy based on deformable registration of respiratory correlated CT images of lung cancer patients. Radiotherapy and Oncology. 2007;85:232-8.
3. Lao Z, Shen D, Liu D, et al. Computer-Assisted Segmentation of White Matter Lesions in 3D MR Images Using Support Vector Machine. Academic Radiology. 2008;15:300-13.
4. Kostopoulos S, Glotsos D, Kagadis GC, et al. A hybrid pixel-based classification method for blood vessel segmentation and aneurysm detection on CTA. Computers & Graphics. 2007;31:493-500.