Quantitative evaluation of image segmentation incorporating medical consideration functions

Purpose: A quantitative and objective metric, the medical similarity index (MSI), has been developed for evaluating the accuracy of a medical image segmentation relative to a reference segmentation. The MSI uses the medical consideration function (MCF) as its basis. Methods: Currently, no indices provide quantitative evaluations of segmentation accuracy with medical considerations. Variations in segmentation can occur due to individual skill levels and medical relevance-curable or palliative intent, boundary uncertainty due to olume averaging, contrast levels, spatial resolution, and unresolved motion all affect the accuracy of a patient segmentation. Current accuracy measuring indices are not medically relevant. For example, undercontouring the tumor volume is not differentiated from overcontouring tumor. Dice similarity coefficient (DSC) and Hausdorff distance (HD) are two similarity measures often used. However, these metrics consider only geometric difference without considering medical implications. Two segments (under- vs overcontouring tumor) with similar DSC and HD measures could produce significantly different medical treatment results. The authors are proposing a MSI involving a user-defined MCF derived from an asymmetric Gaussian function. The shape of the MCF can be determined by a user, reflecting the anatomical location and characteristics of a particular tissue, organ, or tumor type. The peak of MCF is set along the reference contour; the inner and outer slopes are selected by the user. The discrepancy between the test and reference contours is calculated at each pixel by using a bidirectional local distance measure. The MCF value corresponding to that distance is summed and averaged to produce the MSI. Synthetic segmentations and clinical data from a 15 multi-institutional trial for a head-and-neck case are scored and compared by using MSI, DSC, and Hausdorff distance. Results: The MSI was shown to reflect medical considerations through the choice of MCF penalties for under- and overcontouring. Existing similarity scores were either insensitive to medical realities or simply inaccurate. Conclusions: The medical similarity index, a segmentation evaluation metric based on medical considerations, has been proposed, developed, and tested to incorporate clinically relevant considerations beyond geometric parameters alone.