Magnetic Resonance Sequences: Experimental Assessment of Achievements and Limitations

Abstract

Deep Learning can be applied to learn segmentations of abdominal organs in MRI sequences, a challenging task due to changing morphologies of organs along different slices. Evaluation of outcome is important to decide on applicability and to command further improvements. Software tools include evaluation metrics. Some metrics indicate quasi-perfection, with potential erroneous conclusions, visual inspection and some per organ metrics say otherwise. Our aim is the correct interpretation of commonly available metrics on organs segmentation. The method to do that is to build two architectures (DeepLab, FCN), run segmentation experiments, interpret results. Examples of results as aggregates (mean accuracy 98% weighted IoU 97%) are overly optimistic. Further analysis shows much lower scores (mean IoU 68% IoU of individual organs 78, 66, 59, 41%). We conclude that correct interpretation of the metrics, importance of further architectural or post-processing improvements on false positives.