Comparison of MRI response evaluation methods in rectal cancer: a multicentre and multireader validation study

Abstract

Objectives To compare four previously published methods for rectal tumor response evaluation after chemoradiotherapy on MRI. Methods Twenty-two radiologists (5 rectal MRI experts, 17 general/abdominal radiologists) retrospectively reviewed the postchemoradiotherapy MRIs of 90 patients, scanned at 10 centers (with non-standardized protocols). They applied four response methods; two based on T2W-MRI only (MRI tumor regression grade (mrTRG); split-scar sign), and two based on T2W-MRI+ DWI (modified-mrTRG; DWI-patterns). Image quality was graded using a 0–6-point score (including slice thickness and inplane resolution; sequence angulation; DWI b-values, signal-to-noise, and artefacts); scores < 4 were classified below average. Mixed model linear regression was used to calculate average sensitivity/specificity/accuracy to predict a complete response (versus residual tumor) and assess the impact of reader experience and image quality. Group interobserver agreement (IOA) was calculated using Krippendorff’s alpha. Readers were asked to indicate their preferred scoring method(s). Results Average sensitivity/specificity/accuracy was 57%/64%/62% (mrTRG), 36%/79%/66% (split-scar), 40%/79%/67% (modified- mrTRG), and 37%/82%/68% (DWI-patterns); mrTRG showed higher sensitivity but lower specificity and accuracy (p < 0.001) compared to the other methods. IOA was lower for the split scar method (0.18 vs. 0.39–0.43). Higher reader experience had a significant positive effect on diagnostic performance and IOA (except for the split scar sign); below-average imaging quality had a significant negative effect on diagnostic performance. DWI pattern was selected as the preferred method by 73% of readers. Conclusions Methods incorporating DWI showed the most favorable results when combining diagnostic performance, IOA, and reader preference. Reader experience and image quality clearly impacted diagnostic performance emphasizing the need for stateof- the-art imaging and dedicated radiologist training.