Multi-institutional observer performance study for bladder cancer treatment response assessment in CT urography with and without computerized decision support

We evaluated whether a computerized decision support system for bladder cancer treatment response assessment (CDSS-T) can assist physicians from different institutions in identifying patients who have complete response after neoadjuvant chemotherapy. Pre- and post-chemotherapy CTU scans of 96 patients (114 pre- and post-treatment lesion pairs) were collected retrospectively. The pathological cancer stage after treatment was collected as the reference standard of response to treatment. 24% of the lesion pairs had T0 cancer stage (complete response) after chemotherapy. Our CDSST that combined DL-CNN and radiomics features was trained to distinguish between T0 and <T0 cases. Five abdominal radiologists and 3 oncologists participated in the observer study. One radiologist and one oncologist were from external institutions. All physicians estimated the likelihood of stage T0 disease after treatment by viewing each pre-post-treatment CTU pair displayed side by side on a specialized graphical user interface. The observer provided an estimate without CDSS-T first and then might revise the estimate, if preferred, after the CDSS-T score was displayed. The observers' estimates with and without CDSS-T were analyzed with multi-reader, multi-case (MRMC) methodology. The AUC for prediction of T0 disease after treatment was 0.85±0.04 for the CDSS-T alone. The performance of all but one observers increased with the aid of CDSS-T. The average AUC for the observers was 0.77 (range: 0.69-0.83) without CDSS-T, and increased to 0.80 (range: 0.72-0.86), (p = 0.006) with CDSS-T. The CDSS-T could improve the performance of radiologists and oncologists from different institutions in identifying patients who fully responded to treatment. There was no apparent difference in the performance of the physicians from different institutions.