Video Analysis Framework for Lesion Detection in Narrow Band Imaging Bronchoscopy

The role of advanced bronchoscopic imaging techniques, especially Narrow Band Imaging (NBI), has become more crucial in the detection and staging of lung cancer, which is the leading cause of cancer death. Recent studies show that NBI bronchoscopy clearly enables visualization of certain microvascular structures in the mucosal layer (airway wall) and potential indications of developing cancerous lesions in the airways. To find these vascular patterns, the bronchoscope is navigated through the airways, and the physician manually observes potential mucosal vessel structures. We propose an automated video analysis framework based on deep learning and time-based image analysis, to exploit the richness of the video sequence to: 1) find lesions that are potential indications of developing lung cancer; and 2) isolate abnormal mucosal findings from normals. Our experiments on NBI videos of lung-cancer patients demonstrate that our framework enables effective detection of such cancerous lesions with 89% accuracy, 93% sensitivity, and 86% specificity at 19 fps speed. This is better than an off-the-shelf DL model with 69% accuracy, 57% sensitivity, and 76% specificity at 4 fps speed. Further, our method is able to isolate lesions from normal bronchial findings to mitigate the doctor’s efforts to go through a large amount of data in order to locate and observe potential abnormal lesions. Specifically, we utilize an upgraded Siamese tracker using kinematic motion modeling jointly with a detection network to isolate abnormalities, achieving 95%/90% accuracy, 90%/74% sensitivity, and 99%/99% specificity, with and without the tracker, respectively.