Two AI Systems Appear Comparable for Small Colorectal Polyps

In a head-to-head comparison, two commercially available computer-aided diagnosis systems appeared clinically equivalent for the optical diagnosis of small colorectal polyps, according to a research letter published in Gastroenterology.

Dr Cesare Hassan

For the optical diagnosis of diminutive colorectal polyps, the comparable performances of both CAD-EYE (Fujifilm Co.) and GI-Genius (Medtronic) met cutoff guidelines to implement the cost-saving leave-in-situ and resect-and-discard strategies, wrote Cesare Hassan, MD, PhD, associate professor of gastroenterology at Humanitas University and member of the endoscopy unit at Humanitas Clinical Research Hospital in Milan, and colleagues.

“Screening colonoscopy is effective in reducing colorectal cancer risk but also represents a substantial financial burden,” the authors wrote. “Novel strategies based on artificial intelligence may enable targeted removal only of polyps deemed to be neoplastic, thus reducing patient burden for unnecessary removal of nonneoplastic polyps and reducing costs for histopathology.”

Several computer-aided diagnosis (CADx) systems are commercially available for optical diagnosis of colorectal polyps, the authors wrote. However, each artificial intelligence (AI) system has been trained and validated with different polyp datasets, which may contribute to variability and affect the clinical outcome of optical diagnosis-based strategies.

Dr. Hassan and colleagues conducted a prospective comparison trial at a single center to look at the real-life performances of two CADx systems on optical diagnosis of polyps smaller than 5 mm.

At colonoscopy, the same polyp was visualized by the same endoscopist on two different monitors simultaneously with the respective output from each of the two CADx systems. Pre- and post-CADx human diagnoses were also collected.

Between January 2022 and March 2022, 176 consecutive patients age 40 and older underwent colonoscopy for colorectal cancer screening, polypectomy surveillance, or gastrointestinal symptoms. About 60.8% of participants were men, and the average age was 60.

Among 543 polyps detected and removed, 169 (31.3%) were adenomas, and 373 (68.7%) were nonadenomas. Of those, 325 (59.9%) were rectosigmoid polyps of 5 mm or less in diameter and eligible for analyses in the study. This included 44 adenomas (13.5%) and 281 nonadenomas (86.5%).

The two CADx systems were grouped as CADx-A for CAD-EYE and CADx-B for GI-Genius. CADx-A provided prediction output for all 325 rectosigmoid polyps of 5 mm or less, whereas CADx-B wasn’t able to provide output for six of the nonadenomas, which were excluded from the analysis.

The negative predictive value (NPV) for rectosigmoid polyps of 5 mm or less was 97% for CADx-A and 97.7% for CADx-B, the authors wrote. The American Society for Gastrointestinal Endoscopy recommends a threshold for optical diagnosis of at least 90%.

In addition, the sensitivity for adenomas was 81.8% for CADx-A and 86.4% for CADx-B. The accuracy of CADx-A was slightly higher, at 93.2%, as compared with 91.5% for CADx-B.

Based on AI prediction alone, 269 of 319 polyps (84.3%) with CADx-A and 260 of 319 polyps (81.5%) with CADx-B would have been classified as nonneoplastic and avoided removal. This corresponded to a specificity of 94.9% for CADx-A and 92.4% for CADx-B, which wasn’t significantly different, the authors wrote. Concordance in histology prediction between the two systems was 94.7%.

Based on the 2020 U.S. Multi-Society Task Force on Colorectal Cancer (USMSTF) guidelines, the agreement with histopathology in surveillance interval assignment was 84.7% for CADx-A and 89.2% for CADx-B. Based on the 2020 European Society of Gastrointestinal Endoscopy (ESGE) guidelines, the agreement was 98.3% for both systems.

For rectosigmoid polyps of 5 mm or less, the NPV of unassisted optical diagnosis was 97.8% for a high-confidence diagnosis, but it wasn’t significantly different from the NPV of CADx-A (96.9%) or CADx-B (97.6%). The NPV of a CADx-assisted optical diagnosis at high confidence was 97.7%, without statistically significant differences as compared with unassisted interpretation.

Based on the 2020 USMSTF and ESGE guidelines, the agreement between unassisted interpretation and histopathology in surveillance interval assignment was 92.6% and 98.9%, respectively. There was total agreement between unassisted interpretation and CADx-assisted interpretation in surveillance interval assignment based on both guidelines.

As in previous findings, unassisted endoscopic diagnosis was on par with CADx-assisted, both in technical accuracy and clinical outcomes. The study authors attributed the lack of additional benefit from CADx to a high performance of unassisted-endoscopist diagnosis, with the 97.8% NPV for rectosigmoid polyps and 90% or greater concordance in postpolypectomy surveillance intervals with histology. In addition, a human endoscopist was the only one to achieve 90% or greater agreement in postpolypectomy surveillance intervals under the U.S. guidelines, mainly due to a very high specificity.

“This confirms the complexity of the human-machine interaction that should not be marginalized in the stand-alone performance of the machine,” the authors wrote.

However, the high accuracy of unassisted endoscopists in the academic center in Italy is unlikely to mirror the real performance in community settings, they added. Future studies should focus on nontertiary centers to show the additional benefit, if any, that CADx provides for leave-in-situ colorectal polyps.

“A high degree of concordance in clinical outcomes was shown when directly comparing in vivo two different systems of CADx,” the authors concluded. “This reassured our confidence in the standardization of performance that may be achieved with the incorporation of AI in clinical practice, irrespective of the availability of multiple systems.”

The study authors declared no funding source for this study. Several authors reported consulting relationships with numerous companies, including Fuji and Medtronic, which make the CAD-EYE and GI-Genius systems, respectively.

This article originally appeared on MDedge.com, part of the Medscape Professional Network.

Source: Read Full Article