Columbia Technology Ventures

Evaluating robustness and interpretability of AI models for disease detection

This technology is a method for quantifying the quality of artificial intelligence models and standardizing their application in clinical settings.

Unmet Need: Methodology for evaluating AI-derived tools in clinical settings

Medical diagnostic errors lead to millions of outpatient misdiagnoses annually in the U.S. Artificial intelligence (AI)-derived tools can support clinicians by facilitating rapid and accurate diagnoses. However, there is a lack of standardization and methodology to evaluate if AI models meet specific robustness criteria and corroborate human experts.

The Technology: Quantitative assessment of deep learning models to improve disease diagnosis

This technology introduces a method for quantifying the robustness and interpretability of deep learning models in order to standardize the application of AI-derived tools in clinical settings. It introduces convolutional neural networks (CNN) that robustly detect glaucoma from optical coherence tomography (OCT) images. Quantitative interpretability scores are generated for certain image features and compared to eye fixations of skilled medical professionals, identifying image features used by both human experts and AI. Since this technology provides a method for assessing AI interpretability based on expert feedback, it has the potential to significantly improve disease diagnosis in various medical disciplines.

Applications:

  • Detection of glaucoma and other ocular diseases from OCT images
  • Detection (in tailored embodiment) of COVID-19 using lung CT scans
  • Detection of cancers and other diseases diagnosed via medical imaging
  • Evaluating AI-mediated detection, classification, and prediction
  • Developing CNN architectures based on human expert feedback

Advantages:

  • Standardized method for quantifying AI-derived tool effectiveness
  • Versatile pipeline for identifying useful image features
  • Incorporates feedback from medical experts
  • Improves human disease diagnosis

Lead Inventor:

Paul Sajda, Ph.D.

Related Publications:

Tech Ventures Reference: