DEPARTMENT OF COMPUTATIONAL AND DATA SCIENCES
Ph.D. Thesis Colloquium
__________________________________________________________________________________________
Speaker : Mr. Hariharan Ravishankar
S.R. Number : 06-18-00-11-12-20-1-18917
Title : “Unsupervised test-time adaptation for patient-specific deep learning models in medical imaging”
Research Supervisor: Prof. Phaneendra Kumar Yalavarthy
Date & Time : September 11, 2023 (Monday) at 10:00 AM
Venue : # 102 CDS Seminar Hall
__________________________________________________________________________________________
Abstract
Deep learning (DL) models have achieved state-of-the-art results in multiple medical imaging applications, resulting in the widespread adoption of artificial intelligence (AI) models for radiological workflows. Despite their success, they exhibit two major weaknesses in real-world applications: a) poor generalization and b) lack of patient optimality. The potential “distribution shift” in unseen medical imaging data, owing to changes in demography, acquisition hardware, etc., often leads to a reduction in performance. Despite the high average performance, they are prone to failures on “individual” cases with minor input modulations than the training data. Solving this problem is critical in medical image analysis because this variance in performance across cases and abject failures on select subjects will increase the burden on care-giving experts and reduce trust in AI-based applications.
The aim of this thesis work is to develop methods for patient-specific test-time adaptation of pre-trained DL models on individual subject data. While generalization is often treated as a training time goal, test-time adaptation (TTA) modifies the weights during inference time in an unsupervised manner specific to each subject data to jointly address generalization and patient optimality. While sustaining performance in individual subjects is seen as a challenge for current DL models, this line of research also has an opportunity for next-generation clinical decision support systems. AI methods that would result in patient-specific DL models can be heralded as the next big step in medical imaging.
Information Geometric Test Time Adaptation (IGTTA) for semantic segmentation: The test-time adaptation (TTA) of deep-learning-based semantic segmentation models, specific to individual patient data, was addressed in this part of thesis work. Existing TTA methods in medical imaging are often unconstrained and require prior anatomical information or additional neural networks built during the training phase, making them less practical and prone to performance deterioration. In this part of thesis work, a novel framework based on information geometric principles was proposed to achieve generic, off-the-shelf, regularized patient-specific adaptation of models during test-time. By considering the pre-trained model and adapted models as part of statistical neuromanifolds, test-time adaptation was treated as constrained functional regularization using information geometric measures, leading to improved generalization and patient optimality. The efficacy of the proposed approach was shown on three challenging problems: a) improving the generalization of state-of-the-art models for segmenting COVID-19 anomalies in Computed Tomography (CT) images, b) cross-institutional brain tumor segmentation from magnetic resonance (MR) images, and c) segmentation of retinal layers in Optical Coherence Tomography (OCT) images. Furthermore, it was demonstrated that robust patient-specific adaptation can be achieved without adding a significant computational burden, making it the first of its kind based on information geometric principles.
Test-time adaptation via domain transforms (TTADT): OCT imaging has emerged as the modality of choice for the diagnosis of retinal diseases. The popular, cost-effective variant of OCT – Spectral-Domain Optical Coherence Tomography (SDOCT) often suffers in image quality owing to speckle noise, making expert or automated analysis of these images challenging. This part of thesis work proposes for the first-time inference time adaptation of deep learning models for a given test sample to achieve improved diagnostic performance. The adaptation is driven by domain transforms – a framework in which domain-specific augmentation is employed for a test sample, and the deep learning model weights are updated in a constrained and regularized manner. Performance of the proposed approach was evaluated on both simulated and real-world noisy B-mode OCT images affected by varying degrees of speckle noise from four benchmarking SD-OCT datasets. Systematic studies in this part of thesis work clearly demonstrate the utility of the proposed approach in building generalized and robust deep learning models for automated retinal disease diagnosis using OCT images.
Uncertainty-aware test-time adaptation for inverse problems (UATTA): Deep learning models for inverse problems (both one-pass and unrolled versions): have these limitations 1) inability to provide pixel-wise uncertainty quantification 2) sensitivity to variation in forward model and 3) sub-optimal regularization over different iterations. In this part of thesis work, uncertainty aware test-time adaptation of off-the-shelf models is proposed for improved image reconstruction performance. Different pixel-wise uncertainty quantification methods were explored for utility in test-time adaptation, along with data fidelity constraints. The results were studied for two relevant problems:1) accelerated MRI acquisition and 2) quantitative susceptibility mapping for quantifying tissue magnetic susceptibility. The proposed framework provides a reliable and accurate reconstruction of these images.
Test-time adaptation with foundational models as neural surrogates (TTANS): Recently, foundational models have demonstrated impressive promptable segmentation of natural images. However, these models often perform poorly with medical imaging data. In this part of the thesis, a two-way synergic approach for jointly utilizing foundational models while adapting pretrained medical imaging segmentation models is explored. The results are demonstrated on various medical imaging semantic segmentation tasks to show the ability of foundational models as neural surrogates, thus demonstrating test-time adaptation achieved in real time. As these foundational models are capable of handling high levels of semantics, using them as surrogates helped reduce reliance on costly annotated data needed to achieve state-of-the-art results.
In summary, this thesis work developed methods to improve the generalization of typical deep-learning models utilized in medical imaging by personalizing them to individual patients during testing in an unsupervised and automated manner. The developed methods demonstrated the applicability and utility in different medical imaging tasks across various medical imaging modalities. This thesis work is the first of its kind in medical imaging to showcase the utility of test-time adaptation methods, paving the way for further research towards the goal of precision medicine.
===============================================================================
ALL ARE WELCOME
