Optimization of Deep Learning Models for inference in low resource environments

Artificial Intelligence (AI), and particularly deep learning (DL), has shown great promise to revolutionize healthcare. However, clinical translation is often hindered by demanding hardware requirements.

In this study, we assess the effectiveness of optimization techniques for DL models in healthcare applications, targeting varying AI workloads across the domains of radiology, histopathology, and medical RGB imaging, while evaluating across hardware configurations. The assessed AI workloads focus on both segmentation and classification workloads, by virtue of brain extraction in Magnetic Resonance Imaging (MRI), colorectal cancer delineation in Hematoxylin & Eosin (H&E) stained digitized tissue sections, and diabetic foot ulcer classification in RGB images. We quantitatively evaluate model performance in terms of model runtime during inference (including speedup, latency, and memory usage) and model utility on unseen data.

Our results demonstrate that optimization techniques can substantially improve model runtime, without compromising model utility. These findings suggest that optimization techniques can facilitate the clinical translation of AI models in
low-resource environments, making them more practical for real-world healthcare applications even in underserved regions.