Supervised Domain Adaptation Using Gradients Transfer for Improved Medical Image Analysis

A well known problem in medical imaging is the performance degradation that occurs when using a model learned on source data, in a new site. Supervised Domain Adaptation (SDA) strategies that focus on this challenge, assume the availability of a limited number of annotated samples from the new site. A typical SDA approach is to pre-train the model on the source site and then fine-tune on the target site. Current research has thus mainly focused on which layers should be fine-tuned. Our approach is based on transferring also the gradients history of the pre-training phase to the fine-tuning phase. We present two schemes to transfer the gradients information to improve the generalization achieved during pre-training while fine-tuning the model. We show that our methods outperform the state-of-the-art with different levels of data scarcity from the target site, on multiple datasets and tasks.