Deep Model Based Transfer and Multi-Task Learning for Biological Image Analysis

By Wenlu Zhang, Rongjian Li, Tao Zeng, Qian Sun, Sudhir Kumar, Jieping Ye, Shuiwang Ji. 2015

185 citations recorded by the source corpus.

A central theme in learning from image data is to develop appropriate representations for the specific task at hand. Thus, a practical challenge is to determine what features are appropriate for specific tasks. For example, in the study of gene expression patterns in Drosophila, texture features were particularly effective for determining the developmental stages from in situ hybridization images. Such image representation is however not suitable for controlled vocabulary term annotation. Here, we developed feature extraction methods to generate hierarchical representations for ISH images. Our approach is based on the deep convolutional neural networks that can act on image pixels directly. To make the extracted features generic, the models were trained using a natural image set with millions of labeled examples. These models were transferred to the ISH image domain. To account for the differences between the source and target domains, we proposed a partial transfer learning scheme in which only part of the source model is transferred. We employed multi-task learning method to fine-tune the pre-trained models with labeled ISH images. Results showed that feature representations computed by deep models based on transfer and multi-task learning significantly outperformed other methods for annotating gene expression patterns at different stage ranges.

Subfields: Artificial Intelligence, Biophysics, Computer Vision and Pattern Recognition, Molecular Biology

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Xing Xie
2. Qiang Yang
3. Hongwei Wang
4. Wenjie Li
5. Miao Zhao
6. Fuzheng Zhang
7. Yu Zhang
8. Fuzhen Zhuang

In Biophysics

1. Qiang Yang
2. Yu Zhang

In Computer Vision and Pattern Recognition

1. Qiang Yang
2. Yu Zhang
3. Xing Xie
4. Keyu Duan
5. Yongxin Liu
6. Dongbo Xi
7. Qing He
8. Hui Xiong

In Molecular Biology

1. Yu Zhang
2. Ola Engkvist
3. Xing Xie
4. Qiang Yang
5. Minyi Guo
6. Yinhai Wang
7. Miao Zhao
8. Markus W. Covert

Citations in the corpus

• An Overview of Multi-task Learning (2018, background): “populations [104], construction of personalized brain–computer interfaces [105], MHC-I binding prediction [106], splice-site prediction [106], protein subcellular location prediction [107], Alzheimer’s disease assessment scale cognitive subscale [108], prediction of cognitive outcomes from neuroimag”
• A Survey on Multi-Task Learning (2017, methodology): “1, many deep MTL methods [17], [18], [19], [20], [21] assume that different tasks share the first several hidden layers and then have task-specific parameters in the subsequent layers.”
• A Comprehensive Survey on Transfer Learning (2019, methodology): “[19] employs multitask and transfer learning techniques for biological image analysis.”
• Learning Modality-Specific Representations with Self-Supervised Multi-Task Learning for Multimodal Sentiment Analysis (2021, background): “Though previous works have made impressive improvements on benchmark datasets, MSA is still full of challenges.”
• The rise of deep learning in drug discovery. (2018, background): “Therefore, image features trained from natural scenes and other datasets were ‘borrowed’ to perform biological image segmentations and classi-fications, and robust performances were reported [101,113].”
• Multi-Task Feature Learning for Knowledge Graph Enhanced Recommendation (2019, background): “For example, the cross-stitch network [41] determines the inputs of hidden layers in di erent tasks by a knowledge transfer matrix; Zhou et.”
• Deep learning for cellular image analysis (2019, methodology): “In transfer learning, a deep learning model is trained on a large dataset to learn general image features, and then is fine-tuned on a smaller dataset to learn to perform a specific task 50,51 .”
• A Digital-Twin-Assisted Fault Diagnosis Using Deep Transfer Learning (2019, background): “It trains the deep model to extract high-level representations in the source domain where there is a huge amount of data available, and then transfer it to the target domain which has different data distributions [45]–[47].”
• A Decade Survey of Transfer Learning (2010–2020) (2020, background): “Secondly, most DDTL algorithms are similar to multi-task learning [109], which also benefits from shared knowledge in multiple different but related domains.”