You can't learn without problems:

1. What is the scale of the data in the text? What are the characteristics of these data?

2. Why use transfer learning?

3. How does the article verify the accuracy of its model?

4. How is each conclusion in the abstract justified in the article?

Highlights:

1. It is the first time to use such a large amount of labeled high-quality data for transfer learning to build an artificial intelligence system;

2. Effective identification and quantitative assessment of severity of macular degeneration and diabetic macular edema based on optical coherence tomography (OCT) data;

3. Accurately distinguish between bacterial and viral pneumonia on chest X-rays of children (difference analysis and accurate judgment)

Target:

Transfer learning was used to identify categories in retinal (OCT) images, and transfer learning was also used to test pneumonia identification on chest radiographs.

The study also provides human interpretability for machine diagnostics by showing areas of neural network activation.

The real launch was at the beginning of last year (2017). A total of 1 year from launch to publication.

Heart of the Machine: The reasoning of the neural network is a "black box". How does the new method you propose explain the basis for the "diagnosis" made by the computer?

Zhang Kang: We have added "occlusion testing" to the research of retinal OCT images - by convolving an occlusion core onto the input image, the machine will make a correct diagnosis by calculating the prediction of the most probable part, and output a block containing a bright color These color blocks are the lesions that the AI "thinks", and an intuitive diagnosis basis that is trusted by clinicians is obtained.

First of all, by inputting a large amount of data, the neural network can obtain "experience" far exceeding that of human doctors, and calculate accurate results that exceed human beings. In our system, we use more than 200,000 medical images, through different disease classifications, The machine was eventually trained using nearly 110,000 retinal OCT images. In terms of eye diseases, it can correctly identify choroidal neovascularization, diabetic macular edema, drusen, and OCT images of normal retina within 30 seconds. Humans have similar or even higher accuracy. Secondly, the computer compares the difference between image pixels and pixels, and observes details that humans cannot pay attention to, so as to draw more accurate judgments, and is not subject to the same subjective interference as humans. In addition, through the algorithm of "transfer learning", we can also diagnose diseases of different systems. For example, our system can currently accurately identify pneumonia and normal chest X-ray films, and distinguish whether the pathogen of pneumonia is bacteria or virus. The accuracy rate is acceptable. more than 90%.

"Transfer learning" is considered an efficient learning technique, especially when faced with relatively limited training data. Compared with "starting from scratch" of most other learning models, "transfer learning" uses Convolutional Neural Network (CNN) to learn existing pre-trained network systems that have been labeled, taking medical image learning as an example , the system will recognize the characteristics of the images in the pre-system, we will continue to import the network system containing the similar parameters and structures of the first layer of images, and finally build the final level. In our system, the first-level network is the retinal OCT image, and the second-level network system uses the first-level images to find the corresponding features, fixing the weights in the lower-level images through forward propagation to find the learned discernible structures , and then extract higher-level weights, in which repeated self-adjustment, feedback, and transmission are performed to achieve the purpose of learning to distinguish specific types of images. For the first time, we use such a large and labeled high-quality retinal OCT data for transfer learning to detect common retinal blindness diseases and recommend treatment methods, and obtain accuracy similar to or even higher than that of human doctors. This artificial intelligence system can also "infer others" by applying transfer learning to the diagnosis of pediatric pneumonia.

Transfer learning is a natural development direction of deep learning. Transfer learning can make deep learning more reliable and help us understand deep learning models. For example, we can know which features are easy to transfer, and these features correspond to some high-level and abstract structural concepts in a certain field. Distinguishing their details allows us to develop a deeper understanding of knowledge representation in this field. In this way, machines can learn life-long like a biological nervous system, constantly summarizing and summarizing past knowledge, so that a system can learn faster and faster, and it can also discover how to learn in the process of learning.

Transfer learning has extremely broad application prospects in deep learning. In the medical field with limited image data resources, transfer learning that is more efficient and requires fewer images can be said to be the hot spot of AI development and the successful application of deep learning in the next five years. driving force.

For example, if there are some images with vague image features, such as age-related macular degeneration, some larger drusen and choroidal neovascularization are very similar, we will prefer to take a more serious disease diagnosis, because the ultimate goal of our research It is to help patients be more likely to be referred to the corresponding specialist, so as to obtain treatment faster. In addition, we can also set more realistic filters through our ideas, and continuously adjust them according to the needs of our clinicians; the "occlusion experiment" can reflect the basis of the machine's judgment. Furthermore, our research can guide the determination of treatment options. Therefore, our research may be more able to achieve the desired effect of clinicians, and be trusted by clinicians, and may be applied to the clinic faster and more directly.

Learn the following neural network, suitable for Chinese videos, not too popular: Transfer Learning Transfer Learning

Learning methods and materials: https://github.com/jindongwang/transferlearning

Cell paper data and code address: https://data.mendeley.com/datasets/rscbjbr9sj/2

Run the code and play!

Interpretation of the original article:

Identifying Medical Diagnoses and Treatable Diseases by Image-Based Deep Learning

Identify medical diagnoses and treatable diseases

Image-based deep learning classifies macular degeneration ([ophthalmology] macular degeneration) and diabetic retinopathy ([ophthalmology] diabetic retinopathy) using retinal optical coherence tomography images and has potential for generalized applications in biomedical image interpretation and medical decision making.

An artificial intelligence system using transfer learning（迁移学习） techniques was developed
It effectively classified images for macular degeneration and diabetic retinopathy
It also accurately distinguished bacterial and viral pneumonia on chest X-rays
This has potential for generalized high-impact application in biomedical imaging

Transfer learning under popular science: what is transfer learning? What is the historical development prospect of this field?

The implementation of clinical-decision support algorithms for medical imaging faces challenges with reliability and interpretability.

Here, we establish a diagnostic tool based on a deep-learning framework for the screening of patients with common treatable blinding retinal diseases.

Our framework utilizes transfer learning, which trains a neural network with a fraction of the data of conventional approaches（常规方法）. Applying this approach to a dataset of optical coherence tomography images, we demonstrate performance comparable to that of human experts in classifying agerelated macular degeneration and diabetic macular edema. We also provide a more transparent and interpretable diagnosis by highlighting the regions recognized by the neural network. We further demonstrate the general applicability of our AI system for diagnosis of pediatric pneumonia using chest X-ray images.

This tool may ultimately aid in expediting the diagnosis and referral of these treatable conditions, thereby facilitating earlier treatment, resulting in improved clinical outcomes.

Figure 2. Representative Optical Coherence Tomography Images and the Workflow Diagram

result

The primary application of our transfer learning algorithm was in the diagnosis of retinal OCT images.

Spectral-domain OCT uses light to capture high-resolution in vivo optical cross sections of the retina that can be assembled into three-dimensional-volume images of living retinal tissue. OCT is critical to guiding the administration of anti-VEGF therapy by providing a clear cross-sectional representation of the retinal pathology in these conditions (Figure 2A), allowing visualization of individual retinal layers, which is impossible with clinical examination by the human eye or by color fundus photography.

Mainly do the following four things:

Patient and Image Characteristics

数据：We initially obtained 207,130 OCT images. 108,312 images (37,206 with choroidal neovascularization, 11,349 with diabetic macular edema, 8,617 with drusen, and 51,140 normal) from 4,686 patients passed initial image quality review and were used to train the AI system.

After 100 epochs (iterations through the entire dataset), the training was stopped due to the absence of further improvement in both accuracy (Figure 3A) and cross-entropy loss (Figure 3B).

Performance of the Model

We evaluated our AI system in diagnosing the most common blinding retinal diseases.

Comparison of the Model with Human Experts

An independent test set of 1,000 images from 633 patients was used to compare the AI network’s referral decisions with the decisions made by human experts.

Occlusion Testing

We performed an occlusion test on 491 images to identify the areas contributing most to the neural network’s assignment of the predicted diagnosis.

Application of the AI System for Pneumonia Detection Using Chest X-Ray Images

To investigate the generalizability of our AI system in the diagnosis of common diseases, we applied the same transfer learning framework to the diagnosis of pediatric pneumonia.

案例二：In Silico Labeling: Predicting Fluorescent Labels in Unlabeled Images

Annotation in Computers: Predicting Fluorescent Labels in Unlabeled Images

In silico labeling, a machine-learning approach, reliably infers fluorescent measurements from transmitted-light（透射光） images of unlabeled fixed or live biological samples.

Microscopy is a central method in life sciences.

Many popular methods, such as antibody labeling, are used to add physical fluorescent labels to specific cellular constituents.

However, these approaches have significant drawbacks, including inconsistency; limitations in the number of simultaneous labels because of spectral overlap; and necessary perturbations of the experiment, such as fixing the cells, to generate the measurement.

Here, we show that a computational machine-learning approach, which we call ‘‘in silico labeling’’ (ISL), reliably predicts some fluorescent labels from transmitted-light images of unlabeled fixed or live biological samples. ISL predicts a range of labels, such as those for nuclei, cell type (e.g., neural), and cell state (e.g., cell death). Because prediction happens in silico, the method is consistent, is not limited by spectral overlap, and does not disturb the experiment. ISL generates biological measurements that would otherwise be problematic or impossible to acquire.

As such, it is unclear whether deep learning approaches would provide a significant and broad-based advance in image analysis and are capable of extracting useful, not readily apparent, information from unlabeled images.

we show it can accurately predict the location and texture of cell nuclei, the health of a cell, the type of cell in a mixture, and the type of subcellular structure.

We also show that the trained network exhibits transfer learning: once trained to predict a set of labels, it could learn new labels with a small number of additional data, resulting in a highly generalizable algorithm, adaptable across experiments.

Training and Testing Datasets for Supervised Machine Learning

Developing Predictive Algorithms with Machine Learning

Network Predictions of Cell Nuclei

Network Predictions of Cell Viability

Network Predictions of Cell Type and Subcellular Process Type

Adapting the Generic Learned Network to New Datasets: Transfer Learning

To be continued~

Image Recognition | AI in Medicine | Deep Learning | Transfer Learning