Explainable Deep Learning in Spectral and Medical Image Analysis

Xuyang Liu; Chaoshu Duan; Wensheng Cai; Xueguang Shao

doi:10.7536/PC220512

Progress in Chemistry ›› 2022, Vol. 34 ›› Issue (12) : 2561-2572 DOI: 10.7536/PC220512

research-article

Scan to read & Share

Explainable Deep Learning in Spectral and Medical Image Analysis

Author information +

History +

PDF

Abstract

Deep learning is a modeling method based on neural network, which is constructed of multiple different functional perception layers and optimized by learning the inherent regularity of a large amount of data to achieve end-to-end modeling. The growth of data and the improvement of computing power promoted the applications of deep learning in spectral and medical image analysis. The lack of interpretability of the constructed models, however, constitutes an obstacle to their further development and applications. To overcome this obstacle of deep learning, various interpretability methods are proposed. According to different principles of explanation, interpretability methods are divided into three categories: visualization methods, model distillation, and interpretable models. Visualization methods and model distillation belong to external algorithms, which interpret a model without changing its structure, while interpretable models aim to make the model structure interpretable. In this review, the principles of deep learning and three interpretability methods are introduced from the perspective of algorithms. Moreover, the applications of the interpretability methods in spectral and medical image analysis in the past three years are summarized. In most studies, external algorithms were developed to make the models explainable, and these methods were found to be able to provide reasonable explanation for the abilities of the deep learning models. However, few studies attempt to construct interpretable algorithms within networks. Furthermore, most studies try to train the model through collecting large amounts of labeled data, which leads to huge costs in both labor and expenses. Therefore, training strategies with small data sets, approaches to enhance the interpretability of models, and the construction of interpretable deep learning architectures are still required in future work.

Contents

1 Introduction

2 Principle and algorithm

3 Interpretability method

3.1 Visualization method

3.2 Model distillation

3.3 Interpretable model

4 Spectral analysis

5 Medical image analysis

5.1 Segmentation

5.2 Disease diagnosis

6 Summary and outlook

Graphical abstract

Keywords

deep learning / interpretability method / neural network / medical image analysis / spectral analysis

Cite this article

Download citation ▾

Xuyang Liu, Chaoshu Duan, Wensheng Cai, Xueguang Shao. Explainable Deep Learning in Spectral and Medical Image Analysis. Progress in Chemistry, 2022, 34(12): 2561-2572 DOI:10.7536/PC220512

登录浏览全文

4963

注册一个新账户忘记密码

1 Introduction

Deep learning is a class of machine learning methods that have developed rapidly in recent years. Its core idea is to construct a deep neural network to achieve nonlinear fitting of the prediction target. Since the 21st century, the remarkable improvement of computing power has made the high-speed training of deep neural network models a reality, and a large number of theoretical and applied studies of deep learning have emerged in the field of computer science in a short period of time. In recent years, deep learning has been widely used in computer vision, natural language processing and other fields, and has made remarkable progress. Google network (GoogLeNet), residual network (ResNet) and Visual geometry group network (VGG) based on large-scale data training have been established. Thanks to the disclosure of the above model parameters, the model training time is significantly reduced and the generalization ability is improved by transferring these pre-trained networks. With the gradual maturity of the theory and application of deep learning in the field of computer science, other fields have begun to apply deep learning methods to complete large-scale data analysis tasks. In medicine, deep learning has been used to process medical images for disease diagnosis tasks^[1]. In the field of life sciences, deep learning has been used for protein structure prediction as well as genomics research^[2,3]. In the field of chemistry, deep learning has been used to predict the physicochemical properties of compounds, model spectral analysis, and improve quantum chemical computing methods^[4⇓⇓~7]. However, deep learning models are generally considered to lack interpretability due to their high nonlinearity and large scale of parameters, and researchers cannot make a judgment on the correctness of the predicted results of the models when they are applied in practice, thus seriously hindering the practical application of deep learning. Interpretability methods are proposed to solve the above problems. According to the different interpretation principles, interpretability methods are divided into visualization, model distillation and interpretable model. The visualization method and model distillation belong to the external interpretation algorithm. After the model training is completed, the visualization method realizes the visualization of the model decision basis by analyzing the model parameters or perturbing the input.Model distillation refers to the interpretation of models by means of model simplification, while interpretable models aim to design interpretable model structures in principle.

Spectral analysis tasks include discriminant analysis of samples and qualitative and quantitative analysis of sample components. Many spectral analysis methods have been developed in the field of chemometrics, such as support vector machine and linear discriminant analysis for discriminant analysis, partial least squares regression for quantitative analysis, and other multivariate calibration methods. However, the analysis process of these methods is usually cumbersome, requiring a large number of spectral preprocessing operations before analysis, such as scattering correction, noise removal, baseline correction, spectral derivation, variable selection, etc., and most of the spectral analysis methods have certain limitations.For example, partial least squares regression only has the ability of linear fitting, and support vector machine needs to constantly adjust the penalty factor and kernel function, which leads to the need for trial and error in method selection and model adjustment to adapt to spectral data^[8]^[9⇓~11]. As a highly nonlinear fitting algorithm, deep learning has the ability to simplify the spectral analysis steps and achieve end-to-end high-precision spectral analysis by learning the inherent laws of large amounts of data.

Medical image analysis includes lesion region segmentation, disease diagnosis, etc. Traditional medical image analysis needs experienced experts to complete, which requires huge time and economic costs. Deep learning is expected to change this situation. The development of deep learning in the field of computer vision has paved the way for its application in medical image analysis. Training diagnostic models based on transfer learning has become a mainstream research method, and fully convolutional neural networks such as U-Net, SegNet and DeepLab have been used in a large number of studies to achieve high-precision segmentation of specific regions in medical images^[12]^[13,14].

The existing literature has summarized the application of deep learning in spectral and medical image analysis, and Yang et al. Summarized the application of deep learning in vibrational spectral analysis before 2019^[8]. He et al. Summarized the related work of applying deep learning to biological spectroscopy and imaging in recent years^[15]. Shen et al. Introduced the application of deep learning in medical image registration, segmentation and diagnosis before 2017^[16]. The above literatures all point out that the lack of explainability of deep learning is the key factor that hinders its application, and the attempt to improve the explainability of deep learning models is still in its infancy, so future research should pay more attention to the interpretability of deep learning. This paper introduces the principles of deep learning and three kinds of interpretability methods from the perspective of algorithm, reviews the application of deep learning interpretability methods in spectral analysis and medical image analysis in the past three years, and looks forward to the future development direction.

2 Principle and algorithm

The concept of neural network originates from biology, and a large number of neurons in the human brain are connected by synapses to form a nervous system. Although the structure of a single neuron is simple, the nervous system is able to solve highly complex problems through the collaboration of a large number of neurons. The design of Artificial neural network is inspired by biological neural network. McCulloch et al. First proposed the computational model of Artificial neural network (ANN) in 1943^[17]. Since then, Multilayer perceptron (MLP) architecture and back propagation training algorithm have appeared one after another^[18,19]. Due to the limitation of computing power and data size, ANN models made slow progress before the 1990s. However, thanks to the substantial improvement of computing power and the demand of the era of big data, neural networks have become one of the most popular machine learning technologies, and are widely used to solve the classification and regression problems of large quantities of data.

ANN is a neural network formed by stacking multiple fully connected layers, and the operation process in the fully connected layer can be expressed by formula (1):

(1)

Y = σ X W + b

Where W and B are the weight and bias matrices of the fully connected layer, respectively, and

σ (·)

is the activation function.

In order to improve the computational power of ANN in high-dimensional data analysis, researchers designed and developed Convolutional neural network (CNN). CNN is a neural network architecture composed of convolutional layers, pooling layers, and fully connected layers. Fig. 1 shows the model structure and calculation process of CNN, in which the convolution layer extracts the spatial features of the input through the convolution operation of several constantly moving kernels and high-dimensional matrices. The pooling layer downsamples the input data, thereby reducing the amount of data and attenuating the model's location sensitivity to the input data. Fully-connected layers are used to implement classiﬁcation or regression tasks at the end of the network.

In 2014, Long et al. First proposed Fully convolutional neural network (Fully convolutional network, FCN), which replaced the fully connected layer with deconvolution layer to achieve the output of high-dimensional matrix^[20]. In 2015, Ronneberger et al. proposed the U-Net fully convolutional neural network and used it for the segmentation task of cell images^[21]. The U-Net model consists of a compression process and an expansion process. In the compression process, the model extracts the low-dimensional features of the input through a convolution layer and a pooling layer. In the expansion process, the model reconstructs the low-dimensional features into a high-dimensional matrix through a deconvolution layer and skip connections. Thanks to its excellent image segmentation performance, U-Net has been applied in a large number of image segmentation tasks.

Recurrent neural network (RNN) is proposed to deal with sequence data with unequal length. Fig. 2 shows the structure and calculation process of RNN, where

H t

represents the hidden state of the t-node in the recurrent layer, and equation (2) is the calculation formula of

H t

X t

and

W X

are the data input to the recurrent layer under node t and the corresponding weight parameters,

H t - 1

and

W H

are the hidden state of the previous node and the corresponding weight parameters, B represents the bias, and

σ (·)

represents the activation function of the recurrent layer.

O t

is the output of the t-node of the recurrent layer, and its calculation is the same as that of the fully-connected layer, see equation (1), where

X

is here

H t, Y

is here

O t

(2)

H t = σ X t W X + H t - 1 W H + b

Gradient explosion and gradient disappearance exist in the training of RNN structure, and gradient explosion can usually be dealt with by saturation activation function and gradient clipping strategy. Hochreiter et al. proposed the Long short-term memory (LSTM) model in 1997 to address the gradient vanishing phenomenon of Long sequence RNNs, and Cho et al. simplified the LSTM structure and proposed the Gate recurrent unit (GRU) in 2014^[22]^[23]. In addition, other researchers have developed strategies such as merged attention mechanism, dot product attention mechanism and multi-head attention mechanism to alleviate the gradient disappearance phenomenon of RNN^[24]^[25]^[26].

Generative adversarial network (GAN) was first proposed by Goodfellow et al. In 2014^[27]. GAN is composed of a generator and a discriminator, which aims to improve the model performance through the confrontation between the generator and the discriminator. The discriminator is used to distinguish the data generated by the generator from the real data, and the generator is used to transform the input to approximate the real data. In the training process of GAN, the data generated by the generator is gradually close to the real data through the continuous confrontation between the discriminator and the generator, and the discrimination ability of the discriminator is also continuously improved. The single iteration training of the generative adversarial network is divided into two stages: the first stage is to train the discriminator so that it can recognize the real data and the data generated by the generator, and the second stage is to train the generator so that the data generated by the generator is closer to the real data. Inspired by the classical GAN idea, new GAN structures such as pix2pix and cycleGAN have been developed in recent years^[28]^[29]. The GAN structure has been widely used in the research of super-resolution and semi-supervised learning.

Although many neural networks with advanced structure and function have been developed in the field of deep learning, most of them are difficult to interpret the model because of their complex model parameters and strong nonlinearity. This seriously hinders the practical application of neural networks, so it is necessary to carry out the interpretability research of neural networks.

3 Interpretability method

3.1 Visual interpretation

Visual interpretation is usually used to explain the prediction results of the classification model. When a sample is input into the model, the visual interpretation method calculates a Saliency map matrix (which reflects the degree of influence of each region of the input data on the prediction results of the model) according to the parameters and inputs of the model to explain the predictions of the classification model. According to the calculation method of saliency map, visual interpretation is divided into visual interpretation based on gradient calculation and visual interpretation based on perturbation. The former computes a saliency map from the gradient of the neural network, and the latter computes a saliency map based on the effect of perturbations of the input data on the output.

In 2016, Zhou et al. Proposed a Class activation mapping (CAM) algorithm, which limits the last three layers of the neural network to a concatenated structure of convolutional layer, global average pooling layer and fully connected layer, thus limiting the performance of the model^[30]. In 2017, Selvaraju et al. Proposed the grad-CAM algorithm, which removes the restriction of CAM on the three-layer structure of the model, so it has become one of the most popular visual interpretation methods^[31]. Fig. 3 shows the process of calculating the saliency map by grad-CAM. First, the image "Input" is Input into the classification model to obtain the prediction probability

y c

of the neural network for a specific category. Then extract the output matrix A of the last convolution layer of the model, calculate the

y c

and the gradient value of each point on A according to formula (3), and calculate the mean value of each channel to obtain the

α k c

, where I and J represent the row and column indices of the two-dimensional matrix respectively, K represents the channel index, and Z represents the number of pixels; Then the gradient mean

α k c

is used as a weight term to weight and sum the output of the middle layer of the neural network by formula (4), and the summation result is activated by the ReLU function to obtain the saliency map matrix

L c

; Finally, the

L c

is upsampled to the size of the original image and overlaid with it, and the importance of each region in the input can be seen. In this example, the

L c

pixel value corresponding to the cat's face region is the highest, indicating that this region is the key region affecting the model decision.

(3)

α k c = 1 Z ∑ i ∑ j ∂ y c ∂ A i j k,

(4)

L c = R e L U ∑ k α k c A k,

The grad-CAM algorithm calculates the saliency map based on the partial derivative of the model output to the intermediate layer output, and then measures the importance of each pixel of the input data. However, the neural network has the phenomenon of gradient saturation, when the gradient is saturated, the partial derivative tends to zero, so that the partial derivative can not accurately reflect the contribution of each input region to the model decision. In 2017, Sundararajan et al. Proposed the Integrated gradients algorithm, which measures the importance of each pixel in the input by integrating the gradient, aiming to eliminate the impact of gradient saturation on the model interpretation algorithm^[32]. Fig. 4 shows the calculation process of the integral gradient algorithm. First, the baseline

X'

is defined as a completely black image, and the original image X is interpolated with a uniform step

α × X - X'

from the baseline. Then the interpolated image is input into the model, and the partial derivative "Gradients" of the model output to each pixel on the interpolated image is calculated; Then the partial derivative of each interpolated image is numerically integrated by equation (5) to obtain the saliency map matrix

I G X

; Finally, the saliency map matrix

I G X

is added to the original image to reflect the importance of each region of the input image. In this example, the face and back regions of the cat correspond to the highest

I G X

values, indicating that these regions are the key regions affecting the model decision.

(5)

I G X = (X - X') × ∫ α = 0 1 ∂ F X' + α × X - X' ∂ X i d α

The visual explanation based on perturbation counts the change of the model prediction results before and after perturbing each region of the input data, and thus quantifies the degree of attention of the neural network to each region when making decisions. In 2014, Zeiler et al. Proposed the Occlusion sensitivity algorithm, which covers different areas of the input data and calculates the change of the model's prediction value before and after covering. When the model's prediction result changes significantly after covering a certain area, it is considered that the area is the key area concerned by the model^[33]. Fig. 5 shows the distribution of the changes in the model prediction results before and after the disturbance of each region of the input image. The model prediction results change greatly after the disturbance of the key features such as the face, ears, and paws of the cat, which proves that these regions are more critical to the model decision, indicating that the model pays attention to the reasonable input regions when making decisions. In 2017, Fong et al. Proposed three perturbation methods, including replacing all perturbation areas with constants, adding noise to the perturbation areas, and blurring the perturbation areas^[34]. In the same year, Zintgraf et al. Proposed a Prediction difference analysis algorithm for visual model classification mechanism, which more rigorously measured the impact of disturbance on model Prediction^[35]. Equations (6) and (7) are the calculation principle of the predictive difference analysis algorithm, where C is the category, X is the input data of the neural network,

x i

is the data in the disturbance area, and

x - i

represents the data outside the disturbance area. Equation (6) calculates the prediction contribution

p c x - i

x - i

to class C. Equation (7) measures the prediction contribution

W E i c x

of disturbance data

x i

to class C. Its value reflects the influence degree of regional

x i

on class C prediction.

(6)

p c x - i = ∑ x i p x i | x - i p c x - i, x i

(7)

W E i c x = l o g p c | x 1 - p c | x - l o g p c x - i 1 - p c x - i

3.2 Model distillation

The idea of model distillation is to approximate a complex neural network model to an interpretable simple model, and then explain the neural network by the simple model. Model distillation can be divided into local approximation and global approximation according to the approximation method, and the existing research focuses on the local approximation method. Local approximation is the approximation of a simple model to the neural network prediction results of a single or a few samples. In 2016, Ribeiro et al. Proposed the LIME (Local interpretable model-agnostic explanations) algorithm^[36]. Fig. 6 is an example of the algorithm of LIME to approximate the neural network to a linear model. First, the sample X is converted into a matrix

X', X' ∈ {0,1}

represented by "0,1"; Perturbing the

X'

for a plurality of times to obtain a

Z'

, extracting an original image Z corresponding to the disturbed characteristic

Z'

, and inputting Z into a neural network to obtain a prediction probability

f Z

; Then the formula (8) is minimized, so that the prediction probability

g Z'

of the simple model is close to the prediction probability

f Z

of the neural network, and the approximation of the simple model to the neural network is realized. The formula (9) calculates the sample similarity

π x Z

before and after the disturbance, where

D X, Z

is the distance between the image before and after the disturbance, and σ is the width of the input image; Finally, the weight parameter W of the linear model is extracted and sorted. The higher the weight is, the greater the contribution of the corresponding pixel region to the prediction result is. The higher weight parameters in Figure 6 correspond to the key areas of the cat's face, ears, tail and body, which prove that LIME has the ability to explain the neural network. Shapley value is a concept in game theory, which is a more rigorous measure of the contribution of features to prediction. In 2017, Lundberg et al. Introduced the Shapley value to improve the similarity term Shapley additive explanations in LIME algorithm, and proposed the Shapley additive explanations (SHAP) algorithm^[37].

(8)

ξ X = ∑ Z', Z π x Z f Z - g Z' 2,

(9)

π x Z = e x p - D X, Z 2 σ 2 .

Global approximation is the approximation of a simple model to the prediction results of a neural network with a large number of samples. In 2017, Frosst et al. Proposed a strategy to construct a soft decision tree through a neural network. The generalization ability of the decision tree is better than that of the result learned directly from the training data^[38]. Because of the hierarchical decision characteristics of the decision tree, the decision interpretation model of each leaf node on the path can be predicted by analyzing the decision tree.

3.3 Interpretable model

The idea of an interpretable model is to add interpretable modules to the model to understand the prediction mechanism of the model. At present, the widely used interpretable models are usually divided into attention mechanism model and multi-task model. In 2014, Bahdanau et al. proposed Bahdanau attention mechanism to solve the problem of limited memory of RNN model^[24]. The authors constructed an encoder-decoder RNN structure, and added an attention mechanism to the decoder to selectively focus on each node in the encoder. Equations (10) ~ (13) are the calculation process of Bahdanau attention mechanism,

s i

represents the hidden state of the ith node in the decoder, and

c i

is the result of weighted summation of the hidden state of each node in the encoder.The weight

α i j

represents the attention degree of the decoder

s i

to the jth time node in the encoder, and the weight

α i j

is calculated by equations (12) and (13). Since the

s c o r e i j

in the Bahdanau attention mechanism is solved by combining the hidden States of the encoder and the decoder, it is also called combined attention.

(10)

s i = σ s i - 1, y i - 1, c i,

(11)

c i = ∑ j = 1 T α i j h j,

(12)

α i j = e x p (s c o r e i j) ∑ k = 1 T e x p (s c o r e i k),

(13)

s c o r e i j = σ [s i - 1, h j] W ν T .

The follow-up work of attention mechanism focuses on the improvement of the calculation of

s c o r e i j

and the scope of attention in Bahdanau's attention mechanism. In 2015, Xu et al. Proposed the concepts of soft attention mechanism and hard attention mechanism^[39]. The hard attention mechanism means that the decoder can only focus on a specific time node in the encoder, and the soft attention mechanism means that the decoder can focus on all time nodes in the encoder. In the same year, Luong et al. Proposed the dot-product attention mechanism. Equation (14) is the definition of the

s c o r e i j

in the dot-product attention mechanism, and the

s c o r e i j

is obtained by performing the dot product operation on the hidden state of the encoder and the hidden state of the decoder^[25]. In 2017, Vaswani et al. proposed the concept of self-attention mechanism combined with the idea of dot product attention, and proposed the multi-head attention mechanism based on it^[26]. Combined attention mechanism, dot product attention mechanism and multi-head attention mechanism are the three mainstream attention mechanisms at present. Although the attention mechanism is not proposed for interpretability, there have been many works demonstrating its ability to improve the explanatory power of the model^[26,39]. In the network using the attention mechanism, the

α

can reveal the regions that the model pays attention to when making decisions by extracting the weight coefficients in the attention layer.

(14)

s c o r e i j = s i - 1 T h j .

In the multi-task model, the neural network needs to complete other tasks besides the original classification task, such as generating a paragraph of text to explain the classification of the neural network, or converting the input into interpretable hidden features in the middle layer of the model^[40]. In 2017, Alvarez-Melis et al. Proposed Self-explaining neural networks (SENNs), which explain neural networks by introducing linear computing modules^[41]. SENNs first transform the input into a set of low-dimensional hidden feature "concepts" and a set of correlation parameters, then use the correlation parameters as weights to weight the low-dimensional features, and finally use the weighted results to predict the sample class. Fig. 7 shows the interpretation mechanism of SENNs for the digital image classification model. First, typical samples that maximize the activation of "concepts" are selected from the training data to reveal the meaning of each neuron. For example, the third neuron "Cpt3" in "concepts" is more sensitive to numbers with horizontal structures such as "7" and "8"; Then the contribution of each "concept" to the model prediction is measured by the correlation parameter. When the model input "9", the correlation parameter corresponding to "Cpt3" is the highest. Finally, combined with the meaning and contribution of neurons to explain the model, "9" has a structure similar to the transverse fold, and the correlation parameter of "Cpt3" is the highest in line with expectations, indicating that the model pays attention to reasonable features when making decisions.

Although the explanatory power of the multi-task model is excellent, there are difficulties in sample labeling and model regulation. The attention mechanism has become the most popular interpretable model because of its simple model construction and intuitive interpretation ability.

4 Spectral analysis

The traditional spectral analysis process usually requires tedious data preprocessing steps, including spectral denoising, baseline correction, scattering correction, spectral derivation, variable selection and so on^[8]. It has been proved that traditional machine learning methods perform well in the analysis of complex system spectra. As one of the most advanced machine learning technologies, deep learning is expected to replace the traditional spectral analysis process to simplify the analysis steps and improve the analysis ability^[42,43]. At present, many studies have used deep learning for quantitative analysis of spectra. In 2019, Ju et al. Established a model for three-dimensional fluorescence spectra of human body fluids, thus realizing the detection and semi-quantification of prohibited drugs in body fluids^[44]. The authors constructed GAN and ANN models to detect and quantify illicit drugs, and the GAN and ANN models achieved a prediction accuracy of 88.07% and a mean absolute error of 0.16 mg/mL on the test set, respectively, proving that deep learning has the ability to analyze the spectra of complex systems. Ni et al. Proposed a variable weighted convolutional neural network and used it to quantify the nitrogen content of Pinus massoniana seedling leaves from near-infrared spectroscopy^[45]. The model first calculates the weight of each point in the spectrum, and then predicts the nitrogen content of leaves from the weighted spectrum. The results show that the generalization ability of the weighted model is improved, so that the neural network can pay more attention to the important segments of the spectrum. Zhang et al. Proposed a regression strategy based on autoencoder low-dimensional vectors and used it to predict the content of each chemical component in Lycium barbarum L^[46]. The authors constructed an autoencoder to reduce the dimensionality of the spectrum, and used the data after dimensionality reduction to perform partial least squares regression and least squares support vector regression on the compound content in Lycium barbarum. Xiong et al. Proposed a partial least squares regression strategy combined with attention mechanism, which is superior to the traditional quantitative method in predicting the near-infrared spectral data of three kinds of real samples^[47].

In addition to using deep learning for quantitative analysis of spectra, there are also a large number of studies using deep learning for discriminant analysis. Huang et al. Proposed a strategy to detect milk powder adulteration based on an autoencoder^[48]. The authors used the infrared spectrum of real milk powder to train the model, calculated the coding error and reconstruction error of different milk powder when input into the automatic encoder to cluster the samples, and realized the correct identification of more than 90% adulterated milk powder by adjusting the threshold. This study demonstrates that unsupervised learning has the potential to be used for clustering analysis. Although unsupervised learning has attracted much attention, the mainstream strategy to achieve sample classification is still supervised learning based on a large number of labeled samples. In 2020, Shin et al. Proposed a strategy of combining principal component analysis and neural network to diagnose lung cancer based on Raman spectra of exosomes^[49]. Exosomes are one of the most widely concerned biomarkers. Due to the limitations of traditional methods, it is difficult to achieve cancer diagnosis by analyzing exosomes. Cell line exosomes are the main research object at present, but there are few studies on exosomes in plasma with more complex composition. The cost of extracting cellular exosomes in practical diagnosis is very high, and it is more desirable to achieve cancer diagnosis through easily available plasma exosomes. The authors proposed a strategy to train the model by Surface-enhanced Raman spectroscopy (SERS) of cell line exosomes and use plasma exosomes for diagnosis. The diagnosis strategy is shown in Figure 8. Firstly, the SERS data of plasma exosomes, healthy cell exosomes and cancer cells are input into the trained model, and the output of the previous layer of the model output layer is extracted for Principal component analysis (PCA). Then the Mahalanobis distances of plasma, cancer cells and normal cells on the first and second principal components were calculated, and the similarity between plasma exosomes and cancer cell exosomes was defined as the ratio of the two distances. Finally, the similarity threshold is set to diagnose lung cancer. Based on the above strategy, the area under the Receiver operating characteristic curve (ROC) of the model reached 0.912, which was significantly better than that predicted directly by the model. Shu et al. Established a deep learning model for real-time detection of nasopharyngeal carcinoma in vivo based on fiber Raman spectroscopy^[50]. The authors used Raman spectra of nasopharyngeal tissue in vivo to train the model, and the true positive rate of the prediction results reached 92.18%, and the key wavenumber regions calculated by the perturbation-based visual interpretation algorithm were proved to be related to the characteristic biomolecules of cancer. Huang et al. Established a diagnostic model of COVID-19 based on residual network for SERS of throat swabs from different populations, and the prediction accuracy of the model reached 87.7%^[51]. Yu et al. Established a classification model for the Raman spectra of eight types of bacteria in sea intestines, and obtained the spectral fragments reflecting the characteristics of bacterial nucleic acids through an interpretability algorithm based on perturbation^[52]. Zhou et al. Used the strategy of sparse regularization to perform variable selection on the spectra, and classified the wheat varieties through the spectra after variable selection^[53]. Yang et al. compared various classification strategies for terahertz spectra of coffee beans from different geographical sources, and the results showed that the classification results of convolutional neural network were significantly better than those of traditional methods such as linear discriminant analysis and support vector machine^[54].

In addition to applying deep learning to end-to-end regression and classification, there are also some studies using deep learning methods to preprocess spectral data. Imaging in the near-infrared IIb region can minimize light scattering while maximizing imaging depth, and the imaging quality is high, but the fluorescent probes needed for imaging usually contain toxic elements such As Pb, Cd, and As, which are difficult to be used in clinical trials. Although the fluorescent probe has biocompatibility in near-infrared region I and region IIa imaging, the imaging quality is generally poor due to strong light scattering and small imaging depth. cycleGAN is a generative model that can realize the conversion between two groups of images without paired data training. In 2020, Ma et al. used a large number of near-infrared imaging data of region IIa and region IIb to train the cycleGAN model, and realized the conversion of imaging results from region IIa to region IIb^[55]. As shown in Fig. 9, after cycleGAN processing, the light scattering of region IIa imaging results is significantly reduced, and the signal-to-background ratio is significantly increased, which is close to the experimental results of region IIb imaging, proving the effectiveness of cycleGAN method. In the same year, Guo et al. Built a 1-dimensional U-Net model for removing artifacts in infrared spectra, which can effectively extract the real spectra covered by artifacts^[56].

Many studies on spectral analysis have shown that deep learning has the potential to discover the hidden relationship between the spectrum and the target, and the interpretability algorithm of deep learning used in spectral analysis can not only provide the basis for the prediction results of the model,To enhance the credibility of the model, it is also possible to find some unclear spectral information, thus promoting the development of basic research. In addition, deep learning also has the potential to preprocess spectra, and the adoption of deep learning for spectral preprocessing helps to reduce the imaging cost and simplify the analysis steps.

5 Medical image analysis

5.1 Segmentation

Medical image analysis aims to analyze the medical images of patients to achieve early detection, diagnosis and treatment of diseases, including medical image segmentation, disease diagnosis and other tasks. Segmentation is the classification of images at the pixel level, and it is the gold standard of medical image segmentation to mark the category of each pixel artificially by professional doctors, which requires a lot of time and energy, and the quality of segmentation is affected by the professional level of doctors. In order to improve the segmentation quality of medical images and shorten the segmentation time, a large number of studies have used deep learning to achieve high-precision segmentation of medical images. Real-time detection and removal of polyps during colonoscopy is the best way to prevent colorectal cancer. In 2018, Wang et al. Established a deep learning model to mark polyps in real time during colonoscopy^[13]. In order to achieve high-precision real-time detection of polyps, the author uses the SegNet structure proposed by Badrinarayanan in 2015. First, the input is downsampled to extract the key features, and then the key features are upsampled to complete the segmentation task. The true positive rate of the model diagnosis test set is as high as 90%^[57].

In addition to the SegNet architecture, the U-Net architecture has been largely used for medical image segmentation tasks due to its simplicity of construction and excellent model performance. In 2020, Shi et al. Improved the U-Net and proposed the Dual Attention Residual U-Net (DAResUnet), which was used to segment the aneurysm region in head CT images^[14]. DAResUnet trains the model by first cropping random regions of the input data, and then implements segmentation by cropping the input data with uniform stride when the model is inferred. DAResUnet embeds the dual attention mechanism proposed by Fu in 2019 into the U-Net structure to force the neural network to focus on meaningful features, and replaces the convolutional layer with residual units to alleviate the degradation of the deep network^[58]. The performance of DAResUnet model in segmenting the aneurysm region in head CT images is not only significantly better than that of the traditional U-Net structure trained under the same conditions, but also better than the doctor's judgment in the true positive rate of model prediction, which proves the segmentation ability of DAResUnet structure. IN 2021, Bouteldja et al. Proposed a modified U-Net model for the segmentation of kidney tissue slice images, including deepening the depth of the model, reducing the number of channels, not halving the number of channels when transposing the convolution, replacing the BN (Batch normalization) layer with the IN (Instance normalization) layer, and using Leaky ReLU as the activation function of the model^[59]. The modified U-Net is used to segment kidney tissue images, and the results show that its segmentation performance is better than that of the ordinary U-Net and Context Encoder structure. In the same year, in order to reduce the interference of fat, muscle and other tissues outside the lymph node region in the lymph node tissue slice image on the cancer diagnosis results, Wang et al. Constructed a U-Net structure to extract the lymph node region in the tissue slice image^[60].

DeepLab is another widely used segmentation structure. In 2020, Zhang et al. Proposed a novel coronavirus pneumonia diagnosis strategy based on segmentation model for the detection of the current global epidemic of novel coronavirus pneumonia^[61]. In this study, DeepLab v3 was used to segment lung CT images into seven parts: background, lung field, lung consolidation, ground-glass opacity, pulmonary fibrosis, interstitial thickening, and pleural effusion. In order to distinguish between common pneumonia and COVID-19 and improve the generalization ability of the diagnosis model, the authors directly use the output of the segmentation model as the input of the COVID-19 diagnosis model, and this disease diagnosis strategy based on the segmentation model can provide intuitive diagnosis basis. In 2021, Wang et al. Established a deep learning detection system for COVID-19 based on chest X-ray, which includes three modules: anatomical site detection, lung lesion region segmentation and pneumonia detection^[62]. In this study, DeepLab v3 framework was used to detect anatomical sites and segment lesion areas, and the average absolute error of anatomical sites detection was 5. 568 mm, and the Dice coefficient of lung lesion area segmentation was 0. 95.

At present, the research on medical image segmentation is mostly based on fully convolutional neural networks such as U-Net, DeepLab and SegNet, and focuses on the improvement and fine-tuning of the existing network structure. Deep learning segmentation models have shown segmentation accuracy and speed beyond the expert level in most studies, and are expected to be put into clinical application to assist doctors in medical image segmentation tasks.

5.2 Disease diagnosis

The gold standard of disease diagnosis is usually very time-consuming, while other diagnostic methods often have high false negative rate and false positive rate, so a low-cost, fast and accurate disease diagnosis method is needed. Deep learning model has such characteristics, and a large number of studies have used deep learning to diagnose diseases. In 2018, Kermany et al. Analyzed retinal OCT (Optical coherence tomography) images based on transfer learning, and realized the diagnosis of three eye diseases^[12]. In this study, the Inception v3 model was trained based on more than 100,000 retinal OCT images of more than 4,000 patients through transfer pre-training, and the prediction accuracy on the test set reached 96.1%. The interpretation results of Occlusion testing on the model diagnosis were consistent with expert judgment.

In practical research, the collection and labeling of a large amount of data brings huge costs to the establishment of disease diagnosis models, and because the prediction results of deep learning models are usually difficult to interpret, the users of the models can not measure their accuracy in clinical practice. The demand for data volume and the "black box" characteristics of deep learning seriously hinder its clinical application, so it is necessary to build an interpretable deep learning model with only a small amount of data, while ensuring the diagnostic performance of the model. In 2019, Lee et al. Established a deep learning model for the diagnosis of acute intracerebral hemorrhage based on small-scale heterogeneous samples^[63]. The author collected more than 5000 brain CT images of patients with different types of cerebral hemorrhage and more than 9000 healthy brain CT images, and established a deep learning model for the uneven data. In order to improve the performance of the model, the author proposes three model training strategies. Firstly, the weighted binary cross entropy is used as the loss function to deal with uneven samples, so that the loss is higher when the number of samples is small, so as to balance the impact of different categories on the model training process. Then transfer learning is used to improve model performance and accelerate model convergence. Finally, in order to improve the generalization performance of the model, the training samples are expanded by data enhancement strategies such as cropping, rotation, interpolation and noise addition during model training. With the above three strategies, the model achieves an expert-level diagnostic accuracy. In addition, the authors adopted the prediction mechanism of the CAM algorithm visualization model, which was confirmed by experts. In 2020, Zhou et al. Constructed a CRCnet model to analyze colonoscopy images to diagnose colorectal cancer, and the AUPRC (Area under precision-recall curve) of the model on the test set reached 0.99^[64]. The authors used the grad-CAM algorithm to calculate the area of interest in the model's prediction and demonstrated that grad-CAM was able to correctly locate most of the lesion area. Several other studies have used the grad-CAM algorithm to explain the model. In 2021, Gehrung et al. Transferred the Whole slide image (WSI) based on esophageal cells and compared various CNN models to detect Barrett's esophagus. In addition, the authors also used the grad-CAM algorithm to visualize the importance of each region of the input data^[65]. Qian et al. Constructed a CNN model for diagnosing breast cancer based on ultrasound images, and located the lesion area through grad-CAM algorithm^[66].

In 2021, Liu et al. Established a deep learning model for identifying skin diseases, which integrates skin images of patients and 45 basic conditions and medical history information for diagnosis^[67]. The authors used the Integrated gradients algorithm to explain the model prediction results, and used the Permutation feature importance method to rank the importance of the patient's basic information and medical history information, and obtained several features that had the greatest impact on the model prediction. Although it is a mainstream research trend to train disease diagnosis models based on a large number of labeled samples, some studies have tried to learn the information of unlabeled samples to reduce the cost of data labeling. In 2021, Yu et al. Used the "mean teacher" semi-supervised learning strategy to train the deep learning model, and used it to analyze the pathological images of colorectal cancer tissues and healthy tissues to complete the diagnostic task^[68]. The semi-supervised learning strategy enables unlabeled samples to be used to optimize the model, and the results show that the model trained based on the semi-supervised learning strategy is significantly better than the supervised learning with the same number of labeled samples.

Deep learning has broad application prospects as a low-cost, rapid and high-precision disease diagnosis method. A large number of studies have tried to achieve disease diagnosis through deep learning, but due to the high cost of sample collection, labeling and the complex "black box" characteristics of deep learning models, it is still very difficult to put them into clinical application. Future research not only needs to break through the dual barriers of sample size and interpretability, but also needs to continuously optimize and improve the model structure and training strategy in order to achieve the gold standard level of diagnosis.

6 Conclusion and prospect

In recent years, with the explosive growth of data size and the significant improvement of computing power, the theoretical and applied research of deep learning has made tremendous progress, and deep learning has been widely used in various fields to complete large-scale data analysis tasks. In the field of spectral analysis, deep learning can not only achieve end-to-end high-precision qualitative and quantitative analysis to simplify the analysis steps, but also discover hidden spectral information, thus promoting the development of basic research. In the field of medical image analysis, by combining deep learning with interpretability methods, it can provide a model diagnosis mechanism while achieving high-precision disease diagnosis, which greatly broadens the application prospects of deep learning. At present, most of the research on deep learning applications belongs to the category of supervised learning, and the collection of large-scale labeled samples brings huge costs to the training model, which hinders the practical application of deep learning. Future research still needs to continuously optimize the model training strategy for small-scale samples. The "black box" nature of deep learning is also an important factor that hinders its practical application. Although there are a large number of algorithms for explaining deep learning models, most of the studies focus on building external algorithm explanation models after training, while few studies try to build explainable deep learning models directly, and most of the explainable models are at the expense of the accuracy of model prediction. In addition, the performance of most external interpretation algorithms in practical applications is uneven, and there is a lack of evaluation criteria for interpretation algorithms. The main purpose of most studies using interpretability methods is to prove the effectiveness of the model, resulting in the application background of interpretability methods is relatively single. Future research not only needs to develop more applicable external interpretation algorithms and establish evaluation criteria for interpretability methods, but also should broaden the application background of interpretability methods.For example, the knowledge injection of neural network is realized by combining the interpretability method, so that the neural network has both interpretability and high accuracy, and the understanding of data is deepened through the attention mechanism and other methods. In a word, deep learning has great application potential in spectral and medical image analysis, and end-to-end deep learning methods will completely change the traditional spectral and medical image analysis mode. In order to put deep learning into practice, future research should focus on training strategies based on small-scale data, methods to enhance model interpretability, and the construction of interpretable models.

References

Publishing order | Descend order by publishing year | Descend order by cited within

[1]	Miotto R, Wang F, Wang S, Jiang X Q, Dudley J T. Brief. Bioinform., 2018, 19(6): 1236.

[2]	Senior A W, Evans R, Jumper J, Kirkpatrick J, Sifre L, Green T, Qin C L, Žídek A, Nelson A W R, Bridgland A, Penedones H, Petersen S, Simonyan K, Crossan S, Kohli P, Jones D T, Silver D, Kavukcuoglu K, Hassabis D. Nature, 2020, 577(7792): 706.

[3]	Jing R Y, Li Y Z, Xue L, Liu F J, Li M L, Luo J S. J. Chem. Inf. Model., 2020, 60(8): 3755.

[4]	Yang X, Wang Y F, Byrne R, Schneider G, Yang S Y. Chem. Rev., 2019, 119(18): 10520.

[5]	Zhang X L, Lin T, Xu J F, Luo X, Ying Y B. Anal. Chimica Acta, 2019, 1058: 48.

[6]	Li W Z, Miao W, Cui J X, Fang C, Su S T, Li H Z, Hu L H, Lu Y H, Chen G H. J. Chem. Inf. Model., 2019, 59(5): 1849.

[7]	Li W Z, Wang D H, Yang Z R, Zhang H J, Hu L H, Chen G H. J. Chem. Inf. Model., 2021, acs.jcim.1c01305.

[8]	Yang J, Xu J F, Zhang X L, Wu C Y, Lin T, Ying Y B. Anal. Chimica Acta, 2019, 1081: 6.

[9]	Shao X G, Leung A K M, Chau F T. Acc. Chem. Res., 2003, 36(4): 276.

[10]	Cai W S, Li Y K, Shao X G. Chemom. Intell. Lab. Syst., 2008, 90(2): 188.

[11]	Zhang J, Cui X Y, Cai W S, Shao X G. Sci. China Chem., 2019, 62(2): 271.

[12]

Kermany D S, Goldbaum M, Cai W J, Valentim C C S, Liang H Y, Baxter S L, McKeown A, Yang G, Wu X K, Yan F B, Dong J, Prasadha M K, Pei J, Ting M Y L, Zhu J, Li C, Hewett S, Dong J, Ziyar I, Shi A, Zhang R Z, Zheng L H, Hou R, Shi W, Fu X, Duan Y O, Huu V A N, Wen C, Zhang E D, Zhang C L, Li O L, Wang X B, Singer M A, Sun X D, Xu J, Tafreshi A, Lewis M A, Xia H M, Zhang K. Cell, 2018, 172(5): 1122.

[13]	Wang P, Xiao X, Glissen Brown J R, Berzin T M, Tu M T, Xiong F, Hu X, Liu P X, Song Y, Zhang D, Yang X, Li L P, He J, Yi X, Liu J J, Liu X G. Nat. Biomed. Eng., 2018, 2(10): 741.

[14]	Shi Z, Miao C C, Schoepf U J, Savage R H, Dargis D M, Pan C W, Chai X, Li X L, Xia S, Zhang X, Gu Y, Zhang Y G, Hu B, Xu W D, Zhou C S, Luo S, Wang H, Mao L, Liang K M, Wen L L, Zhou L J, Yu Y Z, Lu G M, Zhang L J. Nat. Commun., 2020, 11: 6090.

[15]	He H, Yan S, Lyu D Y, Xu M X, Ye R Q, Zheng P, Lu X Y, Wang L, Ren B. Anal. Chem., 2021, 93(8): 3653.

[16]	Shen D G, Wu G R, Suk H I. Annu. Rev. Biomed. Eng., 2017, 19: 221.

[17]	McCulloch W S, Pitts W. Bull. Math. Biophys., 1943, 5(4): 115.

[18]	Rosenblatt F. Psychol. Rev., 1958, 65(6): 386.

[19]	Rumelhart D E, Hinton G E, Williams R J. California Univ San Diego La Jolla Inst for Cognitive Science, 1985, 1.

[20]	Long J, Shelhamer E, Darrell T. 2015 IEEE Conference on Computer Vision and Pattern Recognition. Boston, MA, USA. IEEE, 2015, 3431.

[21]	Ronneberger O, Fischer P, Brox T. International Conference on Medical Image Computing and Computer-Assisted Intervention, 2015, 234.

[22]	Hochreiter S, Schmidhuber J. Neural Comput., 1997, 9(8): 1735.

[23]	Cho K, Van Merriënboer B, Gulcehre C, Bahdanau D, Bougares F, Schwenk H, Bengio Y. arXiv preprint arXiv:1406.1078, 2014.

[24]	Bahdanau D, Cho K, Bengio Y. arXiv preprint arXiv:1409.0473, 2014.

[25]	Luong M-T, Pham H, Manning C D. arXiv preprint arXiv:1508.04025, 2015.

[26]	Vaswani A, Shazeer N, Parmar N, Uszkoreit J, Jones L, Gomez A N, Kaiser Ł, Polosukhin I. Advances in neural information processing systems, 2017.

[27]	Goodfellow I, Pouget-Abadie J, Mirza M, Xu B, Warde-Farley D, Ozair S, Courville A, Bengio Y. Advances in neural information processing systems, 2014.

[28]	Isola P, Zhu J Y, Zhou T H, Efros A A. 2017 IEEE Conference on Computer Vision and Pattern Recognition. Honolulu, HI, USA. IEEE, 2017, 5967.

[29]	Zhu J Y, Park T, Isola P, Efros A A. 2017 IEEE International Conference on Computer Vision. Venice, Italy. IEEE, 2017, 2242.

[30]	Zhou B L, Khosla A, Lapedriza A, Oliva A, Torralba A. 2016 IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas, NV, USA. IEEE,2016, 2921.

[31]	Selvaraju R R, Cogswell M, Das A, Vedantam R, Parikh D, Batra D. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2017, 618.

[32]	Sundararajan M, Taly A, Yan Q Q. International Conference on Machine Learning, 2017, 3319.

[33]	Zeiler M D, Fergus R. European Conference on Computer Vision, 2014, 818.

[34]	Fong R C, Vedaldi A. Proceedings of the IEEE/CVF International Conference on Computer Vision, 2017, 3429.

[35]	Zintgraf L M, Cohen T S, Adel T, Welling M. arXiv preprint arXiv:1702.04595, 2017.

[36]	Ribeiro M T, Singh S, Guestrin C.Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. San Francisco California USA. New York, NY, USA: ACM, 2016, 1135.

[37]	Lundberg S M, Lee S-I.Proceedings of the 31st International Conference on Neural Information Processing Systems, 2017, 4768.

[38]	Frosst N, Hinton G. arXiv preprint arXiv:1711.09784, 2017.

[39]	Xu K, Ba J, Kiros R, Cho K, Courville A, Salakhudinov R, Zemel R, Bengio Y. International Conference on Machine Learning, 2015, 2048.

[40]	Zellers R, Bisk Y, Farhadi A, Choi Y. 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Long Beach, CA, USA. IEEE, 2019, 6713.

[41]	Alvarez-Melis D, Jaakkola T S. arXiv preprint arXiv:1806.07538, 2018.

[42]	Bian X H, Li S J, Shao X G, Liu P. Chemom. Intell. Lab. Syst., 2016, 158: 174.

[43]	Zhang M, Cai W S, Shao X G. Anal., 2011, 136(20): 4217.

[44]	Ju L, Lyu A H, Hao H X, Shen W, Cui H. Anal. Chem., 2019, 91(15): 9343.

[45]	Ni C, Wang D Y, Tao Y. Spectrochim. Acta, Part A, 2019, 209: 32.

[46]	Zhang C, Wu W Y, Zhou L, Cheng H, Ye X Q, He Y. Food Chem., 2020, 319: 126536.

[47]	Xiong Y R, Yang W Y, Liao H Y, Gong Z L, Xu Z Z, Du Y P, Li W. Chemom. Intell. Lab. Syst., 2022, 223: 104532.

[48]	Huang G Z, Yuan L M, Shi W, Chen X, Chen X J. Food Chem., 2022, 372: 131219.

[49]	Shin H, Oh S, Hong S, Kang M, Kang D, Ji Y G, Choi B H, Kang K W, Jeong H, Park Y, Hong S, Kim H K, Choi Y. ACS Nano, 2020, 14(5): 5435.

[50]	Shu C, Yan H S, Zheng W, Lin K, James A, Selvarajan S, Lim C M, Huang Z W. Anal. Chem., 2021, 93(31): 10898.

[51]	Huang J L, Wen J X, Zhou M J, Ni S, Le W, Chen G, Wei L, Zeng Y, Qi D J, Pan M, Xu J N, Wu Y, Li Z Y, Feng Y L, Zhao Z Q, He Z B, Li B, Zhao S N, Zhang B H, Xue P L, He S S, Fang K, Zhao Y Y, Du K. Anal. Chem., 2021, 93(26): 9174.

[52]	Yu S X, Li X, Lu W L, Li H F, Fu Y V, Liu F H. Anal. Chem., 2021, 93(32): 11089.

[53]	Zhou L, Zhang C, Taha M F, Wei X H, He Y, Qiu Z J, Liu Y F. Front. Plant Sci., 2020, 11: 575810.

[54]	Yang S, Li C X, Mei Y, Liu W, Liu R, Chen W L, Han D H, Xu K X. Front. Nutr., 2021, 8: 680627.

[55]	Ma Z R, Wang F F, Wang W Z, Zhong Y T, Dai H J. Deep learning for in vivo near-infrared imaging. Proc. Natl. Acad. Sci. U. S. A., 2021, 118(1): e2021446118.

[56]	Guo S X, Mayerhöfer T, Pahlow S, Hübner U, Popp J, Bocklitz T. Anal., 2020, 145(15): 5213.

[57]	Badrinarayanan V, Handa A, Cipolla R. arXiv preprint arXiv:1505.07293, 2015.

[58]	Fu J, Liu J, Tian H J, Li Y, Bao Y J, Fang Z W, Lu H Q. 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Long Beach, CA, USA. IEEE, 2019, 3141.

[59]	Bouteldja N, Klinkhammer B M, Bülow R D, Droste P, Otten S W, von Stillfried S F, Moellmann J, Sheehan S M, Korstanje R, Menzel S, Bankhead P, Mietsch M, Drummer C, Lehrke M, Kramann R, Floege J, Boor P, Merhof D. J. Am. Soc. Nephrol., 2021, 32(1): 52.

[60]	Wang X D, Chen Y, Gao Y S, Zhang H Q, Guan Z H, Dong Z, Zheng Y X, Jiang J R, Yang H Q, Wang L M, Huang X M, Ai L R, Yu W L, Li H W, Dong C S, Zhou Z, Liu X Y, Yu G Z. Nat. Commun., 2021, 12: 1637.

[61]

Zhang K, Liu X H, Shen J, Li Z H, Sang Y, Wu X W, Zha Y F, Liang W H, Wang C D, Wang K, Ye L S, Gao M, Zhou Z G, Li L, Wang J, Yang Z H, Cai H M, Xu J, Yang L, Cai W J, Xu W Q, Wu S X, Zhang W, Jiang S P, Zheng L H, Zhang X, Wang L, Lu L, Li J M, Yin H P, Wang W, Li O L, Zhang C, Liang L, Wu T, Deng R Y, Wei K, Zhou Y, Chen T, Lau J Y N, Fok M, He J X, Lin T X, Li W M, Wang G Y. Cell, 2020, 181(6): 1423.

[62]

Wang G Y, Liu X H, Shen J, Wang C D, Li Z H, Ye L S, Wu X W, Chen T, Wang K, Zhang X, Zhou Z G, Yang J, Sang Y, Deng R Y, Liang W H, Yu T, Gao M, Wang J, Yang Z H, Cai H M, Lu G M, Zhang L Y, Yang L, Xu W Q, Wang W, Olvera A, Ziyar I, Zhang C, Li O L, Liao W H, Liu J, Chen W, Chen W, Shi J C, Zheng L H, Zhang L J, Yan Z H, Zou X G, Lin G P, Cao G Q, Lau L L, Mo L, Liang Y, Roberts M, Sala E, Schönlieb C B, Fok M, Lau J Y N, Xu T, He J X, Zhang K, Li W M, Lin T X. Nat. Biomed. Eng., 2021, 5(8): 509.

[63]	Lee H, Yune S, Mansouri M, Kim M, Tajmir S H, Guerrier C E, Ebert S A, Pomerantz S R, Romero J M, Kamalian S, Gonzalez R G, Lev M H, Do S. Nat. Biomed. Eng., 2019, 3(3): 173.

[64]	Zhou D J, Tian F, Tian X D, Sun L, Huang X H, Zhao F, Zhou N, Chen Z Y, Zhang Q, Yang M, Yang Y C, Guo X X, Li Z B, Liu J, Wang J F, Wang J F, Wang B M, Zhang G L, Sun B C, Zhang W, Kong D L, Chen K X, Li X C. Nat. Commun., 2020, 11: 2961.

[65]	Gehrung M, Crispin-Ortuzar M, Berman A G, O’Donovan M, Fitzgerald R C, Markowetz F. Nat. Med., 2021, 27(5): 833.

[66]	Qian X J, Pei J, Zheng H, Xie X X, Yan L, Zhang H, Han C G, Gao X, Zhang H Q, Zheng W W, Sun Q, Lu L, Shung K K. Nat. Biomed. Eng., 2021, 5(6): 522.

[67]	Liu Y, Jain A, Eng C, Way D H, Lee K, Bui P, Kanada K, de Oliveira Marinho G, Gallegos J, Gabriele S, Gupta V, Singh N, Natarajan V, Hofmann-Wellenhof R, Corrado G S, Peng L H, Webster D R, Ai D, Huang S J, Liu Y, Dunn R C, Coz D. Nat. Med., 2020, 26(6): 900.