Content sorting of artificial intelligence and neurons (continuously updated)

1. Background

Over the past few years, deep learning models have been applied to various tasks/applications in different domains. With the maturity and increase of deep learning in different fields, I decided to organize relevant materials for learning.

2. Preparation basis

Neurons (optional) reading

3. Books

1. "Calculus"

1. Function Limit and Continuity
2. Derivatives and Differentiation
3. indefinite integral
4. Definite integral
5. Ordinary Differential Equations and Mathematical Models
6. Function Integrability and Generalized Integral
7. infinite series
8. Power series and Fourier series
9. Space Analytic Geometry
10. Differential Calculus of Multivariate Functions
11. Re-integration
12. Curve and Surface Integrals
13. Vector Field Preliminary
14. Higher Order Linear ODEs

2. "Linear Algebra"

1. determinant
2. system of linear equations
3. Matrices and their operations
4. Linear Space and Linear Transformation
5. Eigenvalues ​​and quadratic forms of matrices
6. vector space
7. Finite-dimensional vector spaces
8. linear map
9. polynomial
10. Eigenvalues ​​and Eigenvectors
11. inner product space
12. Operators on Inner Product Spaces
13. Operators on Complex Vector Spaces
14. Operators on real vector spaces
15. trace and determinant

3. "Statistics"

4.《Deep Learning》

1. Fundamentals of Applied Mathematics and Machine Learning
2. Deep Web and Modern Practice
3. deep learning research

4. Community and resources

1. https://arxiv.org/
2. https://paperswithcode.com/
3. https://www.kaggle.com/

5. Tools and framework

• torch, torch official document
• tensorflow2+, tensorflow official website tutorial

Preheating

1. A Survey of Deep Learning Research
2. A Survey on Deep Learning: Algorithms, Techniques, and Applications

---

6. Learning path

1. Logistic regression, linear regression

$$\begin{gathered} \hat{y}=s(w^{T}x+b),where\sigma \left(z\right)=\frac{1}{1+e^{-z}} \\ x\in {\mathbb{R}}^n(input\; to\; the\; model,\; withnfeatures) \\ y\in [0,1](the\; output\; of\; the\; model,the\; probability\; value) \\ w\in {\mathbb{R}}^n,b\in R(model\; parameters,training\; content) \\ \hat{y}=w^{T}x+b(linear\; regression) \end{gathered}$$

2. Loss function

Example
$$\begin{gathered} featurendimension \\ \begin{array}{c}number\\ according\; to\\ M\\ \end{array}\left|\begin{array}{cccc}a& b& c& \cdots \\ e& f& g& \cdots \\ h& i& j& \cdots \\ & ⋮& & \cdots \end{array}\right|\begin{array}{c}Pass\; rate1\\ Pass\; rate0\\ Pass\; rate1\\ \end{array} \end{gathered}$$

设 $i\in [1,M]$，$\widehat{y_i}$Represents probability, Target=1 means qualified, Target=0 means unqualified.

$$P(Target=1|x_i)=\widehat{y_i}$$
$$P(Target=0|x_i)=1-\widehat{y_i}$$
Loss of function L(w) : $L(w)={\prod }_{}{\widehat{y_i}}^{y_i}{(1-\widehat{y_i})}^{1-y_i}$, among which $y_i$represent the real value.
For the convenience of calculation, the above formula can be obtained after equivalent transformation: $L(w)={\sum }_{}[y_i\ln \left(\widehat{y_i}\right)+(1-y_i)\ln \left(1-\widehat{y_i}\right)]$

pending upgrade

7. Related Papers ¹

Due to the sheer number of works published in the field of deep learning, below is a comprehensive survey of papers published (or under review) in top journals

1. Computer Vision Related Surveys

Here, we provide a list targeting deep learning applications in computer vision.

A). Object Detection

Deep Learning for General Object Detection: A Survey (IJCV, 2019)
Object Detection Using Deep Learning: A Review (IEEE TNNLS, 2019)
A Review of Deep Learning-Based Object Detection (IEEE Access, 2019)
Recent Advances in Deep Learning for Object Detection (Neurocomputing, 2020)

B). Image Segmentation

Image Segmentation Using Deep Learning: A Survey (IEEE PAMI, 2021) A
Survey of Deep Learning Techniques for Semantic Segmentation of Images and Videos (Applied Soft Computing, 2018)

C). Medical Image Segmentation

Deep Learning Techniques for Medical Image Segmentation: Achievements and Challenges (Journal of Digital Imaging, 2018)
Review: Deep Learning for Medical Image Segmentation Using Multimodal Fusion (Elsevier Array, 2019)
Automatic Breast Ultrasound Image Segmentation: A Survey (Pattern Recognition, 2018)

D). Face Recognition

Deep Face Recognition: A Survey (Arxiv, 2019)
Face Recognition Systems: A Survey (Sensors, 2020)

E). Biometrics Recognition

Biometric Recognition Using Deep Learning (AIRE, 2022)
Deep Learning for Biometric Recognition (ACM CSUR, 2018)

F). Image Super-Resolution

Deep Learning for Image Super-Resolution: A Survey (PAMI, 2020)
Super-Resolution Depth Journey: A Survey (ACM CSUR, 2020)

G). Facial Expression Recognition

Deep Facial Expression Recognition: A Survey (Arxiv, 2019)

H). Action Recognition

Insights into Action Recognition: A Survey (Image and Visual Computing, 2017)
Human Action Recognition and Prediction: A Survey (IJCV, 2022)

I). Generative Adversarial Networks (GANs)

Generative Adversarial Networks: A Survey and Taxonomy (Arxiv, 2019)
Generative Adversarial Networks: A Survey of Private and Secure Applications (ACM CSUR, 2021)

J). Image Captioning

A Comprehensive Survey of Deep Learning for Image Captioning (ACM Computing Surveys, 2019)

K). Human Motion Recognition

RGB-D-Based Human Motion Recognition with Deep Learning (Computer Vision and Image Understanding, 2018)

L). Neural Rendering

The State of the Art in Neural Rendering (Computer Graphics Forum, 2020)

2. NLP Related Surveys

NLP tasks have made great progress in the past few years with the help of deep neural architectures, such as: attentional LSTM, Transformer, BERT, GPT models, and XL-Net.

A). General Natural Language Processing (General NLP)

Text Classification Based on Deep Learning: A Comprehensive Review (ACM Computing Surveys, 2021)
Latest Trends in Natural Language Processing Based on Deep Learning (IEEE Computational intelligence magazine, 2018)

B). Dialogue Systems, Conversational AI (Dialogue Systems, Conversational AI)

Approaches to Conversational AI (Foundations and Trends in Information Retrieval®, 2019) A
Survey of Dialogue Systems-Focused Natural Language Generation Techniques – Past, Present, and Future Directions (Arxiv, 2018) A
Survey of Dialogue Systems: Recent Advances and New Domain (Acm Sigkdd Explorations Newsletter, 2017)

C). Embedding

Word Embeddings (Arxiv, 2019)
Neural Information Retrieval: A Literature Review (Arxiv, 2016)

D). Natural Language Generation

A Survey of the State of Natural Language Generation Technologies: Core Tasks, Applications, and Evaluation (Journal of Artificial Intelligence Research, 2018)

E). Text Summarization

Neural Abstraction Text Summarization Using Sequence-to-Sequence Models (Arxiv, 2019)

F). Sentiment Analysis

Deep Learning for Sentiment Analysis (Wiley Interdisciplinary Review: Data Mining and Knowledge Discovery, 2018)
Deep Learning for Aspect-Based Sentiment Analysis: A Comparative Review (Expert Systems with Applications, 2019)
Survey of Social Media Sentiment Analysis (Knowledge and Information Systems, 2019)

G). Named Entity Recognition

A Survey of Recent Advances in Named Entity Recognition with Deep Learning Models (Arxiv, 2019)
A Survey of Deep Learning for Named Entity Recognition (Arxiv, 2018)

H). Answer Selection

A Survey of Deep Learning Techniques Applied to Answer Selection (Proceedings of the 27th International Conference on Computational Linguistics, 2018)

I). NLP based Financial Forecasting

Natural Language-Based Financial Forecasting: A Survey (Artificial Intelligence Review, 2018)

3. Healthcare Related Surveys

deep learning model

A). Medical Image Analysis

A Survey of Deep Learning in Medical Image Analysis (Medical Image Analysis, 2017)

B). Health-Record Analysis

Deep EHR: A Survey of Recent Advances in Deep Learning Techniques for Electronic Health Record (EHR) Analysis (IEEE Journal of Biomedical and Health Informatics, Published 2017)

C). Bioinformatics

A Survey of Data Mining and Deep Learning in Bioinformatics (Journal of Healthcare Systems, 2018)

D). Medical Imaging (MRI - Medical Imaging)

An overview of MRI-centric deep learning for medical imaging (Zeitschrift für Medizinische Physik, 2019)

E). Radiotherapy

A Survey of Deep Learning in Radiation Therapy (Biomedical Computing, 2018)

F). Pharmacogenomics

Deep Learning in Pharmacogenomics: From Gene Regulation to Patient Stratification (Future Medicine, 2019)

G). Cancer Detection and Diagnosis

Deep Learning for Image-Based Cancer Detection and Diagnosis (Pattern Recognition, 2018)

H). Microscopy Image Analysis

Deep Learning in Microscopy Image Analysis (EEE Transactions on Neural Networks and Learning Systems, 2017)

I). Radiology

Deep learning in radiology: a conceptual overview and a survey of state-of-the-art techniques with an MRI focus (Journal of Magnetic Resonance Imaging, 2019)

4. Other Areas Surveys

In recent years, deep learning models have been used in various other fields. Some of the most popular include

A). Reinforcement Learning

Deep Reinforcement Learning: An Overview (Arxiv, 2017)
Deep Reinforcement Learning: A Brief Survey (IEEE Journal of Signal Processing, 2017)

B). Recommender Systems

Deep Learning-Based Recommender Systems: Surveys and New Perspectives (ACM Computing Survey, 2019)

C). Big Data

Big Data Deep Learning Survey (Information Fusion, 2018)

D). Networking

Deep Learning in Mobile and Wireless Networks (IEEE Communications Surveys, 2019)

E). IoT Big Data

Deep learning for IoT big data and streaming analytics: A survey., 2019

F). Anomly Detection

A Survey of Network Anomaly Detection Based on Deep Learning (Cluster Computing, 2017)

G). Remote sensing

A Comprehensive Survey of Deep Learning for Remote Sensing: Theory, Tools, and Challenges for the Community (Journal of Applied Remote Sensing, 2017)

H). Mobile Multimedia

Deep Learning for Mobile Multimedia (ACM Transactions on Multimedia Computing, Communications, and Applications, 2017)

I). Graph theory (Graphs)

Deep Learning for Graphs (Arxiv, 2018)
Graph Embedding Techniques, Applications, and Performance (Knowledge-Based Systems, 2018)

J). Multimodal Learning

Deep Multimodal Learning: A Survey of Recent Advances and Trends (IEEE Signal Processing Magazine, 2017)
Deep Multimodal Representation Learning (IEEE Access, 2019)

K). Agriculture

Deep Learning in Agriculture (Computers and electronics in agriculture, 2018)

---

1. https://github.com/shervinmin/awesome-deep-learning-survey-papers ↩︎