Deep Learning (deep learning) study notes finishing series (8)

connect

X. Summary and Outlook

1) Deep learning summary

Deep learning is about algorithms that automatically learn multi-layered (complex) representations of the underlying (implicit) distribution of the data to be modeled. In other words, deep learning algorithms automatically extract low-level or high-level features needed for classification. High-level features, one means that the feature can depend on other features hierarchically (hierarchically), for example: for machine vision, deep learning algorithms learn a low-level representation of it from the original image, such as edge detectors, wavelet filters, etc. , and then build expressions on the basis of these low-level expressions, such as linear or non-linear combinations of these low-level expressions, and then repeat this process, and finally get a high-level expression.

Deep learning can obtain features that better represent data. At the same time, because the model has many layers and parameters, and the capacity is sufficient, the model has the ability to represent large-scale data, so it is not obvious for features such as images and speech (requires manual design and many There is no intuitive physical meaning), and it can achieve better results on large-scale training data. In addition, from the perspective of pattern recognition features and classifiers, the deep learning framework combines features and classifiers into one framework, and uses data to learn features, which reduces the huge workload of manual feature design in use (this is the current industry. Therefore, not only the effect can be better, but also there are many conveniences to use. Therefore, it is a set of frameworks worthy of attention, and everyone who does ML should pay attention to understand.

Of course, deep learning itself is not perfect, nor is it a powerful tool for solving any ML problems in the world, and should not be magnified to an omnipotent level.

2) Deep learning future

There is still a lot of work to be done in deep learning. The current focus is to borrow some methods that can be used in deep learning from the field of machine learning, especially in the field of dimensionality reduction. For example, one of the current work is sparse coding, which uses compressed sensing theory to reduce the dimension of high-dimensional data, so that a vector with very few elements can accurately represent the original high-dimensional signal. Another example is semi-supervised popularity learning, by measuring the similarity of training samples and projecting this similarity in high-dimensional data into a low-dimensional space. Another more inspiring direction is evolutionary programming approaches, which allow conceptual adaptive learning and core architecture changes by minimizing engineering energy.

Deep learning still has many core problems to be solved:

(1) For a given framework, for how many dimensional inputs does it perform well (maybe millions of dimensions in the case of images)?

(2) Which architecture is effective for capturing short-term or long-term temporal dependencies?

(3) How to fuse multiple perceptual information for a given deep learning architecture?

(4) What are the correct mechanisms to enhance a given deep learning architecture to improve its robustness and invariance to distortion and data loss?

(5) Are there other more effective and theoretically based deep model learning algorithms in terms of models?

Exploring new feature extraction models is worthy of further study. In addition, effective parallel training algorithms are also a direction worthy of research. Current min-batch-based stochastic gradient optimization algorithms are difficult to train in parallel on multiple computers. The usual approach is to use a graphics processing unit to speed up the learning process. However, a single machine GPU is not suitable for large-scale data recognition or similar task datasets. In terms of deep learning application expansion, how to make full use of deep learning to enhance the performance of traditional learning algorithms is still the focus of current research in various fields.

11. References and Deep Learning learning resources ( continuously updated... )

The first is the Weibo of the big cow in the field of machine learning: @玉凯_西 Erqi migrant workers; @teacher wood; @梁斌penny; @张东_machine learning; @dengkan; @big datapidong;

（1）Deep Learning

http://deeplearning.net/

（2）Deep Learning Methods for Vision

http://cs.nyu.edu/~fergus/tutorials/deep_learning_cvpr12/

（3）Neural Network for Recognition of Handwritten Digits[Project]

http://www.codeproject.com/Articles/16650/Neural-Network-for-Recognition-of-Handwritten-Digi

（4）Training a deep autoencoder or a classifier on MNIST digits

http://www.cs.toronto.edu/~hinton/MatlabForSciencePaper.html

（5）Ersatz：deep neural networks in the cloud

http://www.ersatz1.com/

（6）Deep Learning

http://www.cs.nyu.edu/~yann/research/deep/