Some concepts I have read recently, I feel that I have not seen them before, or I am not very impressed, so I will simply record them.
- End-to-end learning (End-to-End Learning), also known as end-to-end training, refers to the overall goal of directly optimizing tasks without performing module or stage training in the learning process. In end-to-end learning, generally The functions of different modules or stages do not need to be explicitly given, and the intermediate process does not require human intervention. The training data for end-to-end learning is in the form of "input-output" pairs, and no other additional information is required. Therefore, end-to-end learning and Like deep learning, it is necessary to solve the problem of contribution distribution. At present, most deep learning using neural network models can also be regarded as a kind of end-to-end learning.
- Automatic differentiation, which is currently mainly used in deep learning, can be seen as an extension of the backpropagation algorithm.
- Deep learning framework (commonly used):
- TensorFlow: A deep learning framework developed by Google that can run on any device with a CPU Torch or GPU. The calculation process of TensorFlow is represented by a data flow graph. The name of TensorFlow comes from the fact that the operation object in its calculation process is a multidimensional array, namely Tensor. TensorFlow version 1.0 uses static calculation graphs, and version 2.0 also supports dynamic calculation graphs;
- PyTorch: A deep learning framework developed and maintained by Facebook, NVIDIA, Twitter and other companies. Its predecessor is Torch4 in Lua language. PyTorch is also a framework based on dynamic computing graphs, which has obvious advantages in tasks that require dynamic changes to the neural network structure.
- PaddlePaddle (PaddlePaddle): An efficient and scalable deep learning framework developed by Baidu, which supports both dynamic and static graphs. Paddle provides powerful deep learning parallel technology, which can support ultra-large-scale depth of dense parameter and sparse parameter scenes at the same time Learning parallel training supports efficient parallel training of hundreds of billions of parameters and hundreds of nodes.
- On top of these basic frameworks, there are some highly modular neural network libraries built on top of these frameworks, making building a neural network model as easy as building blocks. Among them, the more famous modular neural network frameworks are: (1) Keras based on TensorFlow; (2) Lasagne based on Theano.
- The research progress of deep learning is very rapid. Therefore, the latest literature is generally published in academic conferences. Academic conferences related to deep learning mainly include:
- International Conference on Learning Representations (ICLR): Mainly focused on deep learning;
- Annual Conference on Neural Information Processing Systems (NeurIPS): Interdisciplinary conference, but with a focus on machine learning. Mainly including neural information processing, statistical methods, learning theory and applications, etc.;
- International Conference on Machine Learning (ICML): The top machine learning conference, deep learning, as a hot spot in recent years, also occupies a large proportion of ICML;
- International Joint Conference on Artificial Intelligence (IJCAI): the top comprehensive conference in the field of artificial intelligence. It has a long history and has been held since 1969;
- AAAI Conference on Artificial Intelligence (AAAI): The top conference in the field of artificial intelligence, held around February every year, generally in North America.