This article is suitable for beginners to understand the basics of deep learning frameworks, and it is also suitable for beginners to think about various deep learning frameworks.
Article Directory
1. What is the deep learning framework?
2. What are the deep learning frameworks?
3. How to choose the right framework?
1. TensorFlow
2. PyTorch
3. Caffe 1.0
4. Theano
5. MXNet
6. Keras
7. Chainer
4. Reference links
1. What is the deep learning framework?
Deep learning frameworks like Caffe and tensorflow are tools for deep learning. Simply put, they are libraries. You need to import caffe and import tensorflow when programming.
To make a simple analogy, a set of deep learning frameworks is a set of building blocks of this brand, and each component is a part of a certain model or algorithm. You can design how to use building blocks to stack building blocks that match your data set.
2. What are the deep learning frameworks?
Google: TensorFlow
Facebook: pytorch
Baidu: Paddle Paddle
Microsoft: CNTK
Amazon's AWS: MXNet Huawei
: mindspore
First-class technology: oneflow
Kuangshi: MegEngine
Tsinghua University: Jittor
3. How to choose the right framework?
Each framework has its advantages and disadvantages, and the choice needs to be based on the actual needs of your own business: for example, if you need to analyze time series, then you will use the cyclic neural network RNN, and although Caffe and MXNet are used for image convolution processing Very friendly but lacks recurrent neural networks. Google's TensorFlow supports other machine learning algorithms and also supports Reinforcement learning algorithms.
1. TensorFlow
development language:
written based on python and accelerated by C/C++ engine, it is the second-generation deep learning framework open sourced by Google.
Programming Language:
Python is the most convenient client language to work with TensorFlow. However, JavaScript, C++, Java, Go, C#, and Julia also provide experimental interfaces.
advantage:
Dealing with recurrent neural network RNN is very friendly. Its uses go beyond deep learning and can support reinforcement learning and other algorithms.
The internal implementation uses the symbolic graph method of vector operations, and graphs are used to represent computing tasks, which makes it easy to specify new networks and supports rapid development. TF operates using a static computation graph. That is, we first define the graph, then run computations, and if changes need to be made to the architecture, we retrain the model. TF chose this approach for efficiency, but many modern neural network tools are able to allow for improvements in the learning process without significantly slowing down learning. In this regard, TensorFlow's main competitor is PyTorch.
shortcoming:
At present, TensorFlow does not support "inline" matrix operations, and the matrix must be copied to perform operations on it. Copying a huge matrix will reduce the operating efficiency of the system and occupy part of the memory.
TensorFlow does not provide commercial support, it is only a new tool provided by researchers, so companies need to consider open source agreements if they want to commercialize.
2. PyTorch
development language:
Facebook's open source computing framework written in Lua, which supports machine learning algorithms. The main software tool for deep learning after Tensorflow is PyTorch.
Facebook open-sourced Torch's Python API — PyTorch — in January 2017.
advantage:
It has better flexibility and speed. Unlike TensorFlow, the PyTorch library operates with a dynamically updating graph, which means you can make changes along the way. Realized and optimized the basic calculation unit, you can easily implement your own algorithm on this basis, without wasting energy on calculation optimization.
It supports dynamic calculation graphs and can handle variable-length input and output, which is especially suitable for the application of cyclic neural network RNN.
Standard debuggers such as pdb or PyCharm are available in PyTorch.
The process of training a neural network in Pytorch is simple and straightforward.
PyTorch supports data-parallel and distributed learning models, and includes many pre-trained models.
shortcoming:
Pytorch is better for small projects and prototyping. When it comes to cross-platform solutions, TensorFlow seems more suitable. However, it is worth noting that for the same task, users can use the Caffe2 mobile framework introduced in 2017.
3. Caffe 1.0
development language:
A deep learning framework based on C and C++ developed by Jia Yangqing of UC Berkely at the end of 2013, the first mainstream industrial-grade deep learning tool.
advantage:
It has a very good convolutional neural network implementation, especially during the period from 13 to 2016, most of the deep learning papers related to vision have adopted the caffe framework. Caffe is by far the most popular toolkit in the field of computer vision.
shortcoming:
Due to early development and historical problems, the disadvantage of its architecture is that it is not flexible enough.
Lack of support for recurrent network RNN and language modeling, so Caffe is not suitable for other types of deep learning applications such as text, sound or time series data.
4. Theano
development language:
Theano is the veteran of the deep learning framework, written in Python. Theano has derived a large number of Python deep learning libraries, the most famous including Blocks and Keras.
advantage:
The biggest feature is that it is very flexible and suitable for academic research experiments.
There is good support for recurrent networks and language modeling.
shortcoming:
slower.
5. MXNet
development language:
MXNet is mainly written in C/C++. MXNet is an emerging deep learning framework in 2016, so it draws heavily on the advantages and disadvantages of Caffe.
Programming language:
A machine learning framework that provides multiple APIs for languages such as R, Python, and Julia, and is currently used by Amazon Cloud Services as the underlying framework for its deep learning.
This framework supports a large number of languages (C++, Python, R, Julia, JavaScript, Scala, Go, even Perl) from the beginning.
advantage:
The most important feature is that it is an important part of the distributed machine learning general toolkit DMLC, so its distributed ability is strong. It is worth emphasizing that this framework parallelizes very efficiently on multiple GPUs and multiple machines.
MXNet also focuses on flexibility and efficiency.
Documentation is also very detailed.
Emphasis is placed on improving the efficiency of memory usage, and even running tasks such as image recognition on smartphones.
Support for multiple GPUs (optimized computing and fast context switching).
Clean and easy-to-maintain code (Python, R, Scala, and other APIs).
The ability to solve problems quickly (crucial for novices who are learning in depth).
shortcoming:
Like Caffe, it lacks support for cyclic neural network RNN. Compared with NL4J implemented in JAVA, it is not as convenient as JAVA in terms of distribution.
6. Keras
Keras is a deep learning library based on Theano and TensorFlow. The shortcut to using TensorFlow, Theano or CNTK is the advanced Keras shell.
Inspired by the deep learning framework Torch, it has a more intuitive API and is expected to become the standard Python API for developing neural networks in the future.
advantage:
In addition to Tensorflow, Keras also serves as a high-level API on top of other popular low-level libraries such as Theano and CNTK.
Prototyping is very convenient in Keras. User creation of numerous deep learning models in Keras is simplified into one-line functions. Everything has advantages and disadvantages, and this strategy will make Keras less configurable than the underlying framework.
For those who are just starting to learn deep learning, Keras may be the best deep learning framework. It is ideal for learning and recreating simple concepts, allowing beginners to understand the nature of various models and learning processes by using Keras.
Keras is a beautifully written API. The functional characteristics of the API can completely help users build more novel and complex applications. At the same time, Keras does not block access to the underlying framework.
Code written in Keras is more readable and concise.
The Keras model serialization/deserialization API, callbacks, and dataflows using Python generators are very mature.
shortcoming:
Low flexibility, Keras is not as configurable as the underlying framework.
BTW:
Tensorflow is at a lower level: this is where frameworks like MXNet, Theano, and PyTorch live. This is the layer that implements mathematical operations (such as generalized matrix-matrix multiplication) and neural network primitives (such as convolution operations).
Keras at a higher level. At this level, lower-level primitives are used to implement neural network abstractions, such as layers and models. Usually, at this level, other useful APIs such as model saving and model training are also implemented.
7. Chainer
Before the advent of CMU's DyNet and Facebook's PyTorch, Chainer has been the leading neural network framework for dynamic computational graphs or networks that allow variable-length inputs, a popular feature of NLP tasks.
Development language:
Chainer's code is written in pure Python on top of the Numpy and CuPy libraries. Chainer is the first framework (like PyTorch) to use a dynamic architectural model.
Chainer has repeatedly broken records for scaling efficiency when modeling problem solving with neural networks.
advantage:
According to Chainer's benchmarks, Chainer is significantly faster than other Python-oriented frameworks, with TensorFlow being the slowest in the test group including MxNet and CNTK.
Chainer's GPU and GPU data center performance is better than TensorFlow. (TensorFlow is optimized for TPU architecture) Recently, Chainer became the world champion of GPU data center performance.
OOP-like programming style.
4. Reference link
https://www.zhihu.com/question/53465308
https://blog.csdn.net/yeler082/article/details/78755095
https://zhuanlan.zhihu.com/p/157310499
——— ——————————————
Copyright Statement: This article is the original article of CSDN blogger "Chinese Cabbage Who Wants to Become Powerful", and follows the CC 4.0 BY-SA copyright agreement. For reprinting, please attach the original source link and this statement.
Original link: https://blog.csdn.net/weixin_44211968/article/details/120314695