Excerpt from Zhang Yuhong "depth study of the United States", as their study notes, so you can have a understanding of the system to use roughly all of the frame!
Relatively deep learning framework
"We must first of its profits." In fact, applies to the depth of learning "is" there are many, such as Theano, Keras, Caffe Pytorch and so on, they have their own characteristics. Here we compare this several popular deep learning framework were given a brief introduction, in order to provide the reader with a macro cognition.
1. Theano
Theano is a bottom depth deflection learning framework, it opens the precedent frame based on machine learning of arithmetic symbols. Theano supports automatic function gradient calculation with python interface and integrated NumPy. So, strictly speaking, Theano is built on a NumPy python and numerical toolkit.
Compared to TensorFlow, Keras and other frameworks, Theano more academic Fan child, it did not specifically study the depth of the interface. For example, Theano and no hierarchical neural network. Therefore, users need to start from the bottom, doing a lot of work to create a model they need.
After all, the framework is a framework, and ultimately to be able in live, is the last word. Limited by their underlying properties, Theano in live on efficiency, performance was "merely mediocre" when higher development efficiency rising star rush out, it is also the exit date of rivers and lakes.
Theano development began in the early 2007's depth study of today's developers include everyone Joshua Ben Gio (Yoshua Bengio) and his student Ian Goodfellow (Ian Goodfellow, born in 1987, after 80 ", GAN author of the frame). in the September 29, 2017, the Gio announced after the release of version 1.0, Theano the honorable retirement.
While Theano has begun to fade out the stage of history, but its indelible achievements. The biggest "technical heritage" in that it greatly inspired or trained other deep learning framework. To some extent, TensorFlow, Keras have inherited some of its genes.
2. Hard
Keras is a pure python library written in depth study, it provides many advanced neural network API, through a series of configuration, it can work on CNTK (a deep learning development framework developed by Microsoft), Theano and frameworks like TensorFlow . Thus, Keras can be considered in the second package of the frame. That is, if you want to use Keras, must be installed or more of the above frame.
Keras simple syntax, the user can intuitively understand the connection between its instruction, and the function of each module. As a representative of minimalism, Keras is highly encapsulated, in some scenarios, only a few lines of code will be able to construct a neural network to work properly, or with a dozen lines of code to build a AlexNet network (in 2012 ImageNet contest winner Alex Krizhevsky design convolution neural network). This efficiency is another deep learning framework elusive.
It would appear, Keras is fantastic. Keras slogan is "You have just found Keras." (You found Keras), a little "Brief Encounter" flavor. But do not be too optimistic, sometimes, some things are exactly the advantages of its shortcomings, it depends on your point of view from which to look at it.
High degree of encapsulation Keras does allow the user development efficiency becomes high, but just as with the "fool cameras" to take pictures, like, push the button, even without focus, you can shoot the effect can accept photos, but if you I want to manually adjust the camera parameters, shoot personalized, high-quality photos that "fool the camera" like "fool", as powerless. Because of its high degree of encapsulation, you have not given a voice in its personalized settings.
Keras disadvantage is that its encapsulation (also known as black box) brings a certain degree of inflexibility, leading to difficult for a user subscription, and the running speed is not ideal. In addition, compared to TensorFlow community "in full swing", Keras community less active, which leads to part of its document readability is not high.
3. Caffe
Caffe fame, originated from the University of California at Berkeley (University of California, Berkeley) a doctoral thesis written by Jayant clear - a - Caffe: Convolutional architecture for fast feature embedding [3l. As the name suggests, its main purpose is reflected in "feature extraction for fast convolution architecture." This paper is about Caffe, once available, they were of great concern to the world, displayed on "Google Scholar" It has more than 5,000 citations, which can be seen by the high degree of attention. Currently, it has been widely used in industry and academia.
Beginning of the design, Caffe focus only on the field of computer vision processing, therefore, it can be regarded as a special convolution neural network for image processing (CNN) framework. But as it is open source (BSD follow protocol), community staff have been "building blocks", it is getting more perfect versatility, it also started in the field of text, voice and time-series data, it has a remarkable performance.
Caffe an outstanding advantage is that it has a model of a treasure chest -Model Zoo (model zoo), in the garden, bringing together a large number of classic models have been trained, as AlexNet, VGG, Inception, ResNet and so on. As a result, some commonly used models simply no user training, can be used to pick a good use, saving a lot of time training model. If the height of the encapsulation of the achievements of Keras "minimalist", the Model Zoo achievements of Caffe "Speed doctrine."
"No gold Zuse, since Bi has micro-flaws", Caffe naturally, has its shortcomings. First, it is not flexible enough, although the model can be used to use, but if need to make some small changes, require the user to take advantage of CUDA C ++ and update the underlying code.
Secondly, from the birth of their genes, it naturally has excellent support for convolutional neural network (Convolutional Neural Network, CNN), but for recurrent neural networks (Recurrent Neural Network, RNN) and LSTM (Long Short-Term Memory, short or long term memory) support for poor performance.
4. Pytorch
PyTorch, is a deep learning framework developed by Torch7 team, it is very promising rising star. Historically speaking origin, it grew out of another deep learning framework eleven Torcho Torch itself is also very good, but it is the programming language Lua, which is a language used c, scalable lightweight programming language . Precisely because of the small minority of the Lua language, resulting Torch audience is limited, which has become a major obstacle to the development of the Torch.
Currently, the depth of the learning community programming language, the vast majority in order to achieve Python-based. Under this trend, "servers are clever", Torch7 team re-use python developed a Torch, named PyTorch.
PyTorch very young, only open source release in March 2017. Once released, PyTorch it has been warmly supported by the community. Design ideas PyTorch is linear, the code intuitive, easy debugging. Its main advantage lies in that it can support dynamic neural network. We know, for Caffe and TensorFlow and other frameworks, if you want to change the network structure, it is necessary to start from scratch, you need to re-re front wheels again. Unlike these frameworks, PyTorch automatic derivation by means of a reverse count the zero delay allows users to alter the behavior of neural network.
Of course, PyTorch also inadequate. Its biggest shortcomings is "too young." December 2017 launch of the beta or test version, so it is not very mature as a whole. Because it is too new, has also led to community strength is not strong enough. But Houshengkewei, should not be underestimated.
In fact, computing framework for deep learning of another, far more than these before mentioned. For example, there MXNet (Amazon AWS official recommended depth learning engine, the early major developers as the core of Mr. Li Mu), Deepleaming4j (foreign venture company Skymind a product, "4j" meaning more well-known framework is "for Java ", as the name suggests, this is a deep learning framework written for Java users), Lasagne (a), Neon · (deep learning framework Nervana Systems developed lightweight Theano based on neural network library, GPU architecture in Maxwell under faster speed), DIGITS (deep Leaming GPU Training System, is an advanced packaging for Caffe GPU, to complete Caffe related operations in the browser), Chainer (released in 2015 by the Japanese company a deep learning library Preferred Networks ), Leaf (based on a cross-platform language Rust deep learning library). And so added to the list.
According to "there is reasonable" philosophy, the reason why these frameworks can exist, to develop, certainly because they found each market segment has its own existence value. So, readers need according to their own scenarios to choose their own computing framework, mainly on the subject TensorFlow not be confined to this chapter. For example, if your company's products through the development of Java and Java Web form to publish, then TensorFlow not necessarily the most appropriate (of course TensorFlow also provides a Java interface), and a relatively small minority of Deepleaming4j, maybe you is better select.