In recent years, the boom in research and application of deep learning continues to climb, deep learning framework endless variety of open source, including TensorFlow, Keras, MXNet, PyTorch, CNTK, Theano, Caffe, DeepLearning4, Lasagne, Neon, and so on. Google, Microsoft and other commercial giants have joined the war deep learning framework, undoubtedly the most mainstream of the current framework TensorFlow, Keras, MXNet, PyTorch, followed me from several different aspects of these four major depth learning framework simple comparison.
I. Introduction
TensorFlow：
TensorFlow is the second generation of artificial intelligence-based learning system Google Brain DistBelief research and development, and its name derived from the operational principle itself is released under the Apache 2.0 open source license on November 9, 2015, and in December 2017 pre-release dynamic map mechanism Eager Execution.
Keras:
Keras is written in Python open source neural network library that can TensorFlow, CNTK, running on Theano or MXNet. Depth experiments designed to achieve rapid neural network, it focuses on user-friendly, modular and scalable. The main author and maintainer is Google engineer FrançoisChollet.
MXNet:
MXNet is DMLC (Distributed Machine Learning Community) development of an open source, lightweight, portable, and flexible depth learning library, which allows users to mix symbolic programming mode and imperative programming model to maximize efficiency and flexibility, AWS is now officially recommended depth learning framework. MXNet Many authors are Chinese people, its biggest contribution to the organization to Baidu.
PyTorch:
PyTorch is Facebook on January 18, 2017 issued by the end of the open source python library depth study, based on Torch. Support for dynamic calculation map, provide good flexibility. In May of this year's Developers Conference (2018) on, Facebook announced PyTorch achieve seamless integration with Caffe2 PyTorch1.0 version will coming soon.
For some basic properties of four frame comparison as shown in Table 1-1:
Table 1-1 Properties of each relevant frame
Second, the popularity
Four deep learning libraries are open source, we can see the extent of their popularity in the industry through its data on Github, and ended on June 17, 2018 Github data shown in Table 2-1 and Table 2-2 shows.
Table 2-1
Table 2-2
Third, flexibility
TensorFlow major support in the form of static calculation chart, a graph-structure is intuitive, but is very complicated and troublesome in the debugging process, some errors made more difficult. But by the end of 2017 released a dynamic graph mechanism Eager Execution, adding support for dynamic calculation map, but currently still using the original static calculation graphical based. TensorFlow has TensorBoard application can monitor the operation of the process, visual computing FIG.
Keras is based on a number of different high-level API framework, you can quickly design and modeling, while supporting sequential and functional two design model, it can be quickly changed to the idea of the results, but because of the height of the package, for existing model changes may not be so flexible.
MXNet command supports two programming and declarative manner that supports both static and dynamic calculation map calculation map, and has a good package training function, flexibility and efficiency in one set, has been introduced at the same time similar to MXNet after Keras end high-level interface gluon.
PyTorch FIG dynamically calculated as a typical representative, easy debugging and highly modular, very convenient to build the model, along with its excellent support GPU, parameter data migration between CPU and GPU is flexible
Fourth, ease of learning
对于深度学习框架的学习难易程度以及使用的简易度还是比较重要的，我认为应该主要基于框架本身的语言设计、文档的详细程度以及科技社区的规模考虑。对于框架本身的语言设计来讲，TensorFlow是比较不友好的，与Python等语言差距很大，有点像基于一种语言重新定义了一种编程语言，并且在调试的时候比较复杂。每次版本的更新，TensorFlow的各种接口经常会有很大幅度的改变，这也大大增加了对其的学习时间；Keras是一种高级API，基于多种深度学习框架，追求简洁，快速搭建模型，具有完美的训练预测模块，简单上手，并能快速地将所想变现，十分适合入门或者快速实现。但是学习会很快遇到瓶颈，过度的封装导致对于深度学习知识的学习不足以及对于已有神经网络层的改写十分复杂；MXNet同时支持命令式编程和声明式编程，进行了无缝结合，十分灵活，具备完整的训练模块，简单便捷，同时支持多种语言，可以减去学习一门新主语言的时间。上层接口Gluon也极其容易上手；PyTorch支持动态计算图，追求尽量少的封装，代码简洁易读，应用十分灵活，接口沿用Torch，具有很强的易用性，同时可以很好的利用主语言Python的各种优势。对于文档的详细程度，TensorFlow具备十分详尽的官方文档，查找起来十分方便，同时保持很快的更新速度，但是条理不是很清晰，教程众多；Keras由于是对于不同框架的高度封装，官方文档十分详尽，通俗易懂；MXNet发行以来，高速发展，官方文档较为简单，不是十分详细，存在让人十分迷惑的部分，框架也存在一定的不稳定性；PyTorch基于Torch并由Facebook强力支持，具备十分详细条理清晰的官方文档和官方教程。对于社区，庞大的社区可以推动技术的发展并且便利问题的解决，由Google开发并维护的TensorFlow具有最大社区，应用人员团体庞大；Keras由于将问题实现起来简单，吸引了大量研究人员的使用，具有很大的用户社区；MXNet由Amazon，Baidu等巨头支持，以其完美的内存、显存优化吸引了大批用户，DMLC继续进行开发和维护；PyTorch由Facebook支持，并且即将与Caffe2无缝连接，以其灵活、简洁、易用的特点在发布紧一年多的时间内吸引了大量开发者和研究人员，火爆程度依旧在不断攀升，社区也在不断壮大。
五、 性能
为了比较四个框架的性能（主要是运行速度），我进行了三个不同的实验，对于不同的神经网络以及不同类型的数据集在分别在CPU、GPU环境下进行了测试。
CPU环境：Ubuntu14.04 内存 32GB AMD Opteron(tm) Processor 4284
GPU环境1：Ubuntu16.04 内存 32GB Quadro P2000（5GB显存）
Intel(R) Xeon(R) CPU E5-2637 v4 @ 3.50GHz
GPU环境2：Ubuntu16.04 内存 16GB Tesla K40（12GB显存）
Intel(R) Xeon(R) CPU E5-2650 v3 @ 2.30GHz
代码地址：https://github.com/CircleXing001/DL-tools
以下实验时间均为总训练时间，GPU环境下包括数据由内存复制到GPU的时间，不包括数据读入内存所需的时间。
实验一：基于北京pm2.5数据集的多变量时序数据预测问题
数据集：https://archive.ics.uci.edu/ml/datasets/Beijing+PM2.5+Data
模型：简单的单层LSTM+全连接层，如下图所示：
进行训练50epoches，实验结果如表5-1所示：
表5-1
实验二：基于Mnist数据集的分类问题
模型：两层卷积神经网络+全连接层，如下图所示：
进行训练10epoches，实验结果如表5-2所示：
表5-2
实验三：基于DAQUAR数据集的视觉问答问题
数据集：https://www.mpi-inf.mpg.de/departments/computer-vision-and-multimodal-computing/research/vision-and-language/visual-turing-challenge/
模型：卷积神经网络+LSTM，具体如下图所示：
将数据缩放至50*50，进行训练5epoches，实验结果如表5-3所示：
表5-3
在GPU环境2（Ubuntu16.04+内存 16GB +Tesla K40（12GB显存）+
Intel(R) Xeon(R) CPU E5-2650 v3 @ 2.30GHz）
对上述实验三中224*224数据进行实验，对比四种框架对于硬件（GPU）的利用率，结果见表5-4。
表5-4
通过上述实验我们可以发现，不同的深度学习框架对于计算速度和资源利用率的优化存在一定的差异：Keras为基于其他深度学习框架的高级API，进行高度封装，计算速度最慢且对于资源的利用率最差；在模型复杂，数据集大，参数数量大的情况下，MXNet和PyTorch对于GPU上的计算速度和资源利用的优化十分出色，并且在速度方面MXNet优化处理更加优秀；相比之下，TensorFlow略有逊色，但是对于CPU上的计算加速，TensorFlow表现更加良好。
本文转载自：https://blog.csdn.net/Circlecircle3/article/details/82086396
