Introduction to mainstream open source deep learning frameworks
Contents of this article:
1. TensorFlow deep learning framework
2. PyTorch deep learning framework
3. Keras deep learning framework
4. Caffe Deep Learning Framework
5. The status of open source frameworks for deep learning in China
6. Comparison of Several Frameworks
Currently, there are many mainstream open source deep learning frameworks for developers to use.
Mainly including TensorFlow, PyTorch, Keras, Caffe, etc.
The following is a detailed introduction and comparison of these frameworks:
1. TensorFlow deep learning framework
TensorFlow (by Google):
TensorFlow is one of the most popular and widely used deep learning frameworks.
TensorFlow is an open source deep learning framework developed by the Google Brain team. It allows developers to create a variety of machine learning models, including convolutional neural networks, recurrent neural networks, and deep neural networks.
TensorFlow uses dataflow graphs to represent computational graphs, where nodes represent mathematical operations and edges represent data flow. Using TensorFlow can take advantage of GPU and distributed computing to accelerate the training process.
The framework has a wide range of application scenarios, including image recognition, natural language processing, speech recognition, recommendation systems, etc. At the same time, TensorFlow also has rich community support and documentation resources, making it easy to learn and use.
- Suitable for a variety of applications, including computer vision, natural language processing, and recommender systems, among others.
- Provides a rich set of tools and libraries for building and training neural network models.
- Support static calculation graph and dynamic calculation graph. Can be optimized during the graph construction phase
- It has powerful distributed computing capabilities.
- Support multiple language interfaces, including Python, C++, Java, etc.
- Provides many advanced operations such as automatic differentiation, data parallelism, etc.
- Easy to deploy and run on various hardware platforms
- Has strong community support and extensive documentation
TensorFlow deep learning example
The following is a sample code that uses TensorFlow to save the current training model and test it on the test set:
In the code, we use a simple linear model and save the model to the my_model.ckpt file in the current directory during training middle. After the training is complete, we use the test set to test the model and output the loss value on the test set.
import tensorflow as tf
import numpy as np
# 初始化数据和标签
train_data = np.random.rand(1000, 10)
train_labels = np.random.rand(1000, 1)
test_data = np.random.rand(200, 10)
test_labels = np.random.rand(200, 1)
# 创建输入占位符和变量
input_ph = tf.placeholder(tf.float32, shape=[None, 10])
labels_ph = tf.placeholder(tf.float32, shape=[None, 1])
weights = tf.Variable(tf.zeros([10, 1]))
bias = tf.Variable(tf.zeros([1]))
# 创建模型和损失函数
output = tf.matmul(input_ph, weights) + bias
loss = tf.reduce_mean(tf.square(output - labels_ph))
# 创建训练操作和初始化操作
train_op = tf.train.GradientDescentOptimizer(0.1).minimize(loss)
init_op = tf.global_variables_initializer()
# 创建Saver对象
saver = tf.train.Saver()
# 训练模型
with tf.Session() as sess:
sess.run(init_op)
for i in range(100):
_, loss_val = sess.run([train_op, loss], feed_dict={input_ph: train_data, labels_ph: train_labels})
print("Epoch {}: loss = {}".format(i+1, loss_val))
# 保存模型
saver.save(sess, "./my_model.ckpt")
# 在测试集上测试模型
test_loss_val = sess.run(loss, feed_dict={input_ph: test_data, labels_ph: test_labels})
print("Test loss = {}".format(test_loss_val))
The result of running the code:
Epoch 1: loss = 0.33758699893951416 Epoch 2: loss = 0.11031775921583176 Epoch 3: loss = 0.09063640236854553 Epoch 4: loss = 0.0888814628124237 Epoch 5: loss = 0.08867537975311279 Epoch 6: loss = 0.08860388398170471 Epoch 7: loss = 0.0885448306798935 Epoch 8: loss = 0.0884876698255539 Epoch 9: loss = 0.0884314551949501 Epoch 10: loss = 0.08837611228227615 Epoch 11: loss = 0.08832161128520966 Epoch 12: loss = 0.08826790004968643 Epoch 13: loss = 0.08821499347686768 Epoch 14: loss = 0.0881628543138504 Epoch 15: loss = 0.08811149001121521 Epoch 16: loss = 0.08806086331605911 Epoch 17: loss = 0.08801095187664032 Epoch 18: loss = 0.08796177059412003 Epoch 19: loss = 0.08791326731443405 Epoch 20: loss = 0.08786546438932419 Epoch 21: loss = 0.08781831711530685 Epoch 22: loss = 0.08777181059122086 Epoch 23: loss = 0.08772596716880798 Epoch 24: loss = 0.08768074214458466 Epoch 25: loss = 0.08763613551855087 Epoch 26: loss = 0.08759212493896484 Epoch 27: loss = 0.08754870295524597 Epoch 28: loss = 0.08750586211681366 Epoch 29: loss = 0.08746359497308731 Epoch 30: loss = 0.08742187172174454 Epoch 31: loss = 0.08738070726394653 Epoch 32: loss = 0.08734006434679031 Epoch 33: loss = 0.08729996532201767 Epoch 34: loss = 0.08726035058498383 Epoch 35: loss = 0.08722126483917236 Epoch 36: loss = 0.0871826633810997 Epoch 37: loss = 0.08714456111192703 Epoch 38: loss = 0.08710692077875137 Epoch 39: loss = 0.08706976473331451 Epoch 40: loss = 0.08703305572271347 Epoch 41: loss = 0.08699680119752884 Epoch 42: loss = 0.08696100115776062 Epoch 43: loss = 0.08692563325166702 Epoch 44: loss = 0.08689068257808685 Epoch 45: loss = 0.0868561640381813 Epoch 46: loss = 0.08682206273078918 Epoch 47: loss = 0.0867883637547493 Epoch 48: loss = 0.08675506711006165 Epoch 49: loss = 0.08672215789556503 Epoch 50: loss = 0.08668963611125946 Epoch 51: loss = 0.08665750920772552 Epoch 52: loss = 0.08662573993206024 Epoch 53: loss = 0.0865943431854248 Epoch 54: loss = 0.08656331896781921 Epoch 55: loss = 0.08653264492750168 Epoch 56: loss = 0.0865023210644722 Epoch 57: loss = 0.08647233247756958 Epoch 58: loss = 0.08644269406795502 Epoch 59: loss = 0.08641338348388672 Epoch 60: loss = 0.08638441562652588 Epoch 61: loss = 0.0863557755947113 Epoch 62: loss = 0.0863274559378624 Epoch 63: loss = 0.08629942685365677 Epoch 64: loss = 0.0862717255949974 Epoch 65: loss = 0.08624432235956192 Epoch 66: loss = 0.0862172394990921 Epoch 67: loss = 0.08619043231010437 Epoch 68: loss = 0.08616391569375992 Epoch 69: loss = 0.08613769710063934 Epoch 70: loss = 0.08611176162958145 Epoch 71: loss = 0.08608610183000565 Epoch 72: loss = 0.08606073260307312 Epoch 73: loss = 0.08603561669588089 Epoch 74: loss = 0.08601076900959015 Epoch 75: loss = 0.08598621189594269 Epoch 76: loss = 0.08596190065145493 Epoch 77: loss = 0.08593783527612686 Epoch 78: loss = 0.08591403067111969 Epoch 79: loss = 0.08589048683643341 Epoch 80: loss = 0.08586718887090683 Epoch 81: loss = 0.08584412187337875 Epoch 82: loss = 0.08582130074501038 Epoch 83: loss = 0.0857987329363823 Epoch 84: loss = 0.08577638119459152 Epoch 85: loss = 0.08575427532196045 Epoch 86: loss = 0.08573239296674728 Epoch 87: loss = 0.08571072667837143 Epoch 88: loss = 0.08568929135799408 Epoch 89: loss = 0.08566807210445404 Epoch 90: loss = 0.08564707636833191 Epoch 91: loss = 0.08562628924846649 Epoch 92: loss = 0.08560571074485779 Epoch 93: loss = 0.0855853483080864 Epoch 94: loss = 0.08556520193815231 Epoch 95: loss = 0.08554524928331375 Epoch 96: loss = 0.0855254977941513 Epoch 97: loss = 0.08550594002008438 Epoch 98: loss = 0.08548659086227417 Epoch 99: loss = 0.08546742796897888 Epoch 100: loss = 0.08544846624135971 Test loss = 0.09260907769203186
2. PyTorch deep learning framework
PyTorch (Facebook open source):
PyTorch is another very popular deep learning framework.
PyTorch is a deep learning framework open sourced by Facebook, and it is one of the most popular deep learning frameworks on the market. It is based on the Python language and provides powerful GPU acceleration functions and support for dynamic computing graphs.
PyTorch has a wide range of applications, including image and speech recognition, natural language processing, computer vision, recommendation systems, and more.
PyTorch has the characteristics of ease of use, high flexibility, and good code readability, making it one of the preferred frameworks for deep learning research and applications.
- Easy to accelerate training on GPU, with excellent GPU acceleration performance.
- Provides a wide range of pre-trained models and toolkits,
- Tensor library for deep learning with GPUs and CPUs.
- Emphasis on the construction of dynamic calculation graphs allows for more flexible model adjustment and debugging, making model construction and debugging more intuitive.
- It excels in flexibility and ease of use, making it ideal for research and prototyping.
- With rich tools and libraries, such as Torch is easy to use.
- Provide a concise and flexible API to reduce the amount of code writing
- Backed by an active community and detailed documentation
PyTorch Deep Learning Examples
# Fit a third-order polynomial to a sine function using PyTorch tensors. Implement forwarding and backwards through the network manually:
#使用 PyTorch 张量将三阶多项式拟合到正弦函数。
#手动实现转发 并向后通过网络：
# -*- coding: utf-8 -*-
import torch
import math
dtype = torch.float
device = torch.device("cpu")
# device = torch.device("cuda:0") # Uncomment this to run on GPU
# Create random input and output data
x = torch.linspace(-math.pi, math.pi, 2000, device=device, dtype=dtype)
y = torch.sin(x)
# Randomly initialize weights
a = torch.randn((), device=device, dtype=dtype)
b = torch.randn((), device=device, dtype=dtype)
c = torch.randn((), device=device, dtype=dtype)
d = torch.randn((), device=device, dtype=dtype)
learning_rate = 1e-6
for t in range(2000):
# Forward pass: compute predicted y
y_pred = a + b * x + c * x ** 2 + d * x ** 3
# Compute and print loss
loss = (y_pred - y).pow(2).sum().item()
if t % 100 == 99:
print(t, loss)
# Backprop to compute gradients of a, b, c, d with respect to loss
grad_y_pred = 2.0 * (y_pred - y)
grad_a = grad_y_pred.sum()
grad_b = (grad_y_pred * x).sum()
grad_c = (grad_y_pred * x ** 2).sum()
grad_d = (grad_y_pred * x ** 3).sum()
# Update weights using gradient descent
a -= learning_rate * grad_a
b -= learning_rate * grad_b
c -= learning_rate * grad_c
d -= learning_rate * grad_d
print(f'Result: y = {a.item()} + {b.item()} x + {c.item()} x^2 + {d.item()} x^3')
The result of running the code:
99 2351.4306640625 199 1585.7086181640625 299 1071.2376708984375 399 725.2841796875 499 492.4467468261719 599 335.59881591796875 699 229.84210205078125 799 158.46621704101562 899 110.2466812133789 999 77.63826751708984 1099 55.56399154663086 1199 40.60529327392578 1299 30.45751953125 1399 23.5659236907959 1499 18.880510330200195 1599 15.69140911102295 1699 13.518349647521973 1799 12.035942077636719 1899 11.023494720458984 1999 10.331212043762207 Result: y = 0.030692655593156815 + 0.8315182328224182 x + -0.005294993054121733 x^2 + -0.08974269032478333 x^3
3. Keras deep learning framework
Keras (Google):
Keras (Google) (originally developed by François Chollet, now the official TensorFlow API):
Keras is an easy-to-use and powerful high-level neural network API written in Python that can run with TensorFlow, CNTK, or Theano as a backend.
Keras has been developed with a focus on enabling rapid experimentation. Being able to translate your ideas into experimental results with minimal delay is key to good research.
- Keras supports multiple backend engines and can run on frameworks such as Tensorflow, Theano, and CNTK.
- Provides a concise and easy-to-use high-level API, especially suitable for beginners and rapid prototyping
- With a wide range of model libraries, pre-trained models and various toolkits, it makes model building more efficient.
- Can seamlessly switch to TensorFlow to enjoy its powerful features and ecosystem
- Allows for easy and fast prototyping (due to user-friendliness, high modularity, scalability).
- Both Convolutional Neural Networks and Recurrent Neural Networks are supported, as well as combinations of both.
- Runs seamlessly on both CPU and GPU.
Guiding Principles
- user friendly . Keras is an API designed for humans, not machines. It puts user experience front and center. Keras follows best practices for reducing cognitive difficulties: it provides a consistent and simple API, minimizes the number of user actions required for common use cases, and provides clear and actionable feedback when users make mistakes.
- Modular . A model is understood as a sequence or diagram of independent, fully configurable modules. These modules can be assembled together with as few constraints as possible. In particular, neural network layers, loss functions, optimizers, initialization methods, activation functions, and regularization methods are all modules that can be combined to build new models.
- Easy scalability . New modules are easily added (as new classes and functions), and existing modules already provide sufficient examples. Keras is more suitable for advanced research due to the ease of creating new modules that can improve expressiveness.
- Implemented based on Python . Keras does not have a separate configuration file in a specific format. Models are defined in Python code, which is compact, easy to debug, and easy to extend.
4. Caffe Deep Learning Framework
Caffe (Berkeley)
The full name of Caffe is Convolutional Architecture for Fast Feature Embedding, which means "convolutional architecture for feature extraction". It is a clear and efficient deep learning framework, and the core language is C++.
Caffe is a popular deep learning framework developed by researchers at the University of California, Berkeley, for the training and deployment of convolutional neural networks (CNNs) and other deep learning models.
The main advantages of Caffe are speed, ease of use and high portability.
It has been widely used in fields such as computer vision, natural language processing, and speech recognition.
Caffe also has a strong community that provides many pre-trained models and visualization tools, allowing users to easily build their own deep learning models.
- Caffe is a C++-based deep learning framework designed to perform convolution operations efficiently.
- It is particularly suitable for computer vision tasks and performs well in image classification and object detection.
- Caffe provides simple configuration files to define the network structure and hyperparameters.
- With efficient GPU acceleration, it is suitable for training models on large-scale datasets.
5. The status of open source frameworks for deep learning in China
- China's deep learning open source framework market has formed a pattern of three strong players
IDC, an international authoritative data research organization, released the report "China's Deep Learning Framework and Platform Market Share, 2022H2". The report shows that Baidu ranks No. 1 in China's deep learning platform market in terms of overall market share, further expanding its lead. China's deep learning open source framework market has formed a top three structure, and the top three framework market shares exceed 80%.
6. Comparison of Several Frameworks
Comparison table of several frameworks
Currently the most popular deep learning frameworks include TensorFlow, PyTorch, and Caffe.
According to the "2019 AI and Deep Learning Market Survey Report" released by market research company O'Reilly, TensorFlow is the most popular deep learning framework, with 57.2% of respondents using it. PyTorch was a close second with 37.1% of respondents using it. Caffe and Keras are also popular, capturing 16.2% and 13.7% of the market, respectively.
| Comparison of the market share of several common deep learning frameworks (2021) |
||
| TensorFlow： |
more than 40%, |
It is one of the most popular deep learning frameworks. |
| PyTorch： |
more than 25%, |
Developed and maintained by Facebook, it has gradually gained attention and been widely used in recent years. |
| Hard: |
more than 10%, |
Often used with TensorFlow, providing a simpler and easier-to-use framework. |
| Caffe： |
about 5%, |
The market applies to areas such as computer vision and image processing. |
| MXNet： |
about 5%, |
The marketplace is developed and maintained by Amazon for large-scale distributed deep learning. |
A simple comparison table of the main features of these frameworks
| TensorFlow |
PyTorch |
Hard |
|
| Computation graph |
static image |
dynamic picture |
static image |
| language interface |
Python, C++, Java, etc. |
Python |
Python |
| API |
Rich |
concise |
concise |
| hardware support |
widely |
dynamic picture |
limited |
| community support |
powerful |
active |
active |
| frame |
static image / dynamic graph |
diversification Application field |
flexibility and ease of use |
GPU Acceleration performance |
pre-trained model and toolkit |
| TensorFlow |
static image |
widely used |
medium |
excellent |
Rich |
| PyTorch |
dynamic picture |
widely used |
outstanding |
excellent |
Rich |
| Hard |
static image |
widely used |
excellent |
outstanding |
Rich |
| Caffe |
static image |
computer vision |
medium |
medium |
generally |
It should be noted that each of these frameworks has advantages and disadvantages, and may have different optimal choices in different application scenarios. Therefore, when choosing a framework, it is recommended to decide according to project needs and research directions, programming skills and personal preferences, conduct evaluation and comparison, and finally choose a specific framework.
7. Other statistical data
The top three options that .NET (5+) users will expect to use next year are .NET (5+), .NET MAUI, and .NET Framework (1.0 - 4.8). .NET is very partial within their community.
Recommended reading:
|
|
|
|
| Tomcat11, tomcat10 installation configuration (Windows environment) (detailed graphics) |
Tomcat startup flashback problem solving set (eight categories in detail) |
|
