How to choose between TensorFlow and PyTorch?

Original link: How to choose between TensorFlow and PyTorch?

Many students in the background asked me whether I should learn the deep learning framework TensorFlowor not PyTorch? I will give personal suggestions in the following aspects.

1. Ease of learning and operability

Deep learning frameworks use computational graphs to define the order of computation performed in a neural network. TF1The static graph mechanism used, PyTorchthe dynamic graph mechanism used.

• Static graph means that the construction of the calculation graph and the actual calculation are done separately (define and run)
• Dynamic graph means that the construction of the calculation graph and the actual calculation happen at the same time (define by run)

Some students may not understand the concept very well. Let's use an example to compare the differences in programming implementation between dynamic graph and static graph mechanism, based on TF1and PyTorchimplementation respectively.

We define two counters aand b, ainitially 0 and binitially 10. When it is aless bthan, aadd 2 to itself, and badd 1 to itself, until the program ends when ait is greater than or equal to .b

TF1

import tensorflow as tf
# 先构建图
a = tf.constant(0)
b = tf.constant(10)

def cond(first_counter, second_counter, *args):
    return first_counter < second_counter

def body(first_counter, second_counter):
    first_counter = tf.add(first_counter, 2)
    second_counter = tf.add(second_counter, 1)
    return first_counter, second_counter
    
c1, c2 = tf.while_loop(cond, body, [a, b])

# 再创建会话进行计算
with tf.Session() as sess:
    a, b = sess.run([c1, c2])

print(a)
print(b)

In the static graph, the entire calculation process and framework need to be defined in advance, and then the calculation can only be performed after a session is created.

PyTorch

import torch

a = torch.Tensor([0])
b= torch.Tensor([10])

while (a < b):
    a += 2
    b += 1

print(a)
print(b)

Seeing this, students should be able to see the difference. Except for defining tensors Tensor, the rest of the code Pythonis exactly the same as that of Because PyTorchit is a dynamic graph, each operation will build a new calculation graph.

So if you will Python, it will mean that you PyTorchdo not need additional learning costs in programming.

Debug

In study or work, debugging code is definitely inevitable, and I think TF1the biggest disadvantage is that it is difficult Debug. No error was reported when it was established Graph, but I believe many friends have experienced the pain of various collapses once the data is fed.

And TF1as a static graph framework, printing intermediate results must be Sessionrun with the help of a session to take effect, or you need to learn additional tfdbgtools. And if you use PyTorchsuch a dynamic graph framework, you don't need to learn an additional tool, you only need to use normal Pythondebugging tools.

2. Future Prospects

To learn a technology, first of all, it must be to meet the current needs, and secondly, it is best to become the mainstream in the future.

Use the crowd

The picture below is from the PapersWithCode website, and the colored area represents the number of public code bases of papers using the framework.

In the newly published paper codes in the latest month (September 2021), the usage of the two has differed by 5 times, which can be inferred that the share PyTorchin the scientific research field is much higher than TFthat of dominant position.

After writing this, it is not difficult for us to imagine that a large number of these researchers and students will enter the workplace in the future.

why didn't you talkTF2

TFIts consistent advantage is that there are many choices and mature deployment frameworks. In addition, most programmers in our country are code porters and repairers. So it has occupied the industrial market for a long time.

TF2The birth of made users who were accustomed to static images feel a little uncomfortable. TF2It is a dynamic graph framework, which is TF1used differently from . If TF1the users who used before have to re-learn if they want to transition to it TF2, then re-learn the dynamic graph framework, why not learn a more mature and stable one in the dynamic graph field PyTorch? And in recent years, Pytorchit has matured a lot in terms of landing deployment, and it can definitely meet most application needs.

To sum up, whether you are a student or a new migrant worker, if you want to learn the deep learning framework, I would recommend it PyTorch.

More dry goods related to AI deep learning are available on the official account: AI has temperature