Below are the study notes for lecture1.
Introduction The mathematical concepts behind
deep learning have been around for over 10 years, but deep learning frameworks have only been around in the last few years. A large number of frameworks now have a trade-off between flexibility and ease of use in industry. Flexibility is very important for scientific research, but too slow for industry, but in other words, frameworks that are fast enough for distributed use are only suitable for Due to the special network structure, it is not flexible enough for scientific research. This leaves users with a paradoxical situation: should we try to do research with a less flexible framework so that when applied to industry we don't have to reproduce the code in another framework; or should we How about using one framework for research and a completely different framework for industrial applications?
If you choose the former, it is not convenient to try many different types of networks when doing research. If you choose the latter, we have to reproduce the code, which will easily lead to different experimental results and industrial applications, and we also need to put a lot of effort into Learn.
The advent of TensorFlow hopes to resolve this paradox.
What is TensorFlow?
Open source software for numerical computation using dataflows and graphs for machine intelligence
Mainly developed by the Google Brain team for machine learning and deep neural network research
Can be applied to a wide range of fields
Although TensorFlow is open source, But only the part on GitHub is open source. Google also has an internal version. The official statement is that Google's internal version has many tools and services that are customized for it. The public does not need to use it. It is not that Google's open source is not sincere, I hope so Bar.
Why use TensorFlow?
Python API, which is available in most deep learning frameworks
Ability to use multiple CPUs and GPUs, and most importantly, easy deployment to servers and mobiles, something that many frameworks cannot do Flexible
enough , very low-level
tensorboard visualization is very good
Checkpoints as experiment management, the ability to save models at any time
Automatic Differentiation
Huge Community
Lots of great projects using TensorFlow
Getting Started
tensor
0-d tensor: scalar, 1-d tensor: vector, 2-d tensor:
matrix Dataflow graph
import tensorflow as tf
a = tf.add(3, 5 )
print(a)
>> Tensor("Add: 0", shape=(), dtype=int32)
can't get 8, you need to open the session, the operation can be performed in the session, and the Tensor can be calculated, which is a bit contradictory. Humans are different from general inferential programming, such as PyTorch
import tensorflow as tf
a = tf.add(3, 5)
sess = tf.Session()
print(sess.run(a))
sess.close()
> > 8
can certainly be written in a more efficient way
import tensorflow as tf
a = tf.add(3, 5)
with tf.Session() as sess:
print(sess.run(a))
>> 8
Of course more complex computation graphs can be built as follows:
x = 2
y = 3
add_op = tf.add(x, y)
mul_op = tf.mul(x, y)
useless = tf.mul(x, add_op)
pow_op = tf.pow(add_op, mul_op)
with tf.Session() as sess:
z , not_useless = sess.run([op3, useless])
use [] when calling `sess.run` to get multiple results.
You can also divide the graph into many small pieces and let them run in parallel on multiple CPUs and GPUs.
You can put part of the computational graph on a specific GPU or CPU
with tf.device('/gpu:2'):
a = tf. constant([[1.0, 2.0, 3.0], [4.0, 5.0, 6.0]], name='a')
b = tf.constant([[1.0, 2.0], [3.0, 4.0], [5.0, 6.0] ], name='b')
c = tf.matmul(a, b)
sess = tf.Session(config=tf.ConfigProto(log_device_placement=True))
print(sess.run(c))
Try not to use multiple computational graphs, because each computational graph needs One session, and each session will use all the graphics card resources. You must use python/numpy to transfer data between two graphs. It is best to create two subgraphs that are not connected in one graph.
Why use Graph
1. Saving Computational resources, each operation only needs to run the subgraph related to the result 2. The graph can
be divided into small pieces for automatic differentiation
3. It is easy to deploy on multiple devices These are the study notes for the first lecture.
Introduction The mathematical concepts behind
deep learning have been around for over 10 years, but deep learning frameworks have only been around in the last few years. A large number of frameworks now have a trade-off between flexibility and ease of use in industry. Flexibility is very important for scientific research, but too slow for industry, but in other words, frameworks that are fast enough for distributed use are only suitable for Due to the special network structure, it is not flexible enough for scientific research. This leaves users with a paradoxical situation: should we try to do research with a less flexible framework so that when applied to industry we don't have to reproduce the code in another framework; or should we How about using one framework for research and a completely different framework for industrial applications?
If you choose the former, it is not convenient to try many different types of networks when doing research. If you choose the latter, we have to reproduce the code, which will easily lead to different experimental results and industrial applications, and we also need to put a lot of effort into Learn.
The advent of TensorFlow hopes to resolve this paradox.
What is TensorFlow?
Open source software for numerical computation using dataflows and graphs for machine intelligence
Mainly developed by the Google Brain team for machine learning and deep neural network research
Can be applied to a wide range of fields
Although TensorFlow is open source, But only the part on GitHub is open source. Google also has an internal version. The official statement is that Google's internal version has many tools and services that are customized for it. The public does not need to use it. It is not that Google's open source is not sincere, I hope so Bar.
Why use TensorFlow?
Python API, which is available in most deep learning frameworks
Ability to use multiple CPUs and GPUs, and most importantly, easy deployment to servers and mobiles, something that many frameworks cannot do Flexible
enough , very low-level
tensorboard visualization is very good
Checkpoints as experiment management, the ability to save models at any time
Automatic Differentiation
Huge Community
Lots of great projects using TensorFlow
Getting Started
tensor
0-d tensor: scalar, 1-d tensor: vector, 2-d tensor:
matrix Dataflow graph
import tensorflow as tf
a = tf.add(3, 5 )
print(a)
>> Tensor("Add: 0", shape=(), dtype=int32)
can't get 8, you need to open the session, the operation can be performed in the session, and the Tensor can be calculated, which is a bit contradictory. Humans are different from general inferential programming, such as PyTorch
import tensorflow as tf
a = tf.add(3, 5)
sess = tf.Session()
print(sess.run(a))
sess.close()
> > 8
can certainly be written in a more efficient way
import tensorflow as tf
a = tf.add(3, 5)
with tf.Session() as sess:
print(sess.run(a))
>> 8
Of course more complex computation graphs can be built as follows:
x = 2
y = 3
add_op = tf.add(x, y)
mul_op = tf.mul(x, y)
useless = tf.mul(x, add_op)
pow_op = tf.pow(add_op, mul_op)
with tf.Session() as sess:
z , not_useless = sess.run([op3, useless])
use [] when calling `sess.run` to get multiple results.
You can also divide the graph into many small pieces and let them run in parallel on multiple CPUs and GPUs.
You can put part of the computational graph on a specific GPU or CPU
with tf.device('/gpu:2'):
a = tf. constant([[1.0, 2.0, 3.0], [4.0, 5.0, 6.0]], name='a')
b = tf.constant([[1.0, 2.0], [3.0, 4.0], [5.0, 6.0] ], name='b')
c = tf.matmul(a, b)
sess = tf.Session(config=tf.ConfigProto(log_device_placement=True))
print(sess.run(c))
Try not to use multiple computational graphs, because each computational graph needs One session, and each session will use all the graphics card resources. You must use python/numpy to transfer data between two graphs. It is best to create two subgraphs that are not connected in one graph.
Why use Graph
1. Saving Computational resources, each operation only needs to run the subgraph related to the result 2. The graph can
be divided into small pieces for automatic differentiation
3. It is easy to deploy on multiple devices These are the study notes for the first lecture.