import tensorflow as tf
import numpy as np
'''
MYPS: Learning it requires a lot of basic knowledge. It is recommended that you buy video courses and take notes and accumulate learning step by step.
If there is something you don't understand in the first round, or something you don't understand, skip it first.
It's not because the teacher didn't speak clearly, but because I didn't have enough knowledge and I couldn't understand it.
Then, we started iterating... This process is very difficult, even if you don't understand it, you have to read it, keep reading,
Slowly, you will find that you don't need to spend time watching videos where you drew question marks before, you will understand by yourself
The courses of the training institutions can only get us started. Then how far and how deep you can go depends on your own efforts!
tensorflow introductory tutorial and a brief explanation of the underlying mechanism - the essence is graph computing, automatically finding dependencies, think about the spark mechanism and you will understand
Introduction, the purpose of this chapter is to get you up and running with TensorFlow!
Before getting started, take a look at a TensorFlow sample code written using the Python API to get an initial impression of what you'll learn.
This very short Python program generates some 3D data and then fits it with a plane.
One, tensorflow has a class of functions that evaluate on a certain dimension of tensor. Such as:
Find the maximum value tf.reduce_max(input_tensor, reduction_indices=None, keep_dims=False, name=None)
Average tf.reduce_mean(input_tensor, reduction_indices=None, keep_dims=False, name=None)
Parameter 1--input_tensor: The tensor to be evaluated.
Parameter 2--reduction_indices: in which dimension (column) to solve.
Parameters (3) (4) can be ignored
for example:
# 'x' is [[1., 2.]
# [3., 4.]]
x is a 2-dimensional array, and the reduce_* functions are called as follows:
First find the average:
tf.reduce_mean(x) ==> 2.5 #If the second parameter is not specified, then take the average of all elements
tf.reduce_mean(x, 0) ==> [2., 3.] #Specify the second parameter as 0, then the elements of the first dimension are averaged, that is, each column is averaged
tf.reduce_mean(x, 1) ==> [1.5, 3.5] #Specify the second parameter as 1, then the elements of the second dimension are averaged, that is, each row is averaged
Similarly, you can also use tf.reduce_max() to find the maximum value, etc.
Difference between tf.multiply() and tf.matmul()
(1) tf.multiply is dot multiplication, bit-by-bit/corresponding position element multiplication, ie Returns x * y element-wise.
(2) tf.matmul is matrix multiplication, that is, Multiplies matrix a by matrix b, producing a * b.
'''
# Use NumPy to generate phony data, 100 points in total.
x_data = np.float32(np.random.rand(2, 100)) # random input
y_data = np.dot([0.100, 0.200], x_data) + 0.300
# construct a linear model
b = tf.Variable(tf.zeros([1]))
W = tf.Variable(tf.random_uniform([1, 2], -1.0, 1.0))
y = tf.matmul(W, x_data) + b
# minimize variance
loss = tf.reduce_mean(tf.square(y - y_data))
optimizer = tf.train.GradientDescentOptimizer(0.5)
train = optimizer.minimize(loss)
# Initialize variables
init = tf.initialize_all_variables()
# start graph (graph)
sex = tf.Session ()
sess.run(init)
# fit plane
for step in range(0, 201):
sess.run (train)
if step % 20 == 0:
print (step, sess.run (W), sess.run (b))
# get the best fit result W: [[0.100 0.200]], b: [0.300]
'''
Question: How does the bottom layer work?
Does it automatically sense the following variables? How do you know that you need to train and optimize the model? I didn't tell it what model to use?
loss = tf.reduce_mean(tf.square(y - y_data))
optimizer = tf.train.GradientDescentOptimizer(0.5)
train = optimizer.minimize(loss)
See the document below, which is essentially a graph calculation, and will automatically find variable dependencies when running (xxx)! ! !
An example that best expresses the underlying mechanism of tensorflow is:
'''
# Create a variable, initialized to scalar 0.
state = tf.Variable(0, name="counter")
# Create an op whose role is to increase state by 1
one = tf.constant(1)
new_value = tf.add(state, one)
update = tf.assign(state, new_value)
# After starting the graph, the variable must first be initialized by the `init` (init) op,
# First an `init` op must be added to the graph.
init_op = tf.initialize_all_variables()
# start graph, run op
with tf.Session() as sess:
# run the 'init' op
sess.run (init_op)
# print initial value of 'state'
print (sess.run (state))
# run op, update 'state', and print 'state'
for _ in range(3):
sess.run(update)
print (sess.run (state))
'''
basic use
To use TensorFlow, you must understand TensorFlow: use graphs to represent computational tasks.
Graphs are executed in a context called a Session.
Use tensor to represent data. Maintain state through variables (Variable).
Use feed (feed, feed, feed) and fetch (fetch) to assign values to or fetch data from arbitrary operations.
In general, TensorFlow is a programming system that uses graphs to represent computational tasks. The nodes in the graph are called ops (short for operations).
An op obtains 0 or more Tensors, performs a calculation, and produces 0 or more Tensors. Each Tensor is a typed multidimensional array.
For example, you can represent a small set of images as a four-dimensional array of floating-point numbers, where the four dimensions are [batch, height, width, channels].
A TensorFlow graph describes the process of computation. In order to perform computation, the graph must be started in a session. A session distributes graph ops to devices such as CPUs or GPUs,
It also provides methods for executing op. After these methods are executed, the generated tensor will be returned. In the Python language, the returned tensor is a numpy ndarray object;
In C and C++ languages, the returned tensor is a tensorflow::Tensor instance.
Computational graph TensorFlow programs are usually organized into a build phase and an execution phase.
During the build phase, the execution steps of an op are described as a graph. During the execution phase, the op in the execution graph is executed using a session.
For example, it is common to create a graph to represent and train a neural network during the build phase, and then repeatedly execute the training ops in the graph during the execution phase.
TensorFlow supports C, C++, Python programming languages.
Currently, TensorFlow's Python library is easier to use and provides a large number of helper functions to simplify the job of building graphs, which are not yet supported by the C and C++ libraries.
The session libraries for the three languages are identical.
## The first step in building a graph is to create a source op (source op). The source op does not require any input, such as a constant (Constant). The output of the source op is passed to other ops for operation.
In the Python library, the return value of the op constructor represents the output of the constructed op, and these return values can be passed to other op constructors as input.
The TensorFlow Python library has a default graph to which the op constructor can add nodes. This default graph is sufficient for many programs.
Read the Graph class documentation to learn how to manage multiple graphs.
# Create a constant op, yielding a 1x2 matrix. This op is treated as a node
# Add to default map.
#
# The return value of the constructor represents the return value of the constant op.
matrix1 = tf.constant([[3., 3.]])
# Create another constant op, producing a 2x1 matrix.
matrix2 = tf.constant([[2.],[2.]])
# Create a matrix multiplication matmul op , taking 'matrix1' and 'matrix2' as input.
# The return value 'product' represents the result of matrix multiplication.
product = tf.matmul(matrix1, matrix2)
The default graph now has three nodes, two constant() ops, and a matmul() op. In order to actually do the matrix multiplication and get the result of the matrix multiplication, you must start the graph in the session.
The graph is started in a session, and the graph cannot be started until the construction phase is complete.
The first step in starting a graph is to create a Session object, if there are no creation parameters, the session constructor will start the default graph.
For the full session API, read the Session class.
# Start the default graph.
sex = tf.Session ()
# Call the 'run()' method of sess to perform the matrix multiplication op, passing 'product' as an argument to this method.
# As mentioned above, 'product' represents the output of the matrix multiplication op, and passing it in indicates to the method that we want to get back
# Output of matrix multiplication op.
#
# The entire execution process is automated, the session is responsible for passing all the input required by the op. The op is usually executed concurrently.
#
# The function call 'run(product)' triggers the execution of the three ops in the graph (two constant ops and one matrix multiplication op).
#
# The return value 'result' is a numpy `ndarray` object.
result = sess.run(product)
print result
# ==> [[ 12.]]
# Task completed, close the session.
sess.close()
The Session object needs to be closed after use to free resources. In addition to explicitly calling close, the "with" block can be used to automatically complete the closing action.
with tf.Session() as sess:
result = sess.run([product])
print result
Implementation-wise, TensorFlow transforms graph definitions into operations that are executed in a distributed manner to take full advantage of available computing resources (such as CPU or GPU).
There is generally no need to explicitly specify whether to use a CPU or a GPU, TensorFlow can detect it automatically. If a GPU is detected, TensorFlow will try to use the first GPU it finds to perform the operation.
If there is more than one available GPU on the machine, other than the first GPU are not involved in the calculation by default. In order for TensorFlow to use these GPUs,
The op must be explicitly assigned to them for execution. The with...Device statement is used to assign a specific CPU or GPU to execute an operation:
with tf.Session() as sess:
with tf.device("/gpu:1"):
matrix1 = tf.constant([[3., 3.]])
matrix2 = tf.constant([[2.],[2.]])
product = tf.matmul(matrix1, matrix2)
...
Devices are identified by strings. Currently supported devices include:
"/cpu:0": The CPU of the machine.
"/gpu:0": The machine's first GPU, if any.
"/gpu:1": The second GPU of the machine, and so on.
Read the Working with GPUs chapter to learn more about TensorFlow GPU usage.
interactive use
The Python examples in the documentation use a Session to start the graph, and call the Session.run() method to perform the operation.
To facilitate the use of a Python interactive environment such as IPython, InteractiveSession can be used instead of the Session class,
Use Tensor.eval() and Operation.run() methods instead of Session.run(). This avoids using a variable to hold the session.
# Enter an Interactive (interactive, interactive, interactive) interactive TensorFlow session.
import tensorflow as tf
sex = tf.InteractiveSession ()
x = tf.Variable([1.0, 2.0])
a = tf.constant([3.0, 3.0])
# Initialize 'x' using the run() method of the initializer op
x.initializer.run()
# Add a subtraction sub op, subtract 'a' from 'x'. Run the subtraction op, output the result
sub = tf.sub(x, a)
print sub.eval()
# ==> [-2. -1.]
##Tensor
TensorFlow programs use the tensor data structure to represent all data. In the calculation graph, the data passed between operations are all tensors.
Think of a TensorFlow tensor as an n-dimensional array or list. A tensor contains a static type rank (queue), and a shape (shape, dimension).
To understand how TensorFlow handles these concepts, see Rank, Shape, and Type.
##variable
Variables for more details. Variables maintain state information during the execution of the graph (such as the initial fitting of wx+b, where w and b are the calculation states).
The following example demonstrates how to implement a simple counter using variables. See the Variables chapter for more details.
Deepen your memory and relive it again! ! ! Don't be too bothered by the basics. A good foundation determines the superstructure!
'''
# Create a variable, initialized to scalar 0.
state = tf.Variable(0, name="counter")
# Create an op whose role is to increase state by 1
one = tf.constant(1)
new_value = tf.add(state, one)
update = tf.assign(state, new_value)
# After starting the graph, the variable must first be initialized by the `init` (init) op,
# First an `init` op must be added to the graph.
init_op = tf.initialize_all_variables()
# start graph, run op
with tf.Session() as sess:
# run the 'init' op
sess.run (init_op)
# print initial value of 'state'
print (sess.run (state))
# run op, update 'state', and print 'state'
for i in range(3):
sess.run(update)
print (sess.run (state))
# output:
# 0
# 1
# 2
# 3
'''
The assign() operation in the code is part of the expression depicted in the diagram, just like the add() operation. So it doesn't actually perform the assignment until run() is called to execute the expression.
The parameters in a statistical model are usually represented as a set of variables.
For example, the weights of a neural network can be stored as a variable in a tensor. During training, the tensor is updated by repeatedly running the training graph
Fetch
In order to retrieve the output content of the operation, when using the run() of the Session object to call the execution graph, pass in some tensor to help retrieve the results.
In the previous example, only a single node state was retrieved, and multiple tensors can also be retrieved:
input1 = tf.constant(3.0)
input2 = tf.constant(2.0)
input3 = tf.constant(5.0)
intermed = tf.add(input2, input3)
mul = tf.mul(input1, intermed)
with tf.Session():
result = sess.run([mul, intermed])
print result
# output:
# [array([ 21.], dtype=float32), array([ 7.], dtype=float32)]
The multiple tensor values that need to be obtained are obtained together in one run of op (instead of fetching tensors one by one).
Feed (supply)
The above example introduces tensors in the computation graph, which are stored in the form of constants or variables. TensorFlow also provides a feed mechanism,
This mechanism can temporarily replace the tensor in any operation in the graph. You can submit a patch to any operation in the graph and insert a tensor directly.
feed temporarily replaces the output of an operation with a tensor value.
Feed data can be provided as an argument to the run() call. The feed is only valid within the method that calls it. When the method ends, the feed will disappear.
The most common use case is to designate some special operations as "feed" operations, marked by using tf.placeholder() to create placeholders for these operations.
input1 = tf.placeholder(tf.types.float32)
input2 = tf.placeholder(tf.types.float32)
output = tf.mul(input1, input2)
with tf.Session() as sess:
print sess.run([output], feed_dict={input1:[7.], input2:[2.]})
# output:
# [array([ 14.], dtype=float32)]
for a larger-scale example of feeds. The placeholder() operation will generate an error if the feed is not provided correctly.
The MNIST Fully Connected Feed Tutorial (source code) gives a larger example of using feeds.
'''
The content is excerpted from the TensorFlow Chinese community