常见术语：
特征(集)即某个样本的特点[对个特征描述就是特征集]；
标签即我们尝试预测的内容；
(一个)样本由一个特征集和标签组成；
模型是特征和标签之间的关系；
训练是一种机器学习阶段，在此阶段中，模型会逐渐得到优化(不断学习)
监督式机器学习即模型通过包含标签的样本加以训练;
非监督式机器学习中，样本不包含标签, 相反，模型通常会在特征中发现一些规律。
训练集包含我们用于训练模型的样本；
测试集包含我们用于评估训练后模型的效果的样本。
代码框架
1. 导入和解析数据集。
2. 创建特征列以描述数据。
3. 选择模型类型。
4. 训练模型。
5. 评估模型的效果。
6. 让经过训练的模型进行预测
1.tf.keras.utils.get_file:
下载 URL文件到本地(~/.keras/datasets/fname), 返回下载文件的路径
get_file in module tensorflow.python.keras.utils.data_utils:
get_file(fname, origin,
untar=False, md5_hash=None, file_hash=None,
cache_subdir='datasets', hash_algorithm='auto',
extract=False, archive_format='auto', cache_dir=None)
Downloads a file from a URL if it not already in the cache.
By default the file at the url `origin` is downloaded to the
cache_dir `~/.keras`, placed in the cache_subdir `datasets`,
and given the filename `fname`. The final location of a file
`example.txt` would therefore be `~/.keras/datasets/example.txt`.
Files in tar, tar.gz, tar.bz, and zip formats can also be extracted.
Passing a hash will verify the file after download. The command line
programs `shasum` and `sha256sum` can compute the hash.
Returns:
Path to the downloaded file
2.tf.decode_csv
转换为Tensor对象 list
decode_csv in module tensorflow.python.ops.parsing_ops:
decode_csv(records, record_defaults,
field_delim=',', use_quote_delim=True,
name=None, na_value='', select_cols=None)
Convert CSV records to tensors. Each column maps to one tensor.
Args:
records: A `Tensor` of type `string`.
Each string is a record/row in the csv and all records should have
the same format.
record_defaults: A list of `Tensor` objects with specific types.
Acceptable types are `float32`, `float64`, `int32`, `int64`, `string`.
One tensor per column of the input record, with either a
scalar default value for that column or empty if the column is required.
field_delim: An optional `string`. Defaults to `","`.
char delimiter to separate fields in a record.
use_quote_delim: An optional `bool`. Defaults to `True`.
If false, treats double quotation marks as regular
characters inside of the string fields (ignoring RFC 4180, Section 2,
Bullet 5).
name: A name for the operation (optional).
na_value: Additional string to recognize as NA/NaN.
select_cols: Optional sorted list of column indices to select. If specified,
only this subset of columns will be parsed and returned.
Returns:
A list of `Tensor` objects. Has the same type as `record_defaults`.
Each tensor will have the same shape as records.
3.tf.data.TextLineDataSet
TextLineDataset in module tensorflow.python.data.ops.readers:
class TextLineDataset(tensorflow.python.data.ops.dataset_ops.Dataset)
A `Dataset` comprising lines from one or more text files.
4.iter(train_dataset).next()
iter
built-in function iter in module __builtin__:
iter(...)
iter(collection) -> iterator
5.sparse_softmax_cross_entropy
sparse_softmax_cross_entropy 标签实际值和通过模型预测结果间的差
sparse_softmax_cross_entropy in module tensorflow.python.ops.losses.losses_impl:
sparse_softmax_cross_entropy(labels, logits,
weights=1.0, scope=None, loss_collection='losses',
reduction='weighted_sum_by_nonzero_weights')
Cross-entropy loss using `tf.nn.sparse_softmax_cross_entropy_with_logits`.
Returns:
Weighted loss `Tensor` of the same type as `logits`. If `reduction` is
`NONE`, this has the same shape as `labels`; otherwise, it is scalar.
6.GradientTape
使用GradientTape对象，调用其gradient函数，gardient对loss函数对model.variables进行求导
GradientTape in module tensorflow.python.eager.backprop:
class GradientTape(__builtin__.object)
| Record operations for automatic differentiation.
|
| Operations are recorded if they are executed within this context manager and
| at least one of their inputs is being "watched".
|
| Trainable variables (created by `tf.contrib.eager.Variable` or
| @{tf.get_variable}, trainable=True is default in both cases) are automatically
| watched. Tensors can be manually watched by invoking the `watch` method on
| this context manager.
|
| For example, consider the function `y = x * x`. The gradient at `x = 3.0` can
| be computed as:
|
| ```python
| x = tf.constant(3.0)
| with tf.GradientTape() as g:
| g.watch(x)
| y = x * x
| dy_dx = g.gradient(y, x) # Will compute to 6.0
| ```
|
| GradientTapes can be nested to compute higher-order derivatives. For example,
|
| ```python
| x = tf.constant(3.0)
| with tf.GradientTape() as g:
| with tf.GradientTape() as gg:
| gg.watch(x)
| y = x * x
| dy_dx = gg.gradient(y, x) # Will compute to 6.0
| d2y_dx2 = g.gradient(dy_dx, x) # Will compute to 2.0
| ```
|
| By default, the resources held by a GradientTape are released as soon as
| GradientTape.gradient() method is called. To compute multiple gradients over
| the same computation, create a persistent gradient tape. This allows multiple
| calls to the gradient() method as resources are released when the tape object
| is garbage collected. For example:
|
| ```python
| x = tf.constant(3.0)
| with tf.GradientTape(persistent=True) as g:
| g.watch(x)
| y = x * x
| z = y * y
| dz_dx = g.gradient(z, x) # 108.0 (4*x^3 at x = 3)
| dy_dx = g.gradient(y, x) # 6.0
| del g # Drop the reference to the tape
| ```
|
| Note that only tensors with real or complex dtypes are differentiable.
|
| Methods defined here:
| gradient(self, target, sources, output_gradients=None)
| Computes the gradient using operations recorded in context of this tape.
|
| Args:
| target: Tensor (or list of tensors) to be differentiated.
| sources: a list or nested structure of Tensors or Variables. `target`
| will be differentiated against elements in `sources`.
|
| Returns:
| a list or nested structure of Tensors (or IndexedSlices, or None),
| one for each element in `sources`. Returned structure is the same as
| the structure of `sources`.
7.GradientDescentOptimizer
怎样使用梯度？
GradientDescentOptimizer
GradientDescentOptimizer in module tensorflow.python.training.gradient_descent:
class GradientDescentOptimizer(tensorflow.python.training.optimizer.Optimizer)
| Optimizer that implements the gradient descent algorithm.
8.argmax
argmax in module tensorflow.python.ops.math_ops:
argmax(*args, **kwargs)
Returns the index with the largest value across axes of a tensor.
Args:
input: A `Tensor`. Must be one of the following types: .
axis: A `Tensor`. Must be one of the following types: `int32`, `int64`.
int32 or int64, must be in the range `[-rank(input), rank(input))`.
Describes which axis of the input Tensor to reduce across.
For vectors, use axis = 0.
output_type: An optional `tf.DType` from: `tf.int32, tf.int64`. Defaults to `tf.int64`.
name: A name for the operation (optional).
Returns:
A `Tensor` of type `output_type`.
/****************************************************************************************/
import os
import matplotlib.pyplot as plt
import tensorflow as tf
import tensorflow.contrib.eager as tfe
tf.enable_eager_execution()
print("TensorFlow version: {}".format(tf.VERSION))
print("Eager execution: {}".format(tf.executing_eagerly()))
# download training dataset
train_dataset_url = "http://download.tensorflow.org/data/iris_training.csv"
train_dataset_fp = tf.keras.utils.get_file(fname=os.path.basename(train_dataset_url),
origin=train_dataset_url)
print("Local copy of the dataset file: {}".format(train_dataset_fp))
# parse data: for each line
def parse_csv(line):
example_defaults = [[0.], [0.], [0.], [0.], [0]] # sets field types
parsed_line = tf.decode_csv(line, example_defaults)
# First 4 fields are features, combine into single tensor
features = tf.reshape(parsed_line[:-1], shape=(4,))
# Last field is the label
label = tf.reshape(parsed_line[-1], shape=())
return features, label
#line -> dataset -> map[分开特征和标签] -> shuffle -> batch
train_dataset = tf.data.TextLineDataset(train_dataset_fp)
train_dataset = train_dataset.skip(1) # skip the first header row
train_dataset = train_dataset.map(parse_csv) # parse each row
train_dataset = train_dataset.shuffle(buffer_size=1000) # randomize
train_dataset = train_dataset.batch(32)
# 这里只是看看sample数据
# View a single example entry from a batch
features, label = iter(train_dataset).next()
print("example features:", features[0])
print("example label:", label[0])
# 构建model的层：输入特征input_shape
model = tf.keras.Sequential([
tf.keras.layers.Dense(10, activation="relu", input_shape=(4,)), # input shape required
tf.keras.layers.Dense(10, activation="relu"),
tf.keras.layers.Dense(3) //并没有softmat计算
])
# input -> model -> y_ 计算模式输出结果和实际结果间的 loss
def loss(model, x, y):
y_ = model(x)
return tf.losses.sparse_softmax_cross_entropy(labels=y, logits=y_)
# 计算损失函数对weight的梯度
def grad(model, inputs, targets):
with tf.GradientTape() as tape:
loss_value = loss(model, inputs, targets)
return tape.gradient(loss_value, model.variables)
# 怎样使用梯度优化
optimizer = tf.train.GradientDescentOptimizer(learning_rate=0.01)
## Note: Rerunning this cell uses the same model variables
# keep results for plotting
train_loss_results = []
train_accuracy_results = []
num_epochs = 201
for epoch in range(num_epochs):
epoch_loss_avg = tfe.metrics.Mean()
epoch_accuracy = tfe.metrics.Accuracy()
# Training loop - using batches of 32
for x, y in train_dataset:
# Optimize the model
grads = grad(model, x, y)
optimizer.apply_gradients(zip(grads, model.variables),
global_step=tf.train.get_or_create_global_step())
# 到此生成模型的代码已经执行完了
# Track progress 跟踪执行过程
epoch_loss_avg(loss(model, x, y)) # add current batch loss
# compare predicted label to actual label
epoch_accuracy(tf.argmax(model(x), axis=1, output_type=tf.int32), y)
# end epoch 每次epoch执行一遍训练集
train_loss_results.append(epoch_loss_avg.result())
train_accuracy_results.append(epoch_accuracy.result())
if epoch % 50 == 0:
print("Epoch {:03d}: Loss: {:.3f}, Accuracy: {:.3%}".format(epoch,
epoch_loss_avg.result(),
epoch_accuracy.result()))
#测试训练出的模型
test_url = "http://download.tensorflow.org/data/iris_test.csv"
test_fp = tf.keras.utils.get_file(fname=os.path.basename(test_url),
origin=test_url)
test_dataset = tf.data.TextLineDataset(test_fp)
test_dataset = test_dataset.skip(1) # skip header row
test_dataset = test_dataset.map(parse_csv) # parse each row with the funcition created earlier
test_dataset = test_dataset.shuffle(1000) # randomize
test_dataset = test_dataset.batch(32) # use the same batch size as the training set
test_accuracy = tfe.metrics.Accuracy()
for (x, y) in test_dataset:
prediction = tf.argmax(model(x), axis=1, output_type=tf.int32)
test_accuracy(prediction, y)
print("Test set accuracy: {:.3%}".format(test_accuracy.result()))
#预测数据(泛化)
class_ids = ["Iris setosa", "Iris versicolor", "Iris virginica"]
predict_dataset = tf.convert_to_tensor([
[5.1, 3.3, 1.7, 0.5,],
[5.9, 3.0, 4.2, 1.5,],
[6.9, 3.1, 5.4, 2.1]
])
predictions = model(predict_dataset)
for i, logits in enumerate(predictions):
class_idx = tf.argmax(logits).numpy()
name = class_ids[class_idx]
print("Example {} prediction: {}".format(i, name))