Loading Tensorflow 2.0 datasets

Import Package

import tensorflow as tf
from tensorflow import keras

Download Data

tensorflow can call keras own datasets, very convenient, is a little uncomfortable is that people need to download fq, and this agent is not easy to open, so here I put all the data is downloaded and uploaded to the cloud nuts, convenience you download.

Download link (access code: yDmqHd)

After the download is good, to enter into the C:\Users\用户名\.keras\datasetsinside, if there is no datasetsfolder, create a new one, and then put inside the data directly on the line.

Directory structure is as follows

C:.
│  keras.json
│
└─datasets
    │  boston_housing.npz
    │  cifar-10-batches-py.tar.gz
    │  cifar-100-python.tar.gz
    │  imdb.npz
    │  mnist.npz
    │  reuters.npz
    │
    └─fashion-mnist
            t10k-images-idx3-ubyte.gz
            t10k-labels-idx1-ubyte.gz
            train-images-idx3-ubyte.gz
            train-labels-idx1-ubyte.gz

Finally, just a word read data

(x, y), (x_test, y_test) = keras.datasets.mnist.load_data()
(x, y), (x_test, y_test) = keras.datasets.boston_housing.load_data()
...

tf.data.Dataset use

Using the .from_tensor_slicesmethod of loading the data set

ds = tf.data.Dataset.from_tensor_slices((x, y))

Data preprocessing

.map

Use mapcan predict the data, and the same principle comes with python

def prepare_mnist_fea(x, y):
    x = tf.cast(x, tf.float32) / 255.0
    y = tf.cast(y, tf.float32)
    return x, y

ds.map(prepare_mnist_fea)

.shuffle

mess up the order

ds.shuffle(10000)

.batch

Use an batchiterative

ds.batch(32)

.repeat

Repeat the entire data many times, that is, the meaning of epoch

ds.repeat(10)