1. Let's look at the introduction in the tf document, convert the mnist or cifar10 image data set into special binary data, and then pass the placeholder placeholder,
Read data through placeholders. But how to convert pictures to special binary data? Not sure, why is it special? Go to mnist's official website for an introduction format.
2. Read data through the queue, why use the queue method? Because for example, it takes m seconds for you to read the picture, and n seconds for the gpu calculation. It takes you n+m seconds for each completion, gpu
It takes n seconds each time, which wastes resources, so 2 queue parallel threads are used to solve the io problem. One is responsible for reading pictures, and one queue is responsible for computing.
Wait, the data storage method officially recommended by tensorflow is tfrecord format, and the queue can also read tfrecord data.
Why is it so troublesome? How good is it to read the image directly? Because each time you read a picture, you have to read from the beginning of the file to the end. When you have a lot of pictures, it takes a lot of time to read only.
Therefore, it is recommended to convert the image to tfrecord format first, and then process it in the queue. Save time. When converting images to tfrecord, there are also multi-threaded methods to speed up the conversion.
Data enhancement is also very slow, so it is also recommended to use multi-threading.
3. Now after tf1.4, the official push to use the dataset method to read data, instead of the previous queue method. It seems that dataset is more concise and powerful,
Still under study. dataset can also read pictures, and other data.
To be continued. . . .