Reference: Tensorflow Tutorial - Cat and Dog War Dataset
Post a mind map drawn by yourself
The data set is prepared for
the kaggle cat and dog war data set (training), Microsoft does not need to overturn the wall
- 12500 cat
- 12500 dogs
Generate a List of image paths and labels
step1: Get all cat map path names under D:/Study/Python/Projects/Cats_vs_Dogs/data/Cat, store them in cats, and label 0 at the same time, and store them in label_cats. The same is true for dog pictures.
train_dir = 'D:/Study/Python/Projects/Cats_vs_Dogs/data'
def get_files(file_dir):
for file in os.listdir(file_dir+'/Cat'):
cats.append(file_dir +'/Cat'+'/'+ file)
label_cats.append(0)
for file in os.listdir(file_dir+'/Dog'):
dogs.append(file_dir +'/Dog'+'/'+file)
label_dogs.append(1)
step2: Scramble the generated image path and label List
#把cat和dog合起来组成一个list(img和lab)
image_list = np.hstack((cats, dogs))
label_list = np.hstack((label_cats, label_dogs))
#利用shuffle打乱顺序
temp = np.array([image_list, label_list])
temp = temp.transpose()
np.random.shuffle(temp)
#从打乱的temp中再取出list(img和lab)
image_list = list(temp[:, 0])
label_list = list(temp[:, 1])
label_list = [int(i) for i in label_list]
Generate Batch
step1: Pass the List generated above into get_batch(), convert the type, and generate an input queue queue. Because img and lab are separate, use tf.train.slice_input_producer() , and then use tf.read_file() from the queue read image
- image_W, image_H, : Set a fixed image height and width
- Set batch_size: how many pictures to put in each batch
- capacity: the maximum size of a queue
def get_batch(image, label, image_W, image_H, batch_size, capacity):
#转换类型
image = tf.cast(image, tf.string)
label = tf.cast(label, tf.int32)
# make an input queue
input_queue = tf.train.slice_input_producer([image, label])
label = input_queue[1]
image_contents = tf.read_file(input_queue[0]) #read img from a queue
step2: Decode the image, different types of images cannot be mixed together, either only use jpeg, or only use png, etc.
image = tf.image.decode_jpeg(image_contents, channels=3)
Step3: Data preprocessing, rotating, scaling, cropping, normalizing and other operations on the image to make the calculated model more robust.
image = tf.image.resize_image_with_crop_or_pad(image, image_W, image_H)
image = tf.image.per_image_standardization(image)
step4: Generate batch
- image_batch: 4D tensor [batch_size, width, height, 3],dtype=tf.float32
- label_batch: 1D tensor [batch_size], dtype=tf.int32
image_batch, label_batch = tf.train.batch([image, label],
batch_size= batch_size,
num_threads= 32,
capacity = capacity)
#重新排列label,行数为[batch_size]
label_batch = tf.reshape(label_batch, [batch_size])
image_batch = tf.cast(image_batch, tf.float32)
test
step1: Variable initialization, 2 images per batch, size 208x208, set your own image path
BATCH_SIZE = 2
CAPACITY = 256
IMG_W = 208
IMG_H = 208
train_dir = 'D:/Study/Python/Projects/Cats_vs_Dogs/data'
step2: call the previous two functions to generate batch
image_list, label_list = get_files(train_dir)
image_batch, label_batch = get_batch(image_list, label_list, IMG_W, IMG_H, BATCH_SIZE, CAPACITY)
step3: Open the session session and use tf.train.Coordinator() and tf.train.start_queue_runners(coord=coord) to monitor the queue (there is a problem: start_queue_runners() on the official website has two parameters, sess and coord, But adding sess here will report an error).
Use the try-except-finally structure to perform queue operations (the method recommended by the official website) to avoid program stuck or something. i<2 performs two queue operations, each time takes out 2 images and puts them in the batch, and then imshow comes out to see the effect.
with tf.Session() as sess:
i = 0
coord = tf.train.Coordinator()
threads = tf.train.start_queue_runners(coord=coord)
try:
while not coord.should_stop() and i<2:
img, label = sess.run([image_batch, label_batch])
# just test one batch
for j in np.arange(BATCH_SIZE):
print('label: %d' %label[j])
plt.imshow(img[j,:,:,:])
plt.show()
i+=1
except tf.errors.OutOfRangeError:
print('done!')
finally:
coord.request_stop()
coord.join(threads)
step4 : Check the results, there will be 4 pictures, the effect of resize is not very good, I don’t know what is theimage_batch = tf.cast(image_batch, tf.float32)
problem All right