1. What is the gfile module
The main roles of the tf.gfile module are:
1. Provide an API close to the Python file object, and
2. Provide an implementation based on the TensorFlow C++ FileSystem API.
The C++ FileSystem API supports several filesystem implementations, including local files, Google Cloud Storage (starting with gs://), and HDFS (starting with hdfs://). TensorFlow exports them as tf.gfile so that we can use these implementations to save and load checkpoints, write TensorBoard logs, and access training data (among other things). However, if all files are local, the regular Python file API can be used without any problems.
2. Introduction to gfile API
Each of the gfile APIs will be introduced separately below!
2-1）tf.gfile.Copy(oldpath, newpath, overwrite=False)
Copy the source file and create the target file, no return, its formal parameters are described as follows:
oldpath: copy source file with path name;
newpath: copy target file with path name;
overwrite: Whether to overwrite when the target file already exists, the default is false, if the target file already exists, an error will be reported
2-2）tf.gfile.MkDir(dirname)
Create a directory, dirname is the directory name, no return.
2-3）tf.gfile.Remove(filename)
Delete the file, filename is the file name, no return.
2-4）tf.gfile.DeleteRecursively(dirname)
Recursively delete all directories and their files, dirname is the directory name, no return.
2-5）tf.gfile.Exists(filename)
Determines whether a directory or file exists, filename can be a directory path or a path with a file name, and returns True if the directory exists, otherwise False.
2-6）tf.gfile.Glob(filename)
Find files matching pattern and return them as a list. filename can be a specific filename or a regular expression containing wildcards.
2-7）tf.gfile.IsDirectory(dirname)
Determine whether the given directory exists, if it exists, return True, otherwise return False, dirname is the directory name.
2-8）tf.gfile.ListDirectory(dirname)
List all files in the dirname directory and return them as a list. dirname must be the directory name.
2-9）tf.gfile.MakeDirs(dirname)
The parent directory and its subdirectories are created recursively. If the directory already exists and can be overwritten, it will be created successfully. Otherwise, an error will be reported and no return will be returned.
2-10）tf.gfile.Rename(oldname, newname, overwrite=False)
Rename or move a file or directory, no return, its formal parameters are described as follows:
oldname: old directory or old file;
newname: new directory or new file;
overwrite: The default is false, if the new directory or new file already exists, an error will be reported, otherwise the rename or move will be successful.
2-11）tf.gfile.Stat(filename)
Returns the statistical data of the directory. This function will return the FileStatistics data structure. The attributes of the returned data obtained by dir(tf.gfile.Stat(filename)) are as follows:
2-12）tf.gfile.Walk(top, in_order=True)
Recursively obtain the directory information generator, top is the directory name, in_order defaults to True to indicate that the directory is traversed in order, otherwise it will be traversed out of order, and the following format information is returned for each generation (dirname, [subdirname, subdirname, ...], [filename, filename, ...]).
2-13）tf.gfile.GFile(filename, mode)
Get the text operation handle, similar to the text operation open() function provided by python, filename is the name of the file to be opened, mode is how to read and write, and a text operation handle will be returned.
tf.gfile.Open() is the same name of this interface, you can use any one of them!
2-14）tf.gfile.FastGFile(filename, mode)
The difference between this function and tf.gfile.GFile is only "non-blocking", that is, the function will obtain the text operation handle in a faster way without blocking.
Here are two examples:
Image encoding and decoding:
tf.gfile.FastGFile(path, decodestyle)
function: realize the reading of pictures.
Function parameters: (1) path: the path where the picture is located (2) decodestyle: the decoding method of the picture. ('r': UTF-8 encoding; 'rb': non-UTF-8 encoding)
import matplotlib.pyplot as plt
import tensorflow as tf
#tf.gfileGFile() function: read image
image_jpg = tf.gfile.FastGFile( 'dog.jpg', 'rb').read()
image_png = tf.gfile.FastGFile ( 'lizard.png', 'rb').read()
with tf.Session() as sess:
image_jpg = tf.image.decode_jpeg(image_jpg) #image decoding
print(sess.run(image_jpg)) #print after decoding The image (that is, a three-dimensional matrix [w,h,3])
image_jpg = tf.image.convert_image_dtype(image_jpg,dtype=tf.uint8) #Change the image data type
image_png = tf.image.decode_png(image_png)
print(sess .run(image_jpg))
image_png = tf.image.convert_image_dtype(image_png,dtype=tf.uint8)
plt.figure( 1) #image display
plt.imshow(image_jpg.eval())
plt.figure( 2)
plt.imshow(image_png.eval())