Reading and writing files is the most common IO operation. Python has built-in functions for reading and writing files, and its usage is compatible with C.
Before reading and writing files, we must first understand that the function of reading and writing files on the disk is provided by the operating system. Modern operating systems do not allow ordinary programs to directly manipulate the disk . Therefore, to read and write files is to request the operating system to open a file object (usually called a file descriptor) , and then read data from the file object through the interface provided by the operating system (Read file), or write data to this file object (write file).
Read file
To open a file object in the mode of reading a file, use Python's built-in open() function, and pass in the file name and identifier: (simple, learning Unix environment programming)
>>>f = open('/Users/michael/test.txt', 'r')
The identifier'r' means read. In this way, we have successfully opened a file.
If the file does not exist, the open() function will throw an IOError error, and give an error code and detailed information to tell you that the file does not exist:
>>> f=open('/Users/michael/notfound.txt', 'r')
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
FileNotFoundError: [Errno 2] No such file or directory: '/Users/michael/notfound.txt'
If the file is opened successfully, then call the read() method to read the entire contents of the file at once. Python reads the contents into the memory, which is represented by a str object:
>>> f.read()
'Hello, world!'
The last step is to call the close() method to close the file**. After the file is used, it must be closed**, because the file object will occupy the resources of the operating system, and the number of files that the operating system can open at the same time is also limited:
Must be closed, have a deep understanding, not much to say
>>> f.close()
Since IOError may occur during file reading and writing, once an error occurs, the following f.close() will not be called. Therefore, in order to ensure that the file can be closed correctly regardless of whether there is an error , we can use try… finally to achieve:
try:
f = open('/path/to/file', 'r')
print(f.read())
finally:
if f:
f.close()
But it is too cumbersome to be so realistic every time (no wonder that laziness is the ladder of human progress), so Python introduced the with statement to automatically help us call the close() method:
with open('/path/to/file', 'r') as f:
print(f.read())
This is the same as the previous try...finally, but the code is more concise, and there is no need to call the f.close() method.
Calling read() will read the entire content of the file at once . If the file has 10G, the memory will burst. Therefore, to be on the safe side, you can call the read(size) method repeatedly, and read at most size bytes of content each time . In addition, call readline() to read one line at a time , call readlines() to read all the content at once and return the list line by line . Therefore, it is necessary to decide how to call.
If the file is small, read() is the most convenient one-time read; if the file size cannot be determined, it is safer to call read(size) repeatedly; if it is a configuration file, it is most convenient to call readlines():
for line in f.readlines():
print(line.strip()) # 把末尾的'\n'删掉
file-like Object
Objects with a read() method returned by the open() function are collectively called file-like objects in Python. In addition to file, it can also be a byte stream of memory, a network stream, a custom stream, and so on. File-like Object does not require inheritance from a specific class, just write a read() method.
StringIO is a file-like object created in memory and is often used as a temporary buffer.
binary file
The defaults mentioned above are to read text files, and are UTF-8 encoded text files. To read binary files, such as pictures, videos, etc., just open the file in'rb' mode:
>>> f = open('/Users/michael/test.jpg', 'rb')
>>> f.read()
b'\xff\xd8\xff\xe1\x00\x18Exif\x00\x00...' # 十六进制表示的字节
Character Encoding
To read a non-UTF-8 encoded text file, you need to pass the encoding parameter to the open() function. For example, to read a GBK encoded file:
>>> f = open('/Users/michael/gbk.txt', 'r', encoding='gbk')
>>> f.read()
'测试'
When you encounter some files with non-standard encoding, you may encounter UnicodeDecodeError , because some illegally encoded characters may be mixed in the text file. In this case, the open() function also receives an errors parameter, which indicates what to do if it encounters an encoding error. The easiest way is to simply ignore:
>>> f = open('/Users/michael/gbk.txt', 'r', encoding='gbk', errors='ignore')
Write file
Writing a file is the same as reading a file. The only difference is that when the open() function is called, the identifier'w' or'wb' is passed in to indicate writing a text file or writing a binary file:
>>> f = open('/Users/michael/test.txt', 'w')
>>> f.write('Hello, world!')
>>> f.close()
You can call write() repeatedly to write to the file, but you must call f.close() to close the file. When we write a file, the operating system often does not write the data to the disk immediately, but puts it in the memory cache, and then writes it slowly when it is free. Only when the close() method is called, the operating system guarantees that all unwritten data is written to the disk. The consequence of forgetting to call close() is that only part of the data may be written to the disk, and the rest is lost. So, still use the with statement to get insurance :
with open('/Users/michael/test.txt', 'w') as f:
f.write('Hello, world!')
To write a text file with a specific encoding, pass the encoding parameter to the open() function to automatically convert the string to the specified encoding.
Careful children's shoes will find that when writing a file in'w' mode, if the file already exists, it will be overwritten directly (equivalent to writing a new file after deleting it). What if we want to append to the end of the file? You can pass in'a' to write in append mode.