Commonly used modules in Python (2)---file reading and writing

3. File reading and writing

3.1, open function

3.1.1. The complete grammatical format of the open function

open(file, mode='r', buffering=None, encoding=None, errors=None, newline=None, closefd=True)

3 . 1 . 2. Parameter description

parameter name	Meaning
file	Required, file path (relative or absolute path)
mode	Optional, the file opening mode, the default is 'r'; 'r' opens the file for reading only, 'w' only writes; 'a' appends to the file (the file already exists); if the file does not exist, create and write input file;
buffering	set buffer
encoding	Generally use UTF-8
errors	error level
newline	Differentiate newlines
closefd	Pass in the file parameter type

3 . 1 . 3. Common file operations

method	describe
file.read([size])	Read the specified number of bytes from the file, or all if not given or negative
file.readline([size])	Read the entire line; size indicates how many characters are read in the first line;
file.readlines([sizeint])	Read all lines and return a list; if sizeint>0 is given, only the first line of characters will be read, and with '\n', it will also be output in list format;
file.write(str)	Write a string to a file, and return the length of the written string
file.writelines(sequence)	Write a list of sequence strings to the file, if you need a newline, you need to add a newline character for each line
file.close()	Close the file; read and write operations cannot be performed after closing the file;

 Example file (open 1.txt file), file content:

11111111
22222222
33333333
44444444
55555555

3.1.4. Read all content through read

1. Sample code

f = open('demo6/1.txt', 'r', encoding='utf-8')
data1 = f.read()
print(data1)
f.close()

2. Execution results

3 . 1 . 5. Read the content of a certain line of the file through readline

1. Sample code (without parameters, only the first row of data is read by default)

f = open('demo6/1.txt', 'r', encoding='utf-8')
data2 = f.readline()
print(data2)
f.close()

2. Execution result:

3. Sample code (with parameters, for example, with parameter 1, only read the first character of the first line; if the parameter is equal to or exceeds the length of the first line of characters, then output the first line of characters)

f = open('demo6/1.txt', 'r', encoding='utf-8')
data2 = f.readline(3)
print(data2)
f.close()

4. Execution result:

3 . 1 . 6. Read all the content of the file through readlines and return the list

1. Sample code: (When there is no parameter or the parameter is 0 by default, all the contents of the file are read and output as a list)

f = open('demo6/1.txt', 'r', encoding='utf-8')
data2 = f.readlines()
print(data2)
f.close()

2. Execution result:

3. Sample code: (When there is no parameter or parameter >0 by default, read the first line of the file and carry '\n', and output it as a list) 

f = open('demo6/1.txt', 'r', encoding='utf-8')
data2 = f.readlines(2)
print(data2)
f.close()

4. Execution result:

3 . 1 . 7. Write data through write

1. Sample code

f = open('demo6/3.txt', 'w', encoding='utf-8')
data = 'aaaaaaaa'
data2 = f.write(data)
f.close()

2. Execution result:

If the file 3.txt does not exist, create a 3.txt file and write the content;

If the file 3.txt exists and contains content, overwrite the original content of the 3.txt file and write the content;

3.1.8 . Write a list into a file through writelines

1. Sample

f = open('demo6/3.txt', 'w', encoding='utf-8')
data = 'aaaaaaaa'
data2 = f.write(data)
f.close()

2. Execution result:

If the file 2.txt does not exist, create a 2.txt file and write the content;

If the file 2.txt exists and contains content, overwrite the original content of the 2.txt file and write the content;

3. If the parameter 'w' is changed to 'a+', it will append to the original file and continue to write without overwriting (note that the first line added and the last line of the original file will also be together if there is no line break)

3.2. JSON file

3 . 1 . 1. Features of JSON files

1. JSON is a syntax for storing and exchanging text information, similar to XML;
2. JSON is smaller, faster, and easier to parse than XML;
3. JSON is a lightweight text data exchange format;
4. JSON is language independent;
5. JSON is self-describing and easier to understand;

3.1.2 . Grammatical rules of JSON

1. JSON data is enclosed in braces;
2. The data is in a key:value pair, and the name and value are separated by a colon, similar to a dictionary in python;
3. The name should be enclosed in double quotation marks, and the value should be more data type to determine whether it needs to be quoted; for example: int data is not required, but str type is required;
4. data (a set of key:value pairs) separated by commas;

3. 1. 3. JSON value type

1. Number (integer or floating point) such as {'age': null}

2. String (in double quotes) such as {'name': 'zsk'}

3. Logical value (True or False) such as {'flag': True}

4. Array (in square brackets) such as {'S': ['name', 'site']}

5. Object (in braces) such as {'N': {'num': 100}}

6、null，                  如{‘age’: null} 

3.1.4, JSON module function

1、dumps

Function: convert dictionary to string

Sample code:

import json

dict1 = {'name':'zhangsan', 'age':18}
print(dict1)
print(type(dict1))

# 转成字符串
j1 = json.dumps(dict1)
print(j1)
print(type(j1))

Results of the

2、dump

     Function: convert the field into a string and write it into a JSON file

Sample code:

import json

dict1 = {'name':'zhangsan', 'age':18}
print(dict1)
print(type(dict1))

with open('222.txt', 'w') as f:
    j1 = json.dump(dict1, f)
    print(j1)
    print(type(j1))

 Execution result: (file 222.txt is generated and the content is written in)

3、loads

     Function: convert a string into a dictionary

     Sample code ( the key and value in {} should be enclosed in double quotes (the value is added with quotes depending on the value type), single quotes will report an error)

import json

# {}里的key和value要用双引号引起来（value看值类型添加引号），单引号会报错
str1 = '{"name":"zhangsan", "age":18}'
print(str1)
print(type(str1))

# 将字符串转成字典
dic = json.loads(str1)
print(dic)
print(type(dic))

Results of the

4、load

     Function: Open the file and convert the string into a data type

     Sample code:

import json


with open('1.txt', 'r', encoding='utf-8') as f:
    # f = f.read()
    print(f)
    print(type(f))
    dic1 = json.load(f)
    print(dic1)
    print(type(dic1))

Execution result : 

3.1.5 . Read dictionary type data files

1. JSON file

{
  "user1":{"name":"zhangsan1", "age":18},
  "user2":{"name":"zhangsan2", "age":20}
}

2. Sample code:

import json

file = '2.json'

with open(file, 'r') as f:
    users = json.load(f)
    print(type(f))
    print(type(users))
    print(users)

for user in users:
    print(user)
    name = users[user]['name']
    age = users[user]['age']
    print(name, age)

3. Execution result:

3 . 1 . 6. Read list type data files

1. Example file

[
  {
    "name": "zhangsan1",
    "age": 18
  },
  {
    "name":"zhangsan2", 
    "age":20
  }
]

 2. Sample code:

import json

file = '2.json'
with open(file, 'r') as f:
    ss = json.load(f)
    print(ss)
    print(type(ss))


for s in ss:
    print(s)
    print(type(s))
    print(s['name'])
    print(s['age'])

3. Execution result:

3.3 . YAML file

3.3.1 . Features of YAML file

Features: Use blank characters to indicate indentation, and the file extension is ".yaml";

3 . 3 . 2. Grammatical rules of YAML files

1. Case Sensitive;
2. Use indentation to indicate hierarchical relationships;
3. Indentation cannot use the Tab key, only the blank key;
4. The number of spaces indented is not important, as long as the elements at the same level are left-aligned;
5. '#' means comment;

3. 3. 3. Data types supported by YAML files

1. YAML object

      Object: a collection of key-value pairs, also known as a map, hash, dictionary;

2. YAML array

     Array: a set of values ​​arranged in order, also known as sequence, list;

3. YAML file - YAMl object;

Example file: (no space before colon, one space after colon)

name: 'zhangsan'
age: 10

Sample code:

import yaml

with open('1.yaml', 'r', encoding='UTF-8') as f:
    data = yaml.load(f, Loader=yaml.FullLoader)
    print(data)
    print(type(data))
    print(data['name'])
    print(data['age'])

Results of the:

4. YAML file--YAMl array

Example file:

- name
- age
- std_id
- pag

 Sample code: 

import yaml

with open('2.yaml', 'r', encoding='UTF-8') as f:
    data = yaml.load(f, Loader=yaml.FullLoader)
    print(data)
    print(type(data))

   Results of the:

5. YAML file - compound structure

Example file:

school:
  name: zhangsan
  age: 18
  stu_id: 10223

 Sample code:

import yaml

with open('3.yaml', 'r', encoding='UTF-8') as f:
    data = yaml.load(f, Loader=yaml.FullLoader)
    print(data)
    print(type(data))
    print(data['school'])
    print(type(data['school']))
    print(data['school']['name'])
    print(data['school']['age'])
    print(data['school']['stu_id'])

 Results of the:

3.4 . CSV file

1. If you open it with Excel, you will find that it is the same as the '.xls' and '.xlsx' files, but the csv file can be opened with Notepad;

2. Example file:

name,age,stu_id
zhangsan,18,10234
zhangsan2,19,10235
zhangsan3,20,10236

 3. Sample code:

import csv

with open('1.csv', 'r', encoding='utf-8') as f:
    data = csv.reader(f)
    print(data)
    for i in data:
        print(i)

4. Execution result:

5. Write data to CSV file

6. Sample code

# -*- coding: gbk -*-
import csv



def Writer_CSV(title1_value, title2_value, title3_value, title4_value, title5_value, title6_value, title7_value, title8_value):
    file_name = './result_to_csv.csv'
    with open(file_name, 'a+', newline='') as csvfile:
        writer = csv.writer(csvfile)
        with open(file_name, 'r', newline='') as f:
            reader = csv.reader(f)
            # writer.writerow(['title1', 'title2', 'title3', 'title4', 'title5', 'title6', 'title7', 'title8'])
            # writer.writerow([title1_value, title2_value, title3_value, title4_value, title5_value, title6_value, title7_value, title8_value])

            #防止重复写入标题（如果首行为空，则把标题写入首行，在往下按行插入数据；如果首行不为空，则直接按行往下插入数据）
            if not [row for row in reader]:
                writer.writerow(['title1', 'title2', 'title3', 'title4', 'title5', 'title6', 'title7', 'title8'])
                writer.writerow([title1_value, title2_value, title3_value, title4_value, title5_value, title6_value, title7_value, title8_value])
            else:
                writer.writerow([title1_value, title2_value, title3_value, title4_value, title5_value, title6_value, title7_value, title8_value])

        f.close()
    csvfile.close()


Writer_CSV(1, 2, 3, 4, 5, 6, 7, 8)

7. Running result:

Generate a CSV file in the current directory: result_to_csv.csv; if it already exists in the current directory, it will write additional data in the current file;

8. The role of "if not [row for row in reader]:": In order to prevent repeated writing of the title:

Sample code:

# -*- coding: gbk -*-
import csv



def Writer_CSV(title1_value, title2_value, title3_value, title4_value, title5_value, title6_value, title7_value, title8_value):
    file_name = '/Users/lichuanwei/Project_Pytest/BUNDLE_Project/result_to_csv.csv'
    with open(file_name, 'a+', newline='') as csvfile:
        writer = csv.writer(csvfile)
        with open(file_name, 'r', newline='') as f:
            reader = csv.reader(f)
            writer.writerow(['title1', 'title2', 'title3', 'title4', 'title5', 'title6', 'title7', 'title8'])
            writer.writerow([title1_value, title2_value, title3_value, title4_value, title5_value, title6_value, title7_value, title8_value])

            

        f.close()
    csvfile.close()


Writer_CSV(1, 2, 3, 4, 5, 6, 7, 8)

Output result: the title is written repeatedly