[Python web crawler] 150 lectures to easily get the Python web crawler paid course notes chapter 13-data storage: JSON string format

1. JSON overview

JSON (JavaScript Object Notation, JS object notation) is a lightweight data exchange format. The concise and clear hierarchical structure makes JSON an ideal data exchange language.

Data formats supported by JSON:

• Object (dictionary): curly braces
• Array (list): square brackets
• String type: double quotes must be used, single quotes cannot be used
• Integer, floating point, boolean, null, etc.
• Use commas to separate multiple data
• json is essentially a string

Online json parser: http://json.cn/

{"Person": [{"username": "张三", "age":18}]}

json :
{
    "Person":[
        {
            "username":"张三",
            "age":18
        }
    ]
}

 

2. json commonly used functions

2.1 Convert python object to json string

• dumps : Convert python objects into json format strings;
• the dump : converting python json object into a format string, and may also receive a file pointer fp parameters can be written to a file in

Both methods have a parameter, ensure_ascii  is True by default, which means that only ASCII characters can be stored by default; if you want to store Chinese, set the ensure_ascii parameter to False.

When writing python objects, you don’t need to pay extra attention to follow the json format requirements, just write python correctly, because the dumps/dump function will automatically convert python to json format.

import json

books = [
    {
        "name": "三国演义",
        "price": 18.8
    },
    {
        "name": "水浒传",
        "price": 19.9
    }
]
# dumps
result = json.dumps(books, ensure_ascii=False)
print(result, type(result))

#dump
fp = open('books.json', 'w', encoding='utf-8')
json.dump(books, fp, ensure_ascii=False)
fp.close() 
#open 之后需要手动close文件

2.2 Convert json string to python object

• loads : convert json strings into python objects
• load : Convert the json string into a python object, and you can get the json string directly from the file

import json

json_str = '[{"name": "三国演义", "price": 18.8}, {"name": "水浒传", "price": 19.9}]'

print(type(json_str))
result = json.loads(json_str)
print(result)
print(type(result))

print('*'*20)
# load
with open('books.json', 'r', encoding='utf-8') as fp:
    result = json.load(fp)
    print(result)
    print(type(result))

Use with open to open the file without manually closing the file

And when obtaining the content of the file, it is best to specify the parsing code, otherwise it will be easy to report errors