[Pickle] Python saves any type of intermediate results

In the process of processing data with python, it often involves intermediate results with a relatively large amount of calculation but not much change. If you run the calculation every time you want to get this intermediate result, it will be a waste of time (such as pre-processed training data). At this time, I will think that if there is a convenient way to save these intermediate results, there is no need to repeat calculations .

For intermediate results of str type or simple list type, we can directly save them in txt files; for intermediate results of dict type, we can save them in json. But, what if the intermediate result is a special class? Even classes you build yourself. It may have a very complicated logical relationship, not necessarily in the form of a sequence. How to save such intermediate results?

Then pickle comes. Pickle can save intermediate results in any form . It can be saved directly in binary. Look at the example directly:

import pickle

class idol:
    def __init__(self, sing, dance, rap, basketball):
        self.sing = sing
        self.dance = dance
        self.rap = rap
        self.basketball = basketball

cxk = idol(10, 10, 10, 10)
f = open('cxk.pkl', 'wb')
content = pickle.dumps(cxk)
f.write(content)
f.close()

First downgrade the intermediate data cxk to binary without a logical structure, and then write it to the pickle file.

Read as follows:

import pickle

class idol:
    def __init__(self, sing, dance, rap, basketball):
        self.sing = sing
        self.dance = dance
        self.rap = rap
        self.basketball = basketball

f = open('cxk.pkl', 'rb')
cxk = pickle.loads(f.read())
f.close()
print(cxk.basketball)

---

Pickle is stored in a list of support classes and can handle large-scale data sets.

Save the class list:

import pickle
class idol:
    def __init__(self, sing, dance, rap, basketball):
        self.sing = sing
        self.dance = dance
        self.rap = rap
        self.basketball = basketball

cxk = idol(10, 10, 10, 10)
ls = [cxk, cxk]
f = open('cxk_list.pkl', 'wb')
content = pickle.dumps(ls)
f.write(content)
f.close()

Read the classes in the list:

import pickle
class idol:
    def __init__(self, sing, dance, rap, basketball):
        self.sing = sing
        self.dance = dance
        self.rap = rap
        self.basketball = basketball
f = open('cxk_list.pkl', 'rb')
cxk = pickle.loads(f.read())
f.close()
print(cxk[0].basketball)

This is a very practical trick~