Machine learning model persistence-pickle and joblib

             Scenario: save the trained model to local disk or memory

 

pickle

The pickle module and its peer module cPickle provide pickle support to Python. The latter is coded in C, which has better performance. For most applications, this module is recommended.

joblib

joblib is an external library of scikit-learn. For big data, joblib is more efficient than pickle, but joblib can only store objects in disk files, not as strings.

 

• Implement model persistence through joblib

#!/usr/bin/env python
# -*- coding: utf-8 -*-
__author__ = 'Seven'

from sklearn.tree import DecisionTreeClassifier
from sklearn.externals import joblib

# 构建简单数据集
# x = [[2, 3, 4], [4, 6, 5]]
# y = [1, 0]
# 构建决策树模型
clf = DecisionTreeClassifier()
# 模型训练
# res = clf.fit(x, y)

# 可以通过print查看训练后模型的参数
print(res)


# 保存至本地磁盘
# joblib.dump(clf, 'model.pkl')
# 从本地磁盘加载模型
model_joblib = joblib.load('model.pkl')
# 加载出来的模型可以进行predict等功能
print(model_joblib.predict([[4, 6, 10]]))

The operation effect is as follows:

• Model persistence through pickle

#!/usr/bin/env python
# -*- coding: utf-8 -*-
__author__ = 'Seven'

from sklearn.tree import DecisionTreeClassifier
import pickle

# 构建简单数据集
x = [[2, 3, 4], [4, 6, 5]]
y = [1, 0]
# 构建决策树模型
clf = DecisionTreeClassifier()
# 模型训练
res = clf.fit(x, y)

# 可以通过print查看训练后模型的参数
print(res)


# 保存至本地磁盘
# with open('model.pkl', 'wb') as file:
#     pickle.dump(clf, file)
# 从本地磁盘加载模型
with open('model.pkl', 'rb') as file:
    model_joblib = pickle.load(file)
    # 加载出来的模型可以进行predict等功能
    print(model_joblib.predict([[4, 6, 10]]))

# pickle还可以保存至内存
s = pickle.dumps(clf)
model = pickle.loads(s)
print(model.predict([[4, 6, 10]]))

The operation effect is as follows: