tensorflow2.0 regression model --- How good keras api for the sklearn

Written before how to build a model with tf.keras, planted a foreshadowing that time, the problem is ultra-parameter search. How to get the best model, we use sklearn it is the GridSearchCV or RandomizedSearchCV, so today I want to talk about how to implement super-parametric searching through the api tf.keras.

1. Introduction to see official documents

Official documents

found just call the api to write a build_fn, it is to write a function to build the network, after knowing this would be the actual look.

2. ultra-parameter search

1. First import data sets, I use the California Housing dataset
2. Segmentation good training set, validation and test sets, and data standardization
3. We need to write network model, call the keras.wrappers.scikit_learn.KerasRegressor model method can be used to achieve sklearn
4. Get the best training and model parameters

Code:

import pandas as pd
import numpy as np
import tensorflow as tf
from tensorflow import keras
import matplotlib
import matplotlib.pyplot as plt
import sklearn
import os,sys
from sklearn.datasets import fetch_california_housing

housing=fetch_california_housing()

house=pd.DataFrame(housing.data)
house.columns=housing.feature_names
house['price']=housing.target
house.info()
house.head(10)


from sklearn.model_selection import train_test_split

x_train_all,x_test,y_train_all,y_test=train_test_split(housing.data,housing.target,test_size=0.25,random_state=2)
x_train,x_valid,y_train,y_valid=train_test_split(x_train_all,y_train_all,test_size=0.25,random_state=2)



#标准化
from sklearn.preprocessing import StandardScaler

scaler=StandardScaler()
x_train_scaled=scaler.fit_transform(x_train)
x_valid_scaled=scaler.fit_transform(x_valid)
x_test_scaled=scaler.fit_transform(x_test)


#构建自己的模型
def build_model(hidden_layers=1,layer_size=30,learning_rate=3e-3):
    model=keras.models.Sequential()
    model.add(keras.layers.Dense(layer_size,activation="relu",input_shape=x_train.shape[1:]))
    for _ in range(hidden_layers-1):
        model.add(keras.layers.Dense(layer_size,activation="relu"))
    model.add(keras.layers.Dense(1))
    optimizer=keras.optimizers.SGD(learning_rate)
    model.compile(loss="mse",optimizer=optimizer)
    return model


import datetime
current_time = datetime.datetime.now().strftime("%Y%m%d-%H%M%S")
logdir = os.path.join('logs', current_time)
callbacks=[
    keras.callbacks.EarlyStopping(patience=3,min_delta=1e-3),
    keras.callbacks.TensorBoard(log_dir=logdir)
]

model=keras.wrappers.scikit_learn.KerasRegressor(build_model)

callback=[keras.callbacks.EarlyStopping(patience=3,min_delta=1e-3)]
#使用默认参数的模型
history=model.fit(x_train_scaled,y_train,validation_data=(x_valid_scaled,y_valid),epochs=100,callbacks=callback)


def plot_learning_curves(history):
    pd.DataFrame(history.history).plot(figsize=(8,5))
    plt.grid()
    plt.gca().set_ylim(0,1)
    plt.show()

plot_learning_curves(history)


#实现超参数搜索
from scipy.stats import reciprocal

param_distribution={
    "hidden_layers":[3,4],
    "layer_size":np.arange(24,27),
    "learning_rate":reciprocal(0.001,0.005)
}

#随机搜索
from sklearn.model_selection import RandomizedSearchCV

random_searchcv=RandomizedSearchCV(model,param_distribution,n_iter=10,verbose=0)
random_searchcv.fit(x_train_scaled,y_train,validation_data=(x_valid_scaled,y_valid),epochs=100)


print("得到的最好参数为:",random_searchcv.best_params_)
print("最好的得分为:",random_searchcv.best_score_)



model=random_searchcv.best_estimator_.model
history1=model.fit(x_train_scaled,y_train,validation_data=(x_valid_scaled,y_valid),epochs=100,callbacks=callbacks)

plot_learning_curves(history1)

print(model.evaluate(x_test_scaled,y_test,verbose=0))

The effect of the default parameters of the model

using random search results

Open tensorboard look model

3. Summary:

Although the use of ultra-parameter search is very convenient, but there are some caveats. Parameters such as the need to know what is more important, it is as accurate as possible; the contrary requirements can be somewhat relaxed. After all, whether it is using a grid search or random search parameters if too much can lead to long computation time. BTW, when using ultra-parametric search, I want to n_jobs set to greater than 1, will run error, we have not found a good solution, which also led to the calculation of time data will be even longer, after all, tune reference is also a need to experience things that duck!