記事ディレクトリ

ドロップアウトを使用してオーバーフィッティングを抑制する
添付ファイル: 一連の記事

ドロップアウトを使用してオーバーフィッティングを抑制する

ドロップアウトは一般的に使用されるニューラルネットワークの正則化方法であり、主に過学習を防ぐために使用されます。深層学習では、ネットワーク層とパラメーターが多すぎるため、モデルが過剰適合する傾向があり、テスト中に大きな汎化エラーが発生します。Dropout 法は、アンサンブル学習における Bagging のアイデアを利用し、一部のニューロンの出力をランダムに 0 に設定することで、過剰学習の可能性を減らします。

ドロップアウト法はヒントンらによって最初に提案されたもので、その基本的な考え方は、トレーニング中にネットワーク内の一部のニューロンの出力を一定の確率でランダムに 0 に設定するというものです。このランダムな動作は、ネットワークの部分的な枝刈りとして見なすことができ、これによりネットワークの耐性が向上し、ネットワークがより堅牢になります。また、ネットワーク内の特定のニューロンがネットワーク全体に対して過剰に使用されることも回避されます。

Dropout メソッドの具体的な実装は次のとおりです。各トレーニングプロセスで、一部のニューロンが特定の確率 p でランダムに選択され、0 に設定されます。選択されたニューロンは、後続のトレーニングとバックプロパゲーションには参加しません。テスト中は、モデルの安定性と一貫性を維持するために、通常、ランダム化は採用されず、代わりに、各ニューロンの重みに確率 p (p はトレーニング中に選択された確率) が乗算されます。

ドロップアウト法は、完全接続ネットワークだけでなく、畳み込みニューラルネットワークやリカレントニューラルネットワークでも使用して、過学習を軽減できます。さらに、その実装は単純であり、モデルの学習中に各ニューロンを確率 p でランダムに選択して保持するだけでよいため、Dropout 法は広く使用され、普及しています。

つまり、ドロップアウト法はモデルの精度と汎化能力をある程度向上させることができ、過学習の防止に優れた効果を発揮します。ただし、ドロップアウト法では、トレーニングプロセス中に各ミニバッチの勾配が異なるため、ドロップアウト法を使用する場合は、モデルの収束速度と効果を確保するために学習率を調整する必要があることに注意してください。。

1. 環境の準備

# 导入库
import keras
from keras import layers
import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
%matplotlib inline

Using TensorFlow backend.

2. データセットをインポートする

# 导入数据集
data = pd.read_csv('./dataset/credit-a.csv', header=None)
data

	0	1	2	3	4	5	6	7	8	9	10	11	12	13	14	15
0	0	30.83	0.000	0	0	9	0	1.250	0	0	1	1	0	202	0.0	-1
1	1	58.67	4.460	0	0	8	1	3.040	0	0	6	1	0	43	560.0	-1
2	1	24.50	0.500	0	0	8	1	1.500	0	1	0	1	0	280	824.0	-1
3	0	27.83	1.540	0	0	9	0	3.750	0	0	5	0	0	100	3.0	-1
4	0	20.17	5.625	0	0	9	0	1.710	0	1	0	1	2	120	0.0	-1
5	0	32.08	4.000	0	0	6	0	2.500	0	1	0	0	0	360	0.0	-1
6	0	33.17	1.040	0	0	7	1	6.500	0	1	0	0	0	164	31285.0	-1
7	1	22.92	11.585	0	0	2	0	0.040	0	1	0	1	0	80	1349.0	-1
8	0	54.42	0.500	1	1	5	1	3.960	0	1	0	1	0	180	314.0	-1
9	0	42.50	4.915	1	1	9	0	3.165	0	1	0	0	0	52	1442.0	-1
10	0	8月22日	0.830	0	0	0	1	2.165	1	1	0	0	0	128	0.0	-1
11	0	29.92	1.835	0	0	0	1	4.335	0	1	0	1	0	260	200.0	-1
12	1	38.25	6.000	0	0	5	0	1.000	0	1	0	0	0	0	0.0	-1
13	0	48.08	6.040	0	0	5	0	0.040	1	1	0	1	0	0	2690.0	-1
14	1	45.83	10.500	0	0	8	0	5.000	0	0	7	0	0	0	0.0	-1
15	0	36.67	4.415	1	1	5	0	0.250	0	0	10	0	0	320	0.0	-1
16	0	28.25	0.875	0	0	6	0	0.960	0	0	3	0	0	396	0.0	-1
17	1	23.25	5.875	0	0	8	0	3.170	0	0	10	1	0	120	245.0	-1
18	0	21.83	0.250	0	0	1	1	0.665	0	1	0	0	0	0	0.0	-1
19	1	19.17	8.585	0	0	2	1	0.750	0	0	7	1	0	96	0.0	-1
20	0	25.00	11.250	0	0	0	0	2.500	0	0	17	1	0	200	1208.0	-1
21	0	23.25	1.000	0	0	0	0	0.835	0	1	0	1	2	300	0.0	-1
22	1	47.75	8.000	0	0	0	0	7.875	0	0	6	0	0	0	1260.0	-1
23	1	27.42	14.500	0	0	10	1	3.085	0	0	1	1	0	120	11.0	-1
24	1	41.17	6.500	0	0	8	0	0.500	0	0	3	0	0	145	0.0	-1
25	1	15.83	0.585	0	0	0	1	1.500	0	0	2	1	0	100	0.0	-1
26	1	47.00	13.000	0	0	3	2	5.165	0	0	9	0	0	0	0.0	-1
27	0	56.58	18.500	0	0	1	2	15.000	0	0	17	0	0	0	0.0	-1
28	0	57.42	8.500	0	0	11	1	7.000	0	0	3	1	0	0	0.0	-1
29	0	42.08	1.040	0	0	9	0	5.000	0	0	6	0	0	500	10000.0	-1
...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...
623	1	28.58	3.750	0	0	0	0	0.250	1	0	1	0	0	40	154.0	1
624	0	22.25	9.000	0	0	12	0	0.085	1	1	0	1	0	0	0.0	1
625	0	29.83	3.500	0	0	0	0	0.165	1	1	0	1	0	216	0.0	1
626	1	23.50	1.500	0	0	9	0	0.875	1	1	0	0	0	160	0.0	1
627	0	32.08	4.000	1	1	2	0	1.500	1	1	0	0	0	120	0.0	1
628	0	8月31日	1.500	1	1	9	0	0.040	1	1	0	1	2	160	0.0	1
629	0	31.83	0.040	1	1	6	0	0.040	1	1	0	1	0	0	0.0	1
630	1	21.75	11.750	0	0	0	0	0.250	1	1	0	0	0	180	0.0	1
631	1	17.92	0.540	0	0	0	0	1.750	1	0	1	0	0	80	5.0	1
632	0	30.33	0.500	0	0	1	1	0.085	1	1	0	0	2	252	0.0	1
633	0	51.83	2.040	1	1	13	7	1.500	1	1	0	1	0	120	1.0	1
634	0	47.17	5.835	0	0	9	0	5.500	1	1	0	1	0	465	150.0	1
635	0	25.83	12.835	0	0	2	0	0.500	1	1	0	1	0	0	2.0	1
636	1	50.25	0.835	0	0	12	0	0.500	1	1	0	0	0	240	117.0	1
637	1	37.33	2.500	0	0	3	1	0.210	1	1	0	1	0	260	246.0	1
638	1	41.58	1.040	0	0	12	0	0.665	1	1	0	1	0	240	237.0	1
639	1	30.58	10.665	0	0	8	1	0.085	1	0	12	0	0	129	3.0	1
640	0	19.42	7.250	0	0	6	0	0.040	1	0	1	1	0	100	1.0	1
641	1	17.92	10.210	0	0	13	7	0.000	1	1	0	1	0	0	50.0	1
642	1	20.08	1.250	0	0	0	0	0.000	1	1	0	1	0	0	0.0	1
643	0	19.50	0.290	0	0	5	0	0.290	1	1	0	1	0	280	364.0	1
644	0	27.83	1.000	1	1	1	1	3.000	1	1	0	1	0	176	537.0	1
645	0	17.08	3.290	0	0	3	0	0.335	1	1	0	0	0	140	2.0	1
646	0	36.42	0.750	1	1	1	0	0.585	1	1	0	1	0	240	3.0	1
647	0	40.58	3.290	0	0	6	0	3.500	1	1	0	0	2	400	0.0	1
648	0	21.08	10.085	1	1	11	1	1.250	1	1	0	1	0	260	0.0	1
649	1	22.67	0.750	0	0	0	0	2.000	1	0	2	0	0	200	394.0	1
650	1	25.25	13.500	1	1	13	7	2.000	1	0	1	0	0	200	1.0	1
651	0	17.92	0.205	0	0	12	0	0.040	1	1	0	1	0	280	750.0	1
652	0	35.00	3.375	0	0	0	1	8.290	1	1	0	0	0	0	0.0	1

653 rows × 16 columns

data.iloc[:, -1].unique()

array([-1,  1])

3. 对所有数据的预测

3.1 数据集

x = data.iloc[:, :-1].values
y = data.iloc[:, -1].replace(-1, 0).values.reshape(-1, 1)

x.shape, y.shape

((653, 15), (653, 1))

3.2 构建神经网络

model = keras.Sequential()
model.add(layers.Dense(128, input_dim=15, activation='relu'))
model.add(layers.Dense(128, activation='relu'))
model.add(layers.Dense(128, activation='relu'))
model.add(layers.Dense(1, activation='sigmoid'))

model.summary()

Model: "sequential_1"
_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
dense_1 (Dense)              (None, 128)               2048      
_________________________________________________________________
dense_2 (Dense)              (None, 128)               16512     
_________________________________________________________________
dense_3 (Dense)              (None, 128)               16512     
_________________________________________________________________
dense_4 (Dense)              (None, 1)                 129       
=================================================================
Total params: 35,201
Trainable params: 35,201
Non-trainable params: 0
_________________________________________________________________

model.compile(optimizer='adam',
              loss='binary_crossentropy',
              metrics=['acc']
)

WARNING:tensorflow:From /home/nlp/anaconda3/lib/python3.7/site-packages/tensorflow/python/ops/nn_impl.py:180: add_dispatch_support.<locals>.wrapper (from tensorflow.python.ops.array_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Use tf.where in 2.0, which has the same broadcast rule as np.where

3.3 训练模型

history = model.fit(x, y, epochs=1000)

WARNING:tensorflow:From /home/nlp/anaconda3/lib/python3.7/site-packages/keras/backend/tensorflow_backend.py:422: The name tf.global_variables is deprecated. Please use tf.compat.v1.global_variables instead.

Epoch 1/1000
653/653 [==============================] - 0s 434us/step - loss: 7.5273 - acc: 0.5988
Epoch 2/1000
653/653 [==============================] - 0s 92us/step - loss: 3.7401 - acc: 0.6187
Epoch 3/1000
653/653 [==============================] - 0s 75us/step - loss: 3.6464 - acc: 0.5712
Epoch 4/1000
653/653 [==============================] - 0s 56us/step - loss: 10.2291 - acc: 0.6631
Epoch 5/1000
653/653 [==============================] - 0s 63us/step - loss: 2.0400 - acc: 0.6233
Epoch 6/1000
653/653 [==============================] - 0s 120us/step - loss: 2.4279 - acc: 0.6217
Epoch 7/1000
653/653 [==============================] - 0s 105us/step - loss: 2.3289 - acc: 0.6325
Epoch 8/1000
653/653 [==============================] - 0s 159us/step - loss: 3.2521 - acc: 0.6294
Epoch 9/1000
653/653 [==============================] - 0s 89us/step - loss: 2.6005 - acc: 0.6294
Epoch 10/1000
653/653 [==============================] - 0s 118us/step - loss: 1.3997 - acc: 0.6738
……
Epoch 1000/1000
653/653 [==============================] - 0s 106us/step - loss: 0.2630 - acc: 0.9326

3.4 分析模型

history.history.keys()

dict_keys(['loss', 'acc'])

plt.plot(history.epoch, history.history.get('loss'), c='r')
plt.plot(history.epoch, history.history.get('acc'), c='b')

[<matplotlib.lines.Line2D at 0x7fd43c1597f0>]

4. 对未见过数据的预测

4.1 划分数据集

x_train = x[:int(len(x)*0.75)]
x_test = x[int(len(x)*0.75):]
y_train = y[:int(len(x)*0.75)]
y_test = y[int(len(x)*0.75):]

x_train.shape, x_test.shape, y_train.shape, y_test.shape

((489, 15), (164, 15), (489, 1), (164, 1))

4.2 构建神经网络

model = keras.Sequential()

model.add(layers.Dense(128, input_dim=15, activation='relu'))
model.add(layers.Dense(128, activation='relu'))
model.add(layers.Dense(128, activation='relu'))
model.add(layers.Dense(1, activation='sigmoid'))

#admam:利用梯度的一阶矩估计和二阶矩估计动态调整每个参数的学习率.
model.compile(optimizer='adam',
              loss='binary_crossentropy',
              metrics=['acc']
)

4.3 训练模型

history = model.fit(x_train, y_train, epochs=1000, validation_data=(x_test, y_test))

Train on 489 samples, validate on 164 samples
Epoch 1/1000
489/489 [==============================] - 0s 487us/step - loss: 14.4564 - acc: 0.5951 - val_loss: 3.3778 - val_acc: 0.7256
Epoch 2/1000
489/489 [==============================] - 0s 110us/step - loss: 6.0909 - acc: 0.6012 - val_loss: 2.0924 - val_acc: 0.7195
Epoch 3/1000
489/489 [==============================] - 0s 195us/step - loss: 2.4527 - acc: 0.6074 - val_loss: 1.0763 - val_acc: 0.7378
Epoch 4/1000
489/489 [==============================] - 0s 183us/step - loss: 1.0751 - acc: 0.6585 - val_loss: 0.8990 - val_acc: 0.7134
Epoch 5/1000
489/489 [==============================] - 0s 155us/step - loss: 1.2669 - acc: 0.6503 - val_loss: 1.8094 - val_acc: 0.6585
Epoch 6/1000
489/489 [==============================] - 0s 202us/step - loss: 3.6742 - acc: 0.6892 - val_loss: 1.1836 - val_acc: 0.4573
Epoch 7/1000
489/489 [==============================] - 0s 166us/step - loss: 1.7544 - acc: 0.7301 - val_loss: 2.0060 - val_acc: 0.4573
Epoch 8/1000
489/489 [==============================] - 0s 185us/step - loss: 1.4768 - acc: 0.6605 - val_loss: 0.8917 - val_acc: 0.5427
Epoch 9/1000
489/489 [==============================] - 0s 163us/step - loss: 1.6829 - acc: 0.6667 - val_loss: 4.7695 - val_acc: 0.4573
Epoch 10/1000
489/489 [==============================] - 0s 157us/step - loss: 8.4323 - acc: 0.7239 - val_loss: 2.0879 - val_acc: 0.7439
……
Epoch 1000/1000
489/489 [==============================] - 0s 97us/step - loss: 0.0272 - acc: 0.9877 - val_loss: 2.2746 - val_acc: 0.8049

4.4 分析模型

history.history.keys()

dict_keys(['val_loss', 'val_acc', 'loss', 'acc'])

plt.plot(history.epoch, history.history.get('val_acc'), c='r', label='val_acc')
plt.plot(history.epoch, history.history.get('acc'), c='b', label='acc')
plt.legend()

<matplotlib.legend.Legend at 0x7fd417ff5978>

model.evaluate(x_train, y_train)

489/489 [==============================] - 0s 37us/step
[0.021263938083038197, 0.9897750616073608]

model.evaluate(x_test, y_test)

164/164 [==============================] - 0s 46us/step
[2.274633582976715, 0.8048780560493469]

过拟合：在训练数据正确率非常高，在测试数据上比较低

5. 使用Dropout抑制过拟合

5.1 构建神经网络

model = keras.Sequential()
model.add(layers.Dense(128, input_dim=15, activation='relu'))
model.add(layers.Dropout(0.5))
model.add(layers.Dense(128, activation='relu'))
model.add(layers.Dropout(0.5))
model.add(layers.Dense(128, activation='relu'))
model.add(layers.Dropout(0.5))
model.add(layers.Dense(1, activation='sigmoid'))

model.summary()

Model: "sequential_3"
_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
dense_9 (Dense)              (None, 128)               2048      
_________________________________________________________________
dropout_1 (Dropout)          (None, 128)               0         
_________________________________________________________________
dense_10 (Dense)             (None, 128)               16512     
_________________________________________________________________
dropout_2 (Dropout)          (None, 128)               0         
_________________________________________________________________
dense_11 (Dense)             (None, 128)               16512     
_________________________________________________________________
dropout_3 (Dropout)          (None, 128)               0         
_________________________________________________________________
dense_12 (Dense)             (None, 1)                 129       
=================================================================
Total params: 35,201
Trainable params: 35,201
Non-trainable params: 0
________________________________________________________________

model.compile(optimizer='adam',
              loss='binary_crossentropy',
              metrics=['acc']
)

5.2 训练模型

history = model.fit(x_train, y_train, epochs=1000, validation_data=(x_test, y_test))

Train on 489 samples, validate on 164 samples
Epoch 1/1000
489/489 [==============================] - 1s 1ms/step - loss: 41.6885 - acc: 0.5378 - val_loss: 9.9666 - val_acc: 0.6768
Epoch 2/1000
489/489 [==============================] - 0s 298us/step - loss: 53.1358 - acc: 0.5358 - val_loss: 11.0265 - val_acc: 0.6951
Epoch 3/1000
489/489 [==============================] - 0s 173us/step - loss: 36.9899 - acc: 0.5828 - val_loss: 11.6578 - val_acc: 0.6890
Epoch 4/1000
489/489 [==============================] - 0s 177us/step - loss: 43.3404 - acc: 0.5808 - val_loss: 7.5652 - val_acc: 0.6890
Epoch 5/1000
489/489 [==============================] - 0s 197us/step - loss: 23.3085 - acc: 0.6196 - val_loss: 7.9913 - val_acc: 0.6890
Epoch 6/1000
489/489 [==============================] - 0s 254us/step - loss: 24.1833 - acc: 0.6155 - val_loss: 5.5747 - val_acc: 0.7073
Epoch 7/1000
489/489 [==============================] - 0s 229us/step - loss: 19.7051 - acc: 0.5890 - val_loss: 5.5711 - val_acc: 0.7012
Epoch 8/1000
489/489 [==============================] - 0s 180us/step - loss: 22.1131 - acc: 0.5849 - val_loss: 7.0290 - val_acc: 0.6890
Epoch 9/1000
489/489 [==============================] - 0s 172us/step - loss: 23.2305 - acc: 0.6115 - val_loss: 4.2624 - val_acc: 0.6951
Epoch 10/1000
……
Epoch 1000/1000
489/489 [==============================] - 0s 137us/step - loss: 0.3524 - acc: 0.8200 - val_loss: 0.7290 - val_acc: 0.7012

5.3 分析模型

model.evaluate(x_train, y_train)

489/489 [==============================] - 0s 41us/step
[0.3090217998422728, 0.8548057079315186]

model.evaluate(x_test, y_test)

164/164 [==============================] - 0s 64us/step
[0.7289713301309725, 0.7012194991111755]

plt.plot(history.epoch, history.history.get('val_acc'), c='r', label='val_acc')
plt.plot(history.epoch, history.history.get('acc'), c='b', label='acc')
plt.legend()

<matplotlib.legend.Legend at 0x7fd4177c87b8>

6. 正则化

l1：loss = s*abs(w1 + w2 + …) + mse

l2：loss = s*(w12 + w22 + …) + mse

from keras import regularizers

6.1 神经网络太过复杂容易过拟合

#神经网络太过复杂容易过拟合
model = keras.Sequential()
model.add(layers.Dense(128, kernel_regularizer=regularizers.l2(0.001), input_dim=15, activation='relu'))
model.add(layers.Dense(128, kernel_regularizer=regularizers.l2(0.001), activation='relu'))
model.add(layers.Dense(128, kernel_regularizer=regularizers.l2(0.001), activation='relu'))
model.add(layers.Dense(1, activation='sigmoid'))

model.compile(optimizer='adam',
              loss='binary_crossentropy',
              metrics=['acc']
)

history = model.fit(x_train, y_train, epochs=1000, validation_data=(x_test, y_test))

Train on 489 samples, validate on 164 samples
Epoch 1/1000
489/489 [==============================] - 0s 752us/step - loss: 22.2560 - acc: 0.5910 - val_loss: 9.1111 - val_acc: 0.6524
Epoch 2/1000
489/489 [==============================] - 0s 137us/step - loss: 6.8963 - acc: 0.6217 - val_loss: 3.2886 - val_acc: 0.4573
Epoch 3/1000
489/489 [==============================] - 0s 161us/step - loss: 5.0407 - acc: 0.6830 - val_loss: 1.1973 - val_acc: 0.7256
Epoch 4/1000
489/489 [==============================] - 0s 218us/step - loss: 6.6088 - acc: 0.6421 - val_loss: 7.4651 - val_acc: 0.7012
Epoch 5/1000
489/489 [==============================] - 0s 233us/step - loss: 8.3945 - acc: 0.6973 - val_loss: 2.5579 - val_acc: 0.7317
Epoch 6/1000
489/489 [==============================] - 0s 192us/step - loss: 7.0204 - acc: 0.6196 - val_loss: 3.6758 - val_acc: 0.6829
Epoch 7/1000
489/489 [==============================] - 0s 152us/step - loss: 3.9961 - acc: 0.7382 - val_loss: 1.6183 - val_acc: 0.7317
Epoch 8/1000
489/489 [==============================] - 0s 94us/step - loss: 2.3441 - acc: 0.6237 - val_loss: 1.1523 - val_acc: 0.7256
Epoch 9/1000
489/489 [==============================] - 0s 114us/step - loss: 1.8178 - acc: 0.6442 - val_loss: 1.3449 - val_acc: 0.7073
Epoch 10/1000
489/489 [==============================] - 0s 157us/step - loss: 1.6122 - acc: 0.7117 - val_loss: 1.2869 - val_acc: 0.6646
……
Epoch 1000/1000
489/489 [==============================] - 0s 130us/step - loss: 0.1452 - acc: 0.9775 - val_loss: 1.0515 - val_acc: 0.7866
model.evaluate(x_train, y_train)

model.evaluate(x_train, y_train)

489/489 [==============================] - 0s 34us/step
[0.17742264538942426, 0.9611452221870422]

model.evaluate(x_test, y_test)

164/164 [==============================] - 0s 77us/step
[1.0514701096023, 0.7865853905677795]

6.2 太简单容易欠拟合

#太简单容易欠拟合
model = keras.Sequential()
model.add(layers.Dense(4, input_dim=15, activation='relu'))
model.add(layers.Dense(1,  activation='relu'))
model.add(layers.Dense(1, activation='sigmoid'))

model.compile(optimizer='adam',
              loss='binary_crossentropy',
              metrics=['acc']
)

history = model.fit(x_train, y_train, epochs=1000, validation_data=(x_test, y_test))

Train on 489 samples, validate on 164 samples
Epoch 1/1000
489/489 [==============================] - 0s 502us/step - loss: 0.6932 - acc: 0.4765 - val_loss: 0.6931 - val_acc: 0.6341
Epoch 2/1000
489/489 [==============================] - 0s 91us/step - loss: 0.6931 - acc: 0.5174 - val_loss: 0.6930 - val_acc: 0.6341
Epoch 3/1000
489/489 [==============================] - 0s 107us/step - loss: 0.6931 - acc: 0.5174 - val_loss: 0.6924 - val_acc: 0.6341
Epoch 4/1000
489/489 [==============================] - 0s 91us/step - loss: 0.6930 - acc: 0.5174 - val_loss: 0.6916 - val_acc: 0.6341
Epoch 5/1000
489/489 [==============================] - 0s 101us/step - loss: 0.6930 - acc: 0.5174 - val_loss: 0.6914 - val_acc: 0.6341
Epoch 6/1000
489/489 [==============================] - 0s 113us/step - loss: 0.6930 - acc: 0.5174 - val_loss: 0.6914 - val_acc: 0.6341
Epoch 7/1000
489/489 [==============================] - 0s 147us/step - loss: 0.6929 - acc: 0.5174 - val_loss: 0.6908 - val_acc: 0.6341
Epoch 8/1000
489/489 [==============================] - 0s 166us/step - loss: 0.6929 - acc: 0.5174 - val_loss: 0.6905 - val_acc: 0.6341
Epoch 9/1000
489/489 [==============================] - 0s 162us/step - loss: 0.6929 - acc: 0.5174 - val_loss: 0.6904 - val_acc: 0.6341
Epoch 10/1000
489/489 [==============================] - 0s 129us/step - loss: 0.6928 - acc: 0.5174 - val_loss: 0.6901 - val_acc: 0.6341
……
Epoch 1000/1000
489/489 [==============================] - 0s 86us/step - loss: 0.6926 - acc: 0.5174 - val_loss: 0.6849 - val_acc: 0.6341

model.evaluate(x_train, y_train)

489/489 [==============================] - 0s 43us/step
[0.6925447341854587, 0.5173823833465576]

model.evaluate(x_test, y_test)

164/164 [==============================] - 0s 39us/step
[0.684889389247429, 0.6341463327407837]

6.3 选取适当的神经网络

# 选取适当的神经网络
model = keras.Sequential()
model.add(layers.Dense(4, input_dim=15, activation='relu'))
model.add(layers.Dense(4,  activation='relu'))
model.add(layers.Dense(1, activation='sigmoid'))

model.compile(optimizer='adam',
              loss='binary_crossentropy',
              metrics=['acc']
)

history = model.fit(x_train, y_train, epochs=1000, validation_data=(x_test, y_test))

Train on 489 samples, validate on 164 samples
Epoch 1/1000
489/489 [==============================] - 0s 575us/step - loss: 40.1825 - acc: 0.5317 - val_loss: 17.6376 - val_acc: 0.6098
Epoch 2/1000
489/489 [==============================] - 0s 104us/step - loss: 30.0785 - acc: 0.5337 - val_loss: 12.6986 - val_acc: 0.6159
Epoch 3/1000
489/489 [==============================] - 0s 148us/step - loss: 20.0469 - acc: 0.5112 - val_loss: 8.3732 - val_acc: 0.5671
Epoch 4/1000
489/489 [==============================] - 0s 151us/step - loss: 12.5171 - acc: 0.4908 - val_loss: 3.8925 - val_acc: 0.5061
Epoch 5/1000
489/489 [==============================] - 0s 113us/step - loss: 4.4324 - acc: 0.4294 - val_loss: 0.9156 - val_acc: 0.4573
Epoch 6/1000
489/489 [==============================] - 0s 79us/step - loss: 1.0313 - acc: 0.5419 - val_loss: 0.9974 - val_acc: 0.4695
Epoch 7/1000
489/489 [==============================] - 0s 88us/step - loss: 1.0071 - acc: 0.5562 - val_loss: 0.8852 - val_acc: 0.5183
Epoch 8/1000
489/489 [==============================] - 0s 88us/step - loss: 0.9085 - acc: 0.5808 - val_loss: 0.7934 - val_acc: 0.5366
Epoch 9/1000
489/489 [==============================] - 0s 107us/step - loss: 0.8235 - acc: 0.5992 - val_loss: 0.7390 - val_acc: 0.5366
Epoch 10/1000
489/489 [==============================] - 0s 114us/step - loss: 0.7711 - acc: 0.5971 - val_loss: 0.7174 - val_acc: 0.5366
……
Epoch 1000/1000
489/489 [==============================] - 0s 141us/step - loss: 0.3095 - acc: 0.8732 - val_loss: 0.3971 - val_acc: 0.8537

model.evaluate(x_train, y_train)

489/489 [==============================] - 0s 68us/step
[0.30120014958464536, 0.8813905715942383]

model.evaluate(x_test, y_test)

164/164 [==============================] - 0s 45us/step
[0.39714593858253666, 0.8536585569381714]

附：系列文章

序号	文章目录	直达链接
1	波士顿房价预测	https://want595.blog.csdn.net/article/details/132181950
2	鸢尾花数据集分析	https://want595.blog.csdn.net/article/details/132182057
3	特征处理	https://want595.blog.csdn.net/article/details/132182165
4	交叉验证	https://want595.blog.csdn.net/article/details/132182238
5	构造神经网络示例	https://want595.blog.csdn.net/article/details/132182341
6	使用TensorFlow完成线性回归	https://want595.blog.csdn.net/article/details/132182417
7	使用TensorFlow完成逻辑回归	https://want595.blog.csdn.net/article/details/132182496
8	TensorBoard案例	https://want595.blog.csdn.net/article/details/132182584
9	使用Keras完成线性回归	https://want595.blog.csdn.net/article/details/132182723
10	使用Keras完成逻辑回归	https://want595.blog.csdn.net/article/details/132182795
11	使用Keras预训练模型完成猫狗识别	https://want595.blog.csdn.net/article/details/132243928
12	使用PyTorch训练模型	https://want595.blog.csdn.net/article/details/132243989
13	使用Dropout抑制过拟合	https://want595.blog.csdn.net/article/details/132244111
14	使用CNN完成MNIST手写体识别(TensorFlow)	https://want595.blog.csdn.net/article/details/132244499
15	使用CNN完成MNIST手写体识别(Keras)	https://want595.blog.csdn.net/article/details/132244552
16	使用CNN完成MNIST手写体识别(PyTorch)	https://want595.blog.csdn.net/article/details/132244641
17	使用GAN生成手写数字样本	https://want595.blog.csdn.net/article/details/132244764
18	自然语言处理	https://want595.blog.csdn.net/article/details/132276591

【ディープラーニング】実験13 ドロップアウトを利用した過学習の抑制