Each output node is connected to all of the input nodes, this network layer is called fully connected layers, multiplication and addition operation is essentially a matrix;
Neurons interconnected by a network formed is called the neural network, each layer is fully connected network is called fully connected network layer;
6.5 explains why pre-processing data to 0-1 before the right reasons.
There are number of cylinders number of miles per gallon impact of the car, displacement, horsepower, weight, acceleration, and low production year
Which has the following relationship
Figure 6.16 correspond to the book, but what is the fourth diagram can not find!
Prediction neural network as follows:
# encoding: utf-8 import tensorflow as tf from tensorflow import keras from tensorflow.keras import layers, Sequential import pandas as pd import matplotlib.pyplot as plt # input data dataset_path = keras.utils.get_file("auto-mpg.data", "http://archive.ics.uci.edu/ml/" "machine-learning-databases/auto-mpg/auto-mpg.data", cache_dir='G:\2019\python\auto-mpg.data') colum_names = ['MPG', 'Cylinder', 'Displacement', 'Horsepower', 'Weight', 'Acceleration', 'Model Year', 'Origin'] # 398行数据，8列 raw_dataset = pd.read_csv(dataset_path, names=colum_names, na_values="?", comment='\t', sep=" ", skipinitialspace=True) # data process dataset = raw_dataset.copy() dataset.head() dataset.isna().sum() dataset = dataset.dropna() dataset.isna().sum() # dataset.shape = (392, 8) # 拓展dataset到10列 origin = dataset.pop('Origin') # (392, 1)得到产地 dataset['USA'] = (origin == 1) * 1.0 dataset['Europe'] = (origin == 2) * 1.0 DataSet [ ' Japan ' ] = (Origin ==. 3) * 1.0 dataset.tail () # dataset.shape = (392, 10) into three after three origin, indicating whether each column with 0,1 # Get Training and test sets train_dataset = dataset.sample (FRAC = 0.8, random_state = 0) # (314, 10) test_dataset = dataset.drop (train_dataset.index) # (78, 10) train_labels = train_dataset.pop ( ' MPG ' ) # (314, 1) remove the tape tab "MPG" first column test_labels test_dataset.pop = ( ' MPG ' ) #(78, 1) # for statistical train_stats train_dataset.describe = () # (. 8,. 9) obtaining a mean value of the data, standard deviation, maximum value, minimum value train_stats train_stats.transpose = () # (. 9,. 8) turn set function # normalized function DEF NORM (X): return (X - train_stats [ ' Mean ' ]) / train_stats [ ' STD ' ] # training set, a test set of standardized normed_train_data = NORM (train_dataset) # (314,. 9) data - between 1: 1 normed_test_data = NORM (test_dataset) # (78,. 9) Print(normed_train_data.shape, train_labels.shape) # show entered the final model Print (normed_test_data.shape, test_labels.shape) # segmentation training set, testing set train_db = tf.data.Dataset.from_tensor_slices ((normed_train_data.values, train_labels. values)) train_db = train_db.shuffle (314) .batch (32 ) TEST_DB = tf.data.Dataset.from_tensor_slices ((normed_test_data.values, test_labels.values)) TEST_DB = test_db.shuffle (78) .batch (13 is ) # observe the relationship between the original data DEF the Draw (): MPG = dataset.pop ( ' MPG ' ) Cylinder= dataset.pop('Cylinder') Displacement = dataset.pop('Displacement') Weight = dataset.pop('Weight') Horsepower = dataset.pop('Horsepower') Acceleration = dataset.pop('Acceleration') ModelYear = dataset.pop('Model Year') plt.figure() plt.subplot(131) plt.plot(Cylinder, MPG, 'bo' ) Plt.xlabel ( ' Cylinders ' ) plt.ylabel ( ' MPG ' ) plt.subplot ( 132 ) plt.plot (Displacement, MPG, ' bo ' ) plt.xlabel ( ' Displacement ' ) plt.ylabel ( ' MPG ' ) plt.subplot ( 133 ) plt.plot (Weight, MPG, ' bo ' ) plt.xlabel ( ' Weight ' ) plt.ylabel ( 'MPG' ) Plt.savefig ( ' three_fig_auto-MPG.png ' ) plt.show () # twenty-two distribution between features Draw () # create a network class Network (keras.Model): DEF __init__ (Self): # recurrent network model Super (the Network, Self). the __init__ () self.fc1 = layers.Dense (64, Activation = ' RELU ' ) # output node 64 self.fc2 = layers.Dense (64, Activation = ' RELU ' ) self.fc3Layers.Dense = (. 1 ) DEF Call (Self, Inputs, Training = None, mask = None): # create three fully connected layers X = self.fc1 (Inputs) X = self.fc2 (X) X = Self. FC3 (X) return X Model = the network () # Create instance network model.build (= input_shape (4, 9)) # complete the internal tensor created, batch number 4 is arbitrarily set, the input characteristic length 9 model. the Summary () # print network information Optimizer = tf.keras.optimizers.RMSprop (0.001) #Create the optimizer, specify the learning rate # save error situation the training and testing process train_tot_loss = [] train_tot_mae = [] test_tot_loss = [] test_tot_mae = [] for Epoch in the Range (200 ): for the STEP, (the X-, the y-) in the enumerate (train_db): with tf.GradientTape () Tape AS: # gradient recorder OUT = Model (X) # obtain input through the network Loss = tf.reduce_mean (tf.losses.MSE (Y, OUT)) # calculated MSE mae_loss= tf.reduce_mean(tf.losses.MAE(y, out)) # 计算MAE if step % 10 == 0: print(epoch, step, float(loss)) train_tot_loss.append(float(loss)) train_tot_mae.append(float(mae_loss)) # 更新梯度 grads = tape.gradient(loss, model.trainable_variables) optimizer.apply_gradients(zip(grads, model.trainable_variables)) # 测试 if step % 10 == 0: test_loss, test_mae_loss = 0, 0 for x, y in test_db: test_out = model(x) test_loss = tf.reduce_mean(tf.losses.MSE(y, test_out)) test_mae_loss = tf.reduce_mean(tf.losses.MAE(y, test_out)) print(epoch, step, float(test_loss)) test_tot_loss.append(float(test_loss)) test_tot_mae.append(float(test_mae_loss)) plt.figure() plt.plot(train_tot_mae, 'b', label = 'train') plt.plot(test_tot_mae, 'r', label = 'test') plt.xlabel('Step') plt.ylabel('MAE') plt.legend() plt.savefig('train_test_auto-MPG.png') plt.show()
Training and testing error as follows:
Fluctuations is not consistent with the book is very smooth, the overall trend is the same!
Note: (1) Warning: tensorflow: Layer network are input tensor conversion dtype float32 of this layer from dtype float64, this is the tensorflow 2 new behavior.
Unresolved this warning!
The next update reverse propagation algorithm, Himmelbla function optimization and dichotomous fully connected network