Using fit_generator in Keras

eng2019 :

I am new in using fit_generator in Keras, so I tried to write a simple script just to help me understand how it works.

X = np.array([[1,2],[10,3],[2,4],[20,5],[30,1],[3,5],[4,6],[7,4],[5,10],[1,7]])
Y = np.array([[2,3],[30,13],[8,6],[100,25],[30,31],[15,8],[24,10],[28,11],[50,15],[7,8]])

def generator(feat,labels):
    i=0
    while (True):
        yield feat[i],labels[i]
        i+=1

model_fnn = tf.keras.models.Sequential()
model_fnn.add(tf.keras.layers.Dense(50, input_dim=X.shape[1], activation=tf.nn.relu))
model_fnn.add(tf.keras.layers.Dense(Y.shape[1], activation=tf.keras.activations.linear))

nb_epoch = 3000

model_fnn.compile(optimizer='adam', loss='mean_squared_error', metrics=['accuracy'])
model_fnn.fit_generator(generator(X,Y), steps_per_epoch=10, epochs=nb_epoch, verbose=0)

But it gave me error:

ValueError: Error when checking input: expected dense_2_input to have shape (2,) but got array with shape (1,)

Could anyone please help? Thanks!

rusito23 :

This error seems to be caused by the fact that you need to wrap up every input/label into a Numpy array, which will be the training batch. The code for this generator will be:

def generator(feat,labels):
    i=0
    while (True):
        yield np.array([feat[i]]), np.array([labels[i]])
        i+=1

And your error should be solved. You will be training with a batch size 1, as every array passed to the training will contain only 1 object.

But, in order to train using a generator with yield, you need to make sure the cycle will not crash, as it is an infinite loop and you don't have infinite data. This can be achieved using itertools.cycle:

import itertools

def generator(feat,labels):
    pairs = [(x, y) for x in feat for y in labels]
    cycle_pairs = itertools.cycle(pairs)
    while (True):
        f, p = next(cycle_pairs)
        return np.array([f]), np.array([p])

And, to give one step further in the use of generators using yield, you can give the function a parameter to specify the batch size (as stated in the comment by PyGirl). This should look a little bit like this:

def generator(feat, labels, batch_size):
    pairs = [(x, y) for x in feat for y in labels]
    cycle_pairs = itertools.cycle(pairs)
    while (True):
        x = []
        y = []
        for _ in range(batch_size):
            f, p = next(cycle_pairs)
            x.append(f)
            y.append(p)
        yield np.array(x), np.array(y)

This means, in every step, the generator will yield a Numpy array with batch_size elements inside to be trained.

In order to understand more about how to train with fit_generator in Keras I would also recommend to read a bit about the Keras Sequence Util which is safer to use than a generator.

Let me know if this helps!