Keras is a powerful easy-to-use Python library for developing and evaluating deep learningmodels.
It wraps the efficient numerical computation libraries Theano and TensorFlow and allows you to define and train neural network models in a few short lines of code.
In this post, you will discover how to create your first neural network model in Python using Keras.
Let’s get started.
- Update Feb/2017: Updated prediction example so rounding works in Python 2 and Python 3.
- Update Mar/2017: Updated example for Keras 2.0.2, TensorFlow 1.0.1 and Theano 0.9.0.
Develop Your First Neural Network in Python With Keras Step-By-Step
Photo by Phil Whitehouse, some rights reserved.
Tutorial Overview
There is not a lot of code required, but we are going to step over it slowly so that you will know how to create your own models in the future.
The steps you are going to cover in this tutorial are as follows:
- Load Data.
- Define Model.
- Compile Model.
- Fit Model.
- Evaluate Model.
- Tie It All Together.
This tutorial has a few requirements:
- You have Python 2 or 3 installed and configured.
- You have SciPy (including NumPy) installed and configured.
- You have Keras and a backend (Theano or TensorFlow) installed and configured.
If you need help with your environment, see the tutorial:
Create a new file called keras_first_network.py and type or copy-and-paste the code into the file as you go.
Need help with Deep Learning in Python?
Take my free 2-week email course and discover MLPs, CNNs and LSTMs (with sample code).
Click to sign-up now and also get a free PDF Ebook version of the course.
1. Load Data
Whenever we work with machine learning algorithms that use a stochastic process (e.g. random numbers), it is a good idea to set the random number seed.
This is so that you can run the same code again and again and get the same result. This is useful if you need to demonstrate a result, compare algorithms using the same source of randomness or to debug a part of your code.
You can initialize the random number generator with any seed you like, for example:
Now we can load our data.
In this tutorial, we are going to use the Pima Indians onset of diabetes dataset. This is a standard machine learning dataset from the UCI Machine Learning repository. It describes patient medical record data for Pima Indians and whether they had an onset of diabetes within five years.
As such, it is a binary classification problem (onset of diabetes as 1 or not as 0). All of the input variables that describe each patient are numerical. This makes it easy to use directly with neural networks that expect numerical input and output values, and ideal for our first neural network in Keras.
Download the Pima Indian dataset from the UCI Machine Learning repository and place it in your local working directory, the same as your python file. Save it with the file name:
You can now load the file directly using the NumPy function loadtxt(). There are eight input variables and one output variable (the last column). Once loaded we can split the dataset into input variables (X) and the output class variable (Y).
We have initialized our random number generator to ensure our results are reproducible and loaded our data. We are now ready to define our neural network model.
2. Define Model
Models in Keras are defined as a sequence of layers.
We create a Sequential model and add layers one at a time until we are happy with our network topology.
The first thing to get right is to ensure the input layer has the right number of inputs. This can be specified when creating the first layer with the input_dim argument and setting it to 8 for the 8 input variables.
How do we know the number of layers and their types?
This is a very hard question. There are heuristics that we can use and often the best network structure is found through a process of trial and error experimentation. Generally, you need a network large enough to capture the structure of the problem if that helps at all.
In this example, we will use a fully-connected network structure with three layers.
Fully connected layers are defined using the Dense class. We can specify the number of neurons in the layer as the first argument, the initialization method as the second argument as init and specify the activation function using the activation argument.
In this case, we initialize the network weights to a small random number generated from a uniform distribution (‘uniform‘), in this case between 0 and 0.05 because that is the default uniform weight initialization in Keras. Another traditional alternative would be ‘normal’ for small random numbers generated from a Gaussian distribution.
We will use the rectifier (‘relu‘) activation function on the first two layers and the sigmoid function in the output layer. It used to be the case that sigmoid and tanh activation functions were preferred for all layers. These days, better performance is achieved using the rectifier activation function. We use a sigmoid on the output layer to ensure our network output is between 0 and 1 and easy to map to either a probability of class 1 or snap to a hard classification of either class with a default threshold of 0.5.
We can piece it all together by adding each layer. The first layer has 12 neurons and expects 8 input variables. The second hidden layer has 8 neurons and finally, the output layer has 1 neuron to predict the class (onset of diabetes or not).
3. Compile Model
Now that the model is defined, we can compile it.
Compiling the model uses the efficient numerical libraries under the covers (the so-called backend) such as Theano or TensorFlow. The backend automatically chooses the best way to represent the network for training and making predictions to run on your hardware, such as CPU or GPU or even distributed.
When compiling, we must specify some additional properties required when training the network. Remember training a network means finding the best set of weights to make predictions for this problem.
We must specify the loss function to use to evaluate a set of weights, the optimizer used to search through different weights for the network and any optional metrics we would like to collect and report during training.
In this case, we will use logarithmic loss, which for a binary classification problem is defined in Keras as “binary_crossentropy“. We will also use the efficient gradient descent algorithm “adam” for no other reason that it is an efficient default. Learn more about the Adam optimization algorithm in the paper “Adam: A Method for Stochastic Optimization“.
Finally, because it is a classification problem, we will collect and report the classification accuracy as the metric.
4. Fit Model
We have defined our model and compiled it ready for efficient computation.
Now it is time to execute the model on some data.
We can train or fit our model on our loaded data by calling the fit() function on the model.
The training process will run for a fixed number of iterations through the dataset called epochs, that we must specify using the nepochs argument. We can also set the number of instances that are evaluated before a weight update in the network is performed, called the batch size and set using the batch_size argument.
For this problem, we will run for a small number of iterations (150) and use a relatively small batch size of 10. Again, these can be chosen experimentally by trial and error.
This is where the work happens on your CPU or GPU.
5. Evaluate Model
We have trained our neural network on the entire dataset and we can evaluate the performance of the network on the same dataset.
This will only give us an idea of how well we have modeled the dataset (e.g. train accuracy), but no idea of how well the algorithm might perform on new data. We have done this for simplicity, but ideally, you could separate your data into train and test datasets for training and evaluation of your model.
You can evaluate your model on your training dataset using the evaluate() function on your model and pass it the same input and output used to train the model.
This will generate a prediction for each input and output pair and collect scores, including the average loss and any metrics you have configured, such as accuracy.
6. Tie It All Together
You have just seen how you can easily create your first neural network model in Keras.
Let’s tie it all together into a complete code example.
Running this example, you should see a message for each of the 150 epochs printing the loss and accuracy for each, followed by the final evaluation of the trained model on the training dataset.
It takes about 10 seconds to execute on my workstation running on the CPU with a Theano backend.
Note: If you try running this example in an IPython or Jupyter notebook you may get an error. The reason is the output progress bars during training. You can easily turn these off by setting verbose=0 in the call to model.fit().
7. Bonus: Make Predictions
The number one question I get asked is:
After I train my model, how can I use it to make predictions on new data?
Great question.
We can adapt the above example and use it to generate predictions on the training dataset, pretending it is a new dataset we have not seen before.
Making predictions is as easy as calling model.predict(). We are using a sigmoid activation function on the output layer, so the predictions will be in the range between 0 and 1. We can easily convert them into a crisp binary prediction for this classification task by rounding them.
The complete example that makes predictions for each record in the training data is listed below.
Running this modified example now prints the predictions for each input pattern. We could use these predictions directly in our application if needed.
Summary
In this post, you discovered how to create your first neural network model using the powerful Keras Python library for deep learning.
Specifically, you learned the five key steps in using Keras to create a neural network or deep learning model, step-by-step including:
- How to load data.
- How to define neural network in Keras.
- How to compile a Keras model using the efficient numerical backend.
- How to train a model on data.
- How to evaluate a model on data.
Do you have any questions about Keras or about this tutorial?
Ask your question in the comments and I will do my best to answer.
Related Tutorials
Are you looking for some more Deep Learning tutorials with Python and Keras?
Take a look at some of these:
- 5 Step Life-Cycle for Neural Network Models in Keras
- How to Grid Search Hyperparameters for Deep Learning Models in Python With Keras
- Time Series Prediction With Deep Learning in Keras
- Multi-Class Classification Tutorial with the Keras Deep Learning Library
- Regression Tutorial with the Keras Deep Learning Library in Python