13. PyTorch Tutorial---Recurrent Neural Network (RNN)

Recurrent neural networks (RNN) is a deep learning-oriented algorithm type that adopts a sequential method. In neural networks, we usually assume that each input and output is independent of all other layers. This type of neural network is called a recurrent neural network because they perform mathematical calculations in a sequential manner, completing one task before moving on to the next.

The diagram below illustrates the complete approach and working principle of Recurrent Neural Network −

In the above figure, c1, c2, c3 and x1 are considered as inputs, which include some hidden input values, namely h1, h2 and h3, which produce the corresponding output o1. Now, we will focus on implementing a recurrent neural network using PyTorch to create sine waves.

During training, we will use a training method that processes one data point at a time. The input sequence x contains 20 data points and the target sequence is considered to be the same as the input sequence.

Step 1
Use the following code to import the necessary packages required to implement Recurrent Neural Network -

import torch
from torch.autograd import Variable
import numpy as np
import pylab as pl
import torch.nn.init as init

Step 2
We will set the hyperparameters of the model and set the size of the input layer to 7. There will be 6 context neurons and 1 input neuron used to create the target sequence.

dtype = torch.FloatTensor
input_size, hidden_size, output_size = 7, 6, 1
epochs = 300
seq_le