RNN entry

RNN entry

A, RNN interpretation

Recurrent Neural Networks, by definition, represents a neural network containing neurons cycle. RNN reason was invented, mainly because of timing problems can be overcome traditional CNN can not overcome. However, in certain tasks, the timing contains important information. As shown below, if the sentence is determined to deal with CNN tone task, the timing information is lost:
the whole connection structure of CNN, and the inability to understand the changes in order to bring the meaning of the sentence as, the neural network can be circulated by timing and understand the different meaning.

Second, related technologies

1. one shot code

The different forms of encoded information input to the neural network, to achieve this function do not leak. As is the task o hell prediction, coding can cover only four letters to: $h = [1, 0, 0, 0]^T$ , $e = [0, 1, 0, 0]^T$, $h = [0, 0, 1, 0]^T$, $ o = [0, 0, 0, 1]^T$.

2. The time step

In the RNN, a time step is represented by a process inputs.

3. Front Propagation

Propagating forward from the input to the neural structure, to the weight, until the output of the last acquisition process error.

4. After propagation

是从误差，反算回来，获得每个神经元偏导数的过程．
前向传播的一个简单例子如下：

输入为x, 输出为ｙ，假设此时有一个训练样本(x,y) = (1, 10)，初始化权重为w = 0;
那么前向传播计算残差f的过程如下：
$f = (y - xw)^2$
通过前向传播，马上就能计算出第一次训练的误差为 $f= (10 - 1)^2 = 81$
根据反向传播：
$k = \partial f/\partial w = 2(y-xw)(-x)$
根据梯度下降法：
$-(y-xw)^2 = k\Delta w$
因为此处为一维的例子，因此可以得到 $\Delta w$的更新公式为：
$(y-xw)/(2x) = \Delta w$
第一次训练得到的 $\Delta w = 5$, 因此 $w^1 = 5$
第二次训练得到的 $\Delta w = 2.5$, 因此 $w^2 = 7.5$
第三次训练得到的 $\Delta w = 1.25$, 因此 $w^3 = 8.75$
第四次训练得到的 $\Delta w = 0.625$, 因此 $w^4 = 9.375$
第五次训练得到的 $\Delta w = 0.3125$, 因此 $w^5 = 9.6875$
第六次训练得到的 $\Delta w = 0.15625$, 因此 $w^6 = 9.84375$
如上，继续训练将得到一个非常接近理想参数的值．

二，RNN实践，二维速度估计

1. 环境构造

以一个正方形（或者曲线形态）的点作为feature，构造一个环境，如下图所示：

上图中，中心线为车辆行驶的路径，而两边的点为landmark。

2. 观测构造

以激光点云的形式，在每一个点位可以观察到环境中的每一个点，在上图中，一共有100个姿态，其中70个作为训练集，30个作为验证集

3. 特征输出

4.