## 1. L2 Normalization Forward Pass(向前传导)

### 1.2 Implementation(实现)

 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18  template void NormalizationLayer::Forward_gpu(const vector*>& bottom, const vector*>& top) { const Dtype* bottom_data = bottom[0]->gpu_data(); Dtype* top_data = top[0]->mutable_gpu_data(); Dtype* squared_data = squared_.mutable_gpu_data(); Dtype normsqr; int n = bottom[0]->num(); int d = bottom[0]->count() / n; caffe_gpu_powx(n*d, bottom_data, Dtype(2), squared_data); for (int i=0; i(d, squared_data+i*d, &normsqr); caffe_gpu_scale(d, pow(normsqr, -0.5), bottom_data+i*d, top_data+i*d); } } 

## 2. L2 Normalization Backward Propagation

### 2.1 Formula Deduction(公式推导)

First is the gradient regardless of upper layer:

，所以下面的代码 计算

caffe_gpu_dot(d, top_data+i*d, top_diff+i*d, &a);

### 2.2 Implementation(实现)

 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21  `template void NormalizationLayer::Backward_gpu(const vector*>& top, const vector& propagate_down, const vector*>& bottom) { const Dtype* top_diff = top[0]->gpu_diff(); const Dtype* top_data = top[0]->gpu_data(); const Dtype* bottom_data = bottom[0]->gpu_data(); Dtype* bottom_diff = bottom[0]->mutable_gpu_diff(); int n = top[0]->num(); int d = top[0]->count() / n; Dtype a; for (int i=0; i

## 3. Full Codes

I just give the necessary GPU version codes.
For full Implementation of CPU, GPU and other necessary codes, please go to my Github repository for more imformation. link