Caffe source code learning record (1)

Reference from "21 days of actual combat caffe" and Zhihu: How to read the code of deep learning caffe?

caffe is a deep learning framework based on C++/CUDA/python developed by the Berkeley Vision and Learning Center (BVLC).

1. Features :

1. Implementing a Convolutional Neural Network Architecture (CNN)
2. Fast speed, using google's ProtoBuffer data format to improve efficiency
3. Completely open source, written in C++ language
4. caffe provides a complete set of tools for training, prediction, data preprocessing, etc.
5. caffe comes with a series of reference models and quick tutorials

2. Code structure :

The above picture shows the data structure of caffe, a CNN model is represented by Net, Net is composed of multiple layers, and the input and output data of Layer are saved by the Blob data structure.

The major categories of Blob, Layer, Net, and Solver, from bottom to top, are interlinked and run through the entire structure of caffe.

1. Blob : It is the basic data structure and storage unit, which is used to save the learned parameters and the data generated during the network transmission process.
   
2. Layer : It is the basic computing unit of the network, from which various layer classes are derived. The person who revised this part mainly studies the direction of feature expression.
   
3. Net : It is the construction of the network, representing a complete CNN, combining the layer classes derived from Layer into a network. Solver: It is the solution of Net. The main purpose of modifying this part is to study the direction of DL solution.

3. Specific code structure

• Blob
• Blob type description

The data type used internally by Caffe is mainly the inheritance of the data structure defined by the protocol buffer, so it can achieve high efficiency with the smallest possible memory footprint, although Caffe will sacrifice some code readability while pursuing performance .

Blob represents a 4-dimensional array in memory. The dimensions from low to high are (width_,height_,channels_,num_ ), which can respectively represent the width and height of the image, the color channel, and the frame of the image. Blob stores data or weights (data) and weight increments (diff).

Blob is a template class. It is declared in include/caffe/blob.hpp. When using it, you need to include the header file #include<caffe/blob.hpp> and use the namespace caffe. The implementation of the class is in src/caffe/proto/caffe.proto.

BlobProto objects can interact with data on disk and in memory.

• Blob important member functions and variables

shared_ptr<SyncedMemory> data_; //数据
shared_ptr<SyncedMemory> diff_;//梯度

void Blob<Dtype>::Reshape(const int num, const int channels, const int height,const int width); //Re-modify the Blob shape and apply for dynamic memory data and gradients according to the shape

inline int count(int start_axis, int end_axis) const; //Calculate the number of basic data units needed by Blob

• Layer
• Layer type description

Layer has at least one input Blob (Bottom Blob) and one output Blob (Top Blob), and some Layers have weights and biases (Bias), and there are two directions of operation: Forward and Reverse Backward. The forward propagation processes the input blob to obtain the output blob, and the back propagation calculation performs some processing on the diff of the output blob to obtain the diff of the input blob.

Layer is an abstract class and cannot directly create Layer objects. Most of its functions are only virtual functions, and the real implementation of functions is in its derived classes.

Header file include/caffe/layer.hpp Source file src/caffe/layer.cpp Blob important member functions and variables

• Layer important member functions and variables

vector<Dtype> loss_ //Each layer will have a loss value, but only LossLayer will generate non-0 loss
vector<shared_ptr<Blob<Dtype> > > blobs_ ;//Parameters learned by Layer, including weights and biases

virtual void Forward(const vector<Blob<Dtype>*> &bottom,vector<Blob<Dtype>*> *top) = 0;// Layer internal data is propagated forward, from bottom to top

virtual void Backward(const vector<Blob<Dtype>*> &top,const vector<bool> &propagate_down,vector<Blob<Dtype>*> *bottom) = 0;

//Layer internal gradient backpropagation, from top to bottom direction

Inside the Layer, there are two main ways to transfer data, Forward and Backward, which are implemented by CPU and GPU (partly).

Layer function:

layers {
  bottom: "decode1neuron" // The first Layer connected at the bottom of this layer
  bottom: "flatdata" // the second Layer connected to the bottom of this layer
  top: "l2_error" // A Layer connected on top of this layer
  name: "loss" // the name of the layer
  type: EUCLIDEAN_LOSS // The type of the layer
  loss_weight: 0
}

• Net
• Net type description

Net puts multiple layers together in an orderly manner in the form of a container. Its own functions are mainly to initialize layer-by-layer layers and provide an interface for Update( ) (update network parameters), which cannot effectively perform parameters on its own. learning process.

• Important member functions and variables of Net

vector<shared_ptr<Layer<Dtype> > > layers_ ;//The layers that constitute the net

void Init(const NetParameter& param);
//Initialize net according to NetParameter. Simply put, first instantiate the bottom Blobs & top Blobs (no repetition) of all layers in the network, and start from the input layer, and perform the Setup work layer by layer, thus completing the entire network. Build, lay the foundation for the subsequent data transmission before and after.

vector<Blob<Dtype>*>& Forward(const vector<Blob<Dtype>* > & bottom,Dtype* loss = NULL);
void Net<Dtype>::Backward();
//It is the forward and reverse conduction of the entire network, and the loss of the network can be calculated once each call.

• Solver
• Type description of Solver
  

The Solver class contains a pointer to Net, which mainly implements the optimization algorithm used to train the model parameters. Different classes will be derived according to the optimization algorithm, and based on these subclasses, the normal training process of the network can be performed.

• Important member functions and variables of Solver

shared_ptr<Net<Dtype> > net_ ;//net对象

virtual void ComputeUpdateValue() = 0;//Different model training methods implement the core function of calculating update parameters by overloading the function ComputeUpdateValue( )

Finally, when the entire network training process is performed (that is, when you run Caffe to train a model), you are actually running the train( ) function in caffe.cpp, and this function actually instantiates a Solver object. After initialization The Solve( ) method in Solver is called. And this Solve( ) function mainly runs the following two functions iteratively.

ComputeUpdateValue();
net_->Update();

caffemodel is a trained model and weight file.

solverstate is the solver state file, which stores the intermediate process of training, and can continue training after pausing