[Learning] Reptile, gradient descent LSTM, Siamese Network, prototype network, matching network, relational network

---

Reptile

RNN

LSTM

Variation of RNN


Adding Gates

LSTM with Gradient Descent

GD looks like a simplified LSTM

that allows the machine to automatically learn these zf and zi.

The c and x of the typical LSTM have nothing to do with each other, but the θ of GD's LSTM is related to the gradient of the input.


Advantages: reasonable model size; in typical gradient descent, all parameters use the same update rule; training and testing model architectures can be different.

Parameter updates depend not only on the current gradient but also on previous gradients.


Join another LSTM, m stores the previous data

metrics-based approach

Learn a network that can simultaneously input training data and test data and output results.

Siamese Network

Siamese network-intuitive explanation: Binary classification problem: "Are they the same?"


The data obtained after the picture passes through cnn, if it is the same face, it will be very similar, and the difference will be far away.
auto-encoder will keep all the data of the picture.
What distance should we use?

N-way Few/One-shot Learning

Suppose the task is a classification question rather than a yes-no question.

prototype network

matching network

The s layer gets the output after going through the multi-hop similar memory network

relationship network

Use another network to calculate the similarity.

Few-shot learning on virtual data

Using the generator, a picture is generated through G to generate multiple pictures, and then put into the network.

Train + Test as RNN

Input the one-hot vector of embedding and category, it cannot be trained