Summary:
Meta-learning algorithm has nothing to do with the model, the model is compatible with any gradient descent training for classification, regression, and strengthening and so on.
Target meta-learning is a model with a large number of job training, so that it can solve new tasks by a small amount of training samples.
Model parameters are explicitly trained, the small sample a small step size gradient can produce good generalization performance on the new task. The actual training is too easy to fine-tune the model.
Highlights: 1, a small sample set of new tasks; 2, accelerated learning gradient enhanced fine-tuning
1 Introduction
Quick learning is a human intelligence: recognize objects from a few examples, the acquisition of skills or experience a few minutes. For this purpose AI goal: to quickly learn and adapt from a few examples, and further adjusted with the increase of data samples. Fast and flexible learning is challenging, the agent need to integrate knowledge and a few bright new sample information, while avoiding new data over-fitting. In the form of a priori and new data depending on the mission, meta-learning mechanism should be general enough to calculate the form of the task and complete the task in order to obtain maximum practicality.
Meta-learning algorithm to independent model can be applied to any learning problems gradient descent training. The focus here is the depth of the neural network, but can be modified for different architectures and to solve different problems. Yuan learning, training target model is quick to learn new tasks from a small amount of new data, the meta-learner training model can learn a lot of different tasks. The core idea of the article is a training model initialization parameters of the model through one or several gradient will be able to achieve the best performance. Before learning algorithms differ, Schmidhuber, 1987; Bengio et Al, 1992;. Andrychowicz et Al, 2016;. Ravi & Larochelle, 2017 , a study of these algorithms update function or learning rules; Santoro et al, 2016 requirements. recurrent, Koch, 2015 requires Siamese network. The proposed algorithm is characterized by: 1, does not increase the learning parameters 2, 3 does not limit the model structure, using a variety of loss function (or strengthening supervision).
From the perspective of learning characteristics, learning to establish a task for a variety of internal representation, only need to fine-tune the parameters (for example, adjusting the top right before the main feed model of weight) will be able to achieve good results, in fact want to initialize the parameters of the model easy to adjust; from the perspective of the dynamic system, the learning process can be seen as to maximize the sensitivity loss function parameter, the higher the sensitivity, a small change in parameter can greatly improve the local task loss.
2, algorithm
Problem Definition 2.1 yuan learning
references: