Select Via Proxy: Efficient Data Selection For Training DeepNetworks
2019ICLR的文章，介绍了Select Via Proxy（SVP）通过较小规模的模型来确定样本的uncertainty进而决定复杂模型训练使用样本的策略。
原文地址：Select Via Proxy: Efficient Data Selection For Training DeepNetworks
ABSTRACT
we develop a novel approach to efficiently select a subset of training data to achieve faster training with no loss in model predictive performance.
这是论文的主要贡献。
In our approach, we first train a small proxy model quickly, which we then use to estimate the utility of individual training data points, and then select the most informative ones for training the large target model.
先训练一些小的proxy模型去估计训练样本的实用性，然后用信息最丰富（uncertainty最高）的样本去训练目标模型。
1 INTRODUCTION
介绍了一下SVP的目的和uniform subsampling和core-set selection两种方法的缺点。简单介绍了SVP的流程（图1）和结果。
2 RELATED WORK
介绍了关于sampling和training data selection的一些文献，有兴趣的可以看一下。
3 SELECT VIA PROXY
3.1 CREATING A PROXY MODEL
总共就三步：
1）Create a proxy model that is fast to train and to provide us with an approximate decision boundary, 2) use the proxy model to select a subset of uncertain data points around the decision boundary, and 3) train the large target model on the selected subset via proxy to refine the decision boundary and get the final accurate model.
这里作者提出来两个观点：
It has been observed that for deep models with many layers, reducing the dimension (narrowing) or number of hidden layers (shortening) leads to a considerably reduced training times with only a small drop in accuracy.
a significant amount of training is spent to obtain a relatively small reduction in error.
这是使用SVP的基本立足点。
3.2 SUBSET SELECTION VIA PROXY
Quantifying uncertainty
介绍了有以下几种方法：
然后根据Algorithm 1对目标模型进行更新：
4 RESULTS
使用不同的proxy和不同的数据集进行了实验，结果显示在达到同样误差下，使用了SVP的训练时间短训练样本少。图5展示不同的uncertainty metrics效果的差异。表2展示了采用了proxy model进行example selection导致的样本类别不均衡。随着训练的继续和子训练集规模的增加数据将变得均衡，但从每一个类别里通过uncertainty选取相同数量的examples往往会导致模型表现不佳。
5 CONCLUSION
提出了一种通过proxy model来选择子训练集来训练深度学习模型的一种方法买这种方法会使得在同样精度的情况下训练速度的加快。由于不改变目标模型的训练，因此此方法可以与任意的模型配合使用。