Ablation Study:
An ablation study is where you systematically removeparts of the input to see which parts of the inputare relevant to the networks output.
It is derivedfrom similar experiments in psychology which are usedto determine which parts of an image are important forhuman image recognition.
Examples:
- An LSTM has 4 gates: feature, input, output, forget. We might ask: are all 4 necessary? What if I remove one? Indeed, lots of experimentation has gone into LSTM variants, the GRU being a notable example (which is simpler).
- If certain tricks are used to get an algorithm to work, it’s useful to know whether the algorithm is robust to removing these tricks. For example, DeepMind’s original DQN paper reports using (1)only periodically updating the reference network and (2) using a replay buffer rather than updating online. It’s very useful for the research community to know that both these tricks are necessary, in order to build on top of these results.
- If an algorithm is a modification of a previous work, and has multiple differences, researchers want to know what the key difference is.
Simpler is better (inductive prior towards simpler model classes). If you can get the same performance with two models, prefer the simpler one.
解释：为了研究模型中所提出的一些结构是否有效而设计的实验。比如新加了某结构或者某个模块某个新的功能等，但是要想判断这个结构是否有利于最终的效果，是否效果更好，则将去掉该结构的网络与加上该结构的网络所得到的结果进行对比
即一种模型简化测试方法，通过取消模块的一部分后观察模型的性能是否改变，根据奥卡姆剃刀法则，简单和复杂的方法能够达到一样的效果，那么简单的方法更好更可靠。