https://blog.csdn.net/jacke121/article/details/97963336
Original: https://zhuanlan.zhihu.com/p/42924585
AutoML neural architecture and search (NAS), is a new generation king of the depth of field of study.
These methods can quickly rough shoved to get a machine learning task, simple and effective, in line with the height of modern science and technology company's core values.
The rationale behind how they, how to use?
Technology blog TowardDataScience there is an article on a comprehensive introduction to everything about AutoML and NAS you need to know.
NAS
To learn AutoML, had to start from the NAS.
During the development of neural networks, the infrastructure project at stake, infrastructure deficiencies, then how training is also difficult to get good results.
Of course, mention architecture, many people will think of the migration study: the ImageNet ResNet brought on training, I need another set of data retraining training updates about weight, do not like it!
This approach does also possible, however, to get the best effect, or design your own network infrastructure more reliable according to the actual situation.
Neural network architecture design, machine learning process can be regarded as a task of the highest threshold. Want to design a good architecture requires specialized knowledge and skills, but also a lot of trial and error.
NAS just to get this time-consuming task for us.
The goal of this algorithm is to search out the best neural network architecture. Its workflow typically start by defining a set of neural networks might use "building blocks." For example, Google Brain essay NasNet paper, we summarize these commonly used image recognition module for the network:
And wherein the cell comprises a plurality of convolution module.
论文：Learning Transferable Architectures for Scalable Image Recognition
地址：https://arxiv.org/pdf/1707.07012.pdf
NAS algorithm is a loop neural network (RNN) as the controller , from one of those modules, and then put them together, end to end to create some kind of architecture.
This architecture, often showing and ResNet, DenseNet the forefront of network architecture such as the same style, but a combination of internal and configuration modules differ. A quality architecture, often depending on the selected module and the connection between them constructed.
Next, we must train the new network, convergence, verified the accuracy of the leave on the set. The accuracy rate is then used to update the controller via policy gradient, so that the level controller architecture to generate higher and higher.
Process as shown in FIG.
This process is very straightforward. In short, the children very building blocks of style: pick some of the building blocks to make an algorithm, and then assemble them together and make a neural network. Training, testing, adjusting the standard building blocks of selected and assembled according to the result of the way. This algorithm is a big success, NasNet papers show a very good result, part of the reason is due to the limitations and assumptions they make.
Thesis training, testing NAS algorithm found in architecture, we have used a much smaller data set than reality. Of course, this is a way to compromise, to train verify each search result on ImageNet so large data sets, it is too time-consuming.
So they made a hypothesis : if a neural network can get better results in a structure similar to a small set of data, it is also able to perform better on larger and more complex data sets.
In the depth of field of study, this assumption is essentially true.
As also mentioned above a limit , which means that the search space is very limited. They designed NAS, is necessary to build the most advanced architecture and style is very similar to the network to use it.
In the field of image recognition, which means repeatedly arranged with a group of modules, the sampling phase, as shown below:
These modules are also commonly used in the current study. NAS algorithm in which the new work, these modules mainly to another connection.
Here, it is found ImageNet best neural network architecture:
Progress in search of architecture
This paper led to a NASNet progress within the industry, it points out a new direction for the study and research depth.
但是，用450个GPU来训练，找到一个优秀的架构也需要训练3到4天。也就是说，对于除了Google之外的普通贫民用户们，这种方法还是门槛太高、效率太低。
NAS领域最新的研究，就都在想方设法让这个架构搜索的过程更高效。
2017年谷歌提出的渐进式神经架构搜索（PNAS），建议使用名叫“基于序列模型的优化（SMBO）”的策略，来取代NASNet里所用的强化学习。用SMBO策略时，我们不是随机抓起一个模块就试，而是按照复杂性递增的顺序来测试它们并搜索结构。
这并不会缩小搜索空间，但确实用更聪明的方法达到了类似的效果。SMBO基本上都是在讲：相比于一次尝试多件事情，不如从简单的做起，有需要时再去尝试复杂的办法。这种PANS方法比原始的NAS效率高5到8倍，也便宜了许多。
论文：Progressive Neural Architecture Search
地址：https://arxiv.org/pdf/1712.00559.pdf
高效神经架构搜索（ENAS），是谷歌打出的让传统架构搜索更高效的第二枪，这种方法很亲民，只要有GPU的普通从业者就能使用。作者假设NAS的计算瓶颈在于，需要把每个模型到收敛，但却只是为了衡量测试精确度，然后所有训练的权重都会丢弃掉。
论文：Efficient Neural Architecture Search via Parameter Sharing
地址：https://arxiv.org/pdf/1802.03268.pdf
因此，ENAS就要通过改进模型训练方式来提高效率。
在研究和实践中已经反复证明，迁移学习有助在短时间内实现高精确度。因为为相似任务训练的神经网络权重相似，迁移学习基本只是神经网络权重的转移。
ENAS算法强制将所有模型的权重共享，而非从零开始训练模型到收敛，我们在之前的模型中尝试过的模块都将使用这些学习过的权重。因此，每次训练新模型是都进行迁移学习，收敛速度也更快。
深度学习新方法AutoML
很多人将AutoML称为深度学习的新方式，认为它改变了整个系统。有了AutoML，我们就不再需要设计复杂的深度学习网络，只需运行一个预先设置好的NAS算法。
最近，Google提供的Cloud AutoML将这种理念发挥到了极致。只要你上传自己的数据，Google的NAS算法就会为你找到一个架构，用起来又快又简单。
AutoML的理念就是把深度学习里那些复杂的部分都拿出去，你只需要提供数据，随后就让AutoML在神经网络设计上尽情发挥吧。这样，深度学习就变得像插件一样方便，只要有数据，就能自动创建出由复杂神经网络驱动的决策功能。
△ 谷歌云的AutoML pipeline
不过，AutoML价格也并不算亲民，每小时收费20美元。此外，一旦你开始训练，则无法导出模型，并且得用谷歌提供的API在云上运行你的网络，这些限制看起来都不是很方便，
AutoKeras也是一个使用了ENAS算法的GitHub项目，可以使用pip安装。它是用Keras编写的，因此很容易控制和使用，甚至可以自己深入研究ENAS算法并尝试进行一些改动。
如果你喜欢用TensorFlow或者Pytorch，也有一些开源项目可用：
https://github.com/melodyguan/enas
https://github.com/carpedm20/ENAS-pytorch
总的来说，若你想使用AutoML，现在已经有很多不同的选择，这完全取决于你是否会使用你想要的算法，以及你对这件事的预算如何。
AutoML和NAS未来会怎样？
过去几年，在深度学习工作的自动化上，整个领域都在大步向前，让深度学习更贴近大众、更易用。
不过，进步的空间永远都有。
架构搜索已经越来越高效了，用ENAS，一个GPU一天就能找出一种新的网络架构。的确鹅妹子嘤，但是这个搜索空间依然非常有限，而且，现在NAS算法所用的基本结构和模块还是人工设计的，只是将组装的过程自动化了。
将来要想取得重大突破，在更广阔的搜索范围里搜索真正的新架构是一个重要方向。
如果这样的算法出现，甚至能为我们揭示庞大复杂深度神经网络中隐藏的秘密。
当然，要实现这样的搜索空间，需要设计出更高效的算法。
最后，附上原文传送门：
https://towardsdatascience.com/the-end-of-open-ai-competitions-ff33c9c69846
这篇文章全面介绍了用来自动寻找神经网络架构的AutoML，量子位之前还介绍过很多深度学习其他工作的自动化框架，比如自动特征工程、自动调参等等。
<li class="tool-item tool-active is-like "><a href="javascript:;"><svg class="icon" aria-hidden="true">
<use xlink:href="#csdnc-thumbsup"></use>
</svg><span class="name">点赞</span>
<span class="count">2</span>
</a></li>
<li class="tool-item tool-active is-collection "><a href="javascript:;" data-report-click="{"mod":"popu_824"}"><svg class="icon" aria-hidden="true">
<use xlink:href="#icon-csdnc-Collection-G"></use>
</svg><span class="name">收藏</span></a></li>
<li class="tool-item tool-active is-share"><a href="javascript:;"><svg class="icon" aria-hidden="true">
<use xlink:href="#icon-csdnc-fenxiang"></use>
</svg>分享</a></li>
<!--打赏开始-->
<!--打赏结束-->
<li class="tool-item tool-more">
<a>
<svg t="1575545411852" class="icon" viewBox="0 0 1024 1024" version="1.1" xmlns="http://www.w3.org/2000/svg" p-id="5717" xmlns:xlink="http://www.w3.org/1999/xlink" width="200" height="200"><defs><style type="text/css"></style></defs><path d="M179.176 499.222m-113.245 0a113.245 113.245 0 1 0 226.49 0 113.245 113.245 0 1 0-226.49 0Z" p-id="5718"></path><path d="M509.684 499.222m-113.245 0a113.245 113.245 0 1 0 226.49 0 113.245 113.245 0 1 0-226.49 0Z" p-id="5719"></path><path d="M846.175 499.222m-113.245 0a113.245 113.245 0 1 0 226.49 0 113.245 113.245 0 1 0-226.49 0Z" p-id="5720"></path></svg>
</a>
<ul class="more-box">
<li class="item"><a class="article-report">文章举报</a></li>
</ul>
</li>
</ul>
</div>
</div>
<div class="person-messagebox">
<div class="left-message"><a href="https://blog.csdn.net/jacke121">
<img src="https://profile.csdnimg.cn/3/7/5/3_jacke121" class="avatar_pic" username="jacke121">
<img src="https://g.csdnimg.cn/static/user-reg-year/1x/10.png" class="user-years">
</a></div>
<div class="middle-message">
<div class="title"><span class="tit"><a href="https://blog.csdn.net/jacke121" data-report-click="{"mod":"popu_379"}" target="_blank">ShellCollector</a></span>
</div>
<div class="text"><span>发布了2637 篇原创文章</span> · <span>获赞 954</span> · <span>访问量 520万+</span></div>
</div>
<div class="right-message">
<a href="https://bbs.csdn.net/topics/395531331" target="_blank" class="btn btn-sm btn-red-hollow bt-button personal-messageboard">他的留言板
</a>
<a class="btn btn-sm bt-button personal-watch" data-report-click="{"mod":"popu_379"}">关注</a>
</div>
</div>
</div>
