Paper Digest - Neural Architecture Search: A Survey

最新推荐文章于 2025-03-15 18:00:00 发布

原创

最新推荐文章于 2025-03-15 18:00:00 发布 · 1.2k 阅读

标签

收录于

这篇论文探讨了神经架构搜索（NAS）的三个关键方面：搜索空间、搜索策略和性能评估策略。搜索空间包括层类型、操作和相关超参数。搜索策略涉及随机搜索、贝叶斯优化、进化方法、强化学习和梯度基方法。性能评估策略则关注如何减少评估时间，如使用低精度估计、学习曲线外推、权重继承和一拍模型。

NAS is a rather broad topic. The author of the paper categorize methods for NAS according to three dimensions.

search space - what structure to search from
search strategy - how to explore search space
performance estimation strategy - method on reduce the cost of performance estimation.

A simple search space is chain-structured neural network. The search space is then parametrized by:

(i) the (maximum) number of layers n (possibly unbounded);
(ii) the type of operation every layer executes, e.g., pooling, convolution, or more advanced operations like depthwise separable convolutions (Chollet, 2016) or dilated convolutions (Yu and Koltun, 2016);
(iii) hy