Neural Architecture Search with Synchronous Advantage Actor-Critic Methods and Partial Training

ABSTRACT

In recent years, deep neural networks have enabled researchers to solve many difficult learning tasks, including natural language processing, image recognition and translation. Although powerful and flexible tool for many automation tasks, their design requires intensive human effort. Furthermore, recent studies suggest that the architecture itself can contribute more to the network's performance than its training. It is thus easy to see the necessity of automating the task of architecture design, as it will lead to even further improvements in the field. In this paper we implement a synchronous Advantage Actor-Critic Reinforcement Learning method in order to generate fully convolutional architectures, inspired by recent research in Neural Architecture Search and Reinforcement Learning. Furthermore, we explore the possibility of partially training the evaluated architectures, in order to assess the network's quality, greatly reducing the time required to evaluate them. Using Kendall's tau we show that a set of architectures, ordered by their performance retains its relative ranking when evaluated with partial training. Furthermore, the method outperforms random search, when partial training is used, as it finishes faster, produces better results and the overall probability of producing a high-quality architecture is higher.

References

1. Moez Baccouche, Franck Mamalet, Christian Wolf, Christophe Garcia, and Atilla Baskurt. 2011. Sequential deep learning for human action recognition. In International Workshop on Human Behavior Understanding. Springer, 29--39.
2. Bowen Baker, Otkrist Gupta, Nikhil Naik, and Ramesh Raskar. 2016. Designing neural network architectures using reinforcement learning. arXiv preprint arXiv:1611.02167 (2016).
3. James Bergstra and Yoshua Bengio. 2012. Random search for hyper-parameter optimization. Journal of Machine Learning Research 13, Feb (2012), 281--305.
4. Zezhou Cheng, Qingxiong Yang, and Bin Sheng. 2015. Deep colorization. In Proceedings of the IEEE International Conference on Computer Vision. 415--423.
5. Junyoung Chung, Caglar Gulcehre, KyungHyun Cho, and Yoshua Bengio. 2014. Empirical evaluation of gated recurrent neural networks on sequence modeling. arXiv preprint arXiv:1412.3555 (2014).
6. Kun He, Yan Wang, and John Hopcroft. 2016. A powerful generative model using random weights for the deep image representation. In Advances in Neural Information Processing Systems. 631--639.
7. Geoffrey Hinton, Li Deng, Dong Yu, George E Dahl, Abdel-rahman Mohamed, Navdeep Jaitly, Andrew Senior, Vincent Vanhoucke, Patrick Nguyen, Tara N Sainath, et al. 2012. Deep neural networks for acoustic modeling in speech recognition: The shared views of four research groups. IEEE Signal Processing Magazine 29, 6 (2012), 82--97.
8. Risto Miikkulainen, Jason Liang, Elliot Meyerson, Aditya Rawal, Dan Fink, Olivier Francon, Bala Raju, Hormoz Shahrzad, Arshak Navruzyan, Nigel Duffy, and Babak Hodjat. {n. d.}. Evolving Deep Neural Networks. ({n. d.}). arXiv:cs.NE/1703.00548v2
9. Volodymyr Mnih, AdriÃă PuigdomÃÍnech Badia, Mehdi Mirza, Alex Graves, Timothy P. Lillicrap, Tim Harley, David Silver, and Koray Kavukcuoglu. {n. d.}. Asynchronous Methods for Deep Reinforcement Learning. ({n. d.}). arXiv:cs.LG/1602.01783v2
10. Renato Negrinho and Geoff Gordon. 2017. Deeparchitect: Automatically designing and training deep architectures. arXiv preprint arXiv:1704.08792 (2017).
11. OpenAI. 2017. OpenAI Baselines: ACKTR & A2C. (Nov 2017). https://blog.openai.com/baselines-acktr-a2c/
12. Kenneth O Stanley and Risto Miikkulainen. 2002. Evolving neural networks through augmenting topologies. Evolutionary computation 10, 2 (2002), 99--127.
13. Ronald J Williams. 1992. Simple statistical gradient-following algorithms for connectionist reinforcement learning. In Reinforcement Learning. Springer, 5--32.
14. Barret Zoph and Quoc V Le. 2016. Neural architecture search with reinforcement learning. arXiv preprint arXiv:1611.01578 (2016).

Index Terms

• Computing methodologies

  • Distributed computing methodologies

  • Machine learning

    • Learning paradigms

      • Reinforcement learning

    • Machine learning approaches

      • Neural networks