Taking the Human Out of the Loop: A Review of Bayesian Optimization

The paper introduces the reader to Bayesian optimization, highlighting its methodical aspects and showcasing its applications.

By Bobak Shahriari, Kevin Swersky, Ziyu Wang, Ryan P. Adams, and Nando de Freitas


ABSTRACT | Big Data applications are typically associated with
systems involving large numbers of users, massive complex
software systems, and large-scale heterogeneous computing
and storage architectures. The construction of such systems
involves many distributed design choices. The end products
(e.g., recommendation systems, medical analysis tools, realtime
game engines, speech recognizers) thus involve many
tunable configuration parameters. These parameters are
often specified and hard-coded into the software by various
developers or teams. If optimized jointly, these parameters
can result in significant improvements. Bayesian optimization
is a powerful tool for the joint optimization of design choices
that is gaining great popularity in recent years. It promises
greater automation so as to increase both product quality and
human productivity. This review paper introduces Bayesian
optimization, highlights some of its methodological aspects,
and showcases a wide range of applications.

KEYWORDS | Decision making; design of experiments; optimization;
response surface methodology; statistical learning


Proceedings of the IEEE | Vol. 104, No. 1, January 2016