A covariance matrix adaptation evolution strategy in reproducing kernel Hilbert space

Viet Hung Dang, Ngo Anh Vien*, Tae Choong Chung

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review

129 Downloads (Pure)

Abstract

The covariance matrix adaptation evolution strategy (CMA-ES) is an efficient derivative-free optimization algorithm. It optimizes a black-box objective function over a well-defined parameter space in which feature functions are often defined manually. Therefore, the performance of those techniques strongly depends on the quality of the chosen features or the underlying parametric function space. Hence, enabling CMA-ES to optimize on a more complex and general function class has long been desired. In this paper, we consider modeling the input spaces in black-box optimization non-parametrically in reproducing kernel Hilbert spaces (RKHS). This modeling leads to a functional optimisation problem whose domain is a RKHS function space that enables optimisation in a very rich function class. We propose CMA-ES-RKHS, a generalized CMA-ES framework that is able to carry out black-box functional optimisation in RKHS. A search distribution on non-parametric function spaces, represented as a Gaussian process, is adapted by updating both its mean function and covariance operator. Adaptive and sparse representation of the mean function and the covariance operator can be retained for efficient computation in the updates and evaluations of CMA-ES-RKHS by resorting to sparsification. We will also show how to apply our new black-box framework to search for an optimum policy in reinforcement learning in which policies are represented as functions in a RKHS. CMA-ES-RKHS is evaluated on two functional optimization problems and two bench-marking reinforcement learning domains.

Original languageEnglish
Pages (from-to)1-23
JournalGenetic Programming and Evolvable Machines
DOIs
Publication statusPublished - 19 Jun 2019

Keywords

  • Covariance matrix adaptation-evolution strategies (CMA-ES)
  • Functional optimization
  • Kernel methods
  • Policy search
  • Reinforcement learning
  • Reproducing kernel Hilbert space
  • Robot learning

ASJC Scopus subject areas

  • Software
  • Theoretical Computer Science
  • Hardware and Architecture
  • Computer Science Applications

Fingerprint Dive into the research topics of 'A covariance matrix adaptation evolution strategy in reproducing kernel Hilbert space'. Together they form a unique fingerprint.

Cite this