Stochastic quasi-Newton with line-search regularisation

Title: Stochastic quasi-Newton with line-search regularisation
Creator: Wills, Adrian G.; Schön, Thomas B.
Relation: Automatica Vol. 127, Issue May 2021, no. 109503
Publisher Link: http://dx.doi.org/10.1016/j.automatica.2021.109503
Publisher: Elsevier
Resource Type: journal article
Date: 2021
Description: In this paper we present a novel quasi-Newton algorithm for use in stochastic optimisation. Quasi-Newton methods have had an enormous impact on deterministic optimisation problems because they afford rapid convergence and computationally attractive algorithms. In essence, this is achieved by learning the second-order (Hessian) information based on observing first-order gradients. We extend these ideas to the stochastic setting by employing a highly flexible model for the Hessian and infer its value based on observing noisy gradients. In addition, we propose a stochastic counterpart to standard line-search procedures and demonstrate the utility of this combination on maximum likelihood identification for general nonlinear state space models.
Subject: nonlinear system identification; stochastic optimisation; stochastic gradient; stochastic quasi-Newton; sequential Monte Carlo; particle filter; gaussian process
Identifier: http://hdl.handle.net/1959.13/1435726
Identifier: uon:39800
Identifier: ISSN:0005-1098
Language: eng
Reviewed

		Thumbnail	File	Description	Size	Format