Nonlinear optimisation using directional step lengths based on RPROP

Kotsialos, A.

Abstract

A search method based on the backpropagation rule commonly used for training neural networks is proposed here for the optimisation of smooth nonlinear functions. The use of the Resilient backPROPagation (RPROP) heuristic rule for local minimisation is described. The details of employing the directional step length determined by RPROP along with a simple restarting scheme are provided. In the approach proposed here direct use of the directional step determined by the heuristic without using any line search conditions takes place. The overall algorithm has been tested on a number of benchmark functions found in the literature with very positive results. The test problems’ dimension ranges from 100 to 50,000. The results obtained show that the suggested search direction method results to a highly efficient algorithm suitable for large scale optimisation.

Citation

Kotsialos, A. (2014). Nonlinear optimisation using directional step lengths based on RPROP. Optimization Letters, 8(4), 1401-1415. https://doi.org/10.1007/s11590-013-0668-8

• Journal Article Type: Article
• Acceptance Date: Jun 15, 2013
• Publication Date: Apr 1, 2014
• Deposit Date: Sep 16, 2013
• Journal: Optimization Letters
• Print ISSN: 1862-4472
• Electronic ISSN: 1862-4480
• Publisher: Springer
• Peer Reviewed: Peer Reviewed
• Volume: 8
• Issue: 4
• Pages: 1401-1415
• DOI: https://doi.org/10.1007/s11590-013-0668-8
• Keywords: Nonlinear optimization, Search methods, Backpropagation methods, RPROP.
• Public URL: https://durham-repository.worktribe.com/output/1447263