Abstract
We present a new type of optimization algorithms, adapted for neural network training. These algorithms are based upon sequential operator splitting technique for some associated dynamical systems. Furthermore, we investigate through numerical simulations the empirical rate of convergence of these iterative schemes toward a local minimum of the loss function, with some suitable choices of the underlying hyper parameters. We validate the convergence of these optimizers using the results of the accuracy and of the loss function on the MNIST,MNIST-Fashion and CIFAR 10 classification datasets.
Authors
I.D. Voiculescu
(Oxford, MPLS, Computer Science)
I. Boros
(Tiberiu Popoviciu Institute of Numerical Analysis, Romanian Academy, Cluj-Napoca, Romania)
N. Popovici
L. Diosan
A. Andreica
Keywords
Paper coordinates
L. Dioşan, A. Andreica, I. Boros, I. Voiculescu, N. Popovici, Interval-state cellular automata and their applications to image segmentation, SAGE Publications, 2017,
https://ora.ox.ac.uk/objects/uuid:eacaf9ce-99c2-49dd-8e9e-faced821233a
About this paper
Journal
Publisher Name
DOI
Print ISSN
Online ISSN
google scholar link
[1] Laura Diosan, Anca Andreica, Imre Boros and Irina Voiculescu. Avenues for the Use of Multi{State Cellular Automata. Proc. Evolutionary Algorithms and Complex Systems. EvoStar 2017.
[2] Laura Diosan, Anca Andreica and Irina Voiculescu. Parameterized Cellular Automata in Image Segmentation. SYNASC 2016.
[3] Dmitri Chiriaev, G. William Walster. Interval Arithmetic Speci cation. Technical Report 1998.
[4] D. Martin, C. Fowlkes, D. Tal, J. Malik. A Database of Human Segmented Natural Images. Proc 8th Intl Conf Comp Vis 2001.
[5] Nobuyuki Otsu. A Threshold Selection Method from Gray-Level Histograms. IEEE Trans Syst Man Cybernetics 9.1 (1979): 62-66.