Algebraic Dynamical Systems in Machine Learning

Jones, Iolo; Swan, Jerry; Giansiracusa, Jeffrey

Abstract

We introduce an algebraic analogue of dynamical systems, based on term rewriting. We show that a recursive function applied to the output of an iterated rewriting system defines a formal class of models into which all the main architectures for dynamic machine learning models (including recurrent neural networks, graph neural networks, and diffusion models) can be embedded. Considered in category theory, we also show that these algebraic models are a natural language for describing the compositionality of dynamic models. Furthermore, we propose that these models provide a template for the generalisation of the above dynamic models to learning problems on structured or non-numerical data, including ‘hybrid symbolic-numeric’ models.

Citation

Jones, I., Swan, J., & Giansiracusa, J. (2024). Algebraic Dynamical Systems in Machine Learning. Applied Categorical Structures, 32(1), Article 4. https://doi.org/10.1007/s10485-023-09762-9

• Journal Article Type: Article
• Acceptance Date: Dec 13, 2023
• Online Publication Date: Jan 18, 2024
• Publication Date: Feb 1, 2024
• Deposit Date: Dec 7, 2023
• Publicly Available Date: Jan 19, 2024
• Journal: Applied Categorical Structures
• Print ISSN: 0927-2852
• Electronic ISSN: 1572-9095
• Publisher: Springer
• Peer Reviewed: Peer Reviewed
• Volume: 32
• Issue: 1
• Article Number: 4
• DOI: https://doi.org/10.1007/s10485-023-09762-9
• Keywords: Term rewriting, Functional programming, Machine learning, Compositionality, Dynamical systems
• Public URL: https://durham-repository.worktribe.com/output/1984060