Загрузка...

ActNet - Deep Learning Alternatives of the Kolmogorov Superposition Theorem

This talk was given at the University of Pennsylvania's Graduate Applied Math Seminar (GAMeS) series on February, 4th 2025. In it, I talk about my recent paper (presented at ICLR 2025 in Singapore), which exploits alternative versions of the Kolmogorov Superposition Theorem to create the ActNet neural network architecture. A full abstract is listed below the links in this description.

Links:
- arXiv: https://arxiv.org/abs/2410.01990
- GitHub: https://github.com/PredictiveIntelligenceLab/ActNet
- GAMeS Seminar: https://www.sas.upenn.edu/games-seminar/index.html

---------------------
CHAPTERS:
0:00 Intro
0:48 Understanding the KST
10:20 Alternative Versions of the KST
11:55 Laczkovich KST
15:09 Creating the ActLayer
17:52 The ActLayer
21:14 The ActNet
31:48 Intro to Physics Informed Neural Networks (PINNs)
35:45 KST and PINNs
40:53 ActLayer Derivatives Don't Vanish
42:20 Experiments
52:36 Conclusion
---------------------
----------------------
Paper:
Deep Learning Alternatives of the Kolmogorov Superposition Theorem

Authors:
Leonardo Ferreira Guilhoto & Paris Perdikaris

Abstract:
In this talk, I discuss my most recent paper (https://arxiv.org/abs/2410.01990), where we explore alternative formulations of the Kolmogorov Superposition Theorem (KST) as a foundation for neural network design. The original KST formulation, while mathematically elegant, presents practical challenges due to its limited insight into the structure of inner and outer functions and the large number of unknown variables it introduces. The recently proposed Kolmogorov-Arnold Networks (KANs) leverage KST for function approximation, but they have faced scrutiny due to mixed results compared to traditional multilayer perceptrons (MLPs) and practical limitations imposed by the original KST formulation. To address these issues, we introduce ActNet, a scalable deep learning model that builds on the KST and overcomes many of the drawbacks of Kolmogorov's original formulation. We evaluate ActNet in the context of Physics-Informed Neural Networks (PINNs), a framework well-suited for leveraging KST's strengths in low-dimensional function approximation, particularly for simulating partial differential equations (PDEs). In this challenging setting, where models must learn latent functions without direct measurements, ActNet consistently outperforms KANs across multiple benchmarks and is competitive against the current best MLP-based approaches. These results present ActNet as a promising new direction for KST-based deep learning applications, particularly in scientific computing and PDE simulation tasks.

Видео ActNet - Deep Learning Alternatives of the Kolmogorov Superposition Theorem канала Leonardo Ferreira Guilhoto
Яндекс.Метрика

На информационно-развлекательном портале SALDA.WS применяются cookie-файлы. Нажимая кнопку Принять, вы подтверждаете свое согласие на их использование.

Об использовании CookiesПринять