#71 - ZAK JOST (Graph Neural Networks + Geometric DL) [UNPLUGGED]

Special discount link for Zak's GNN course - https://bit.ly/3uqmYVq
Patreon: https://www.patreon.com/mlst
Discord: https://discord.gg/ESrGqhf5CB
Pod: https://anchor.fm/machinelearningstreettalk/episodes/71---ZAK-JOST-Graph-Neural-Networks--Geometric-DL-UNPLUGGED-e1g8dvr

Want to sponsor MLST!? Let us know on Linkedin / Twitter.

[00:00:00] Preamble
[00:03:12] Geometric deep learning
[00:10:04] Message passing
[00:20:42] Top down vs bottom up
[00:24:59] All NN architectures are different forms of information diffusion processes (squashing and smoothing problem)
[00:29:51] Graph rewiring
[00:31:38] Back to information diffusion
[00:42:43] Transformers vs GNNs
[00:47:10] Equivariant subgraph aggregation networks + WL test
[00:55:36] Do equivariant layers aggregate too?
[00:57:49] Zak's GNN course

References;
Welcome AI Overlords YT channel
https://www.youtube.com/channel/UCxw9_WYmLqlj5PyXu2AWU_g

Author Interview - Equivariant Subgraph Aggregation Networks
https://www.youtube.com/watch?v=VYZog7kbXks
https://arxiv.org/abs/2110.02910

Geometric Deep Learning Grids, Groups, Graphs, Geodesics, and Gauges
https://arxiv.org/abs/2104.13478

Joan Bruna sources of error in learning
https://cims.nyu.edu/~bruna/
https://www.youtube.com/watch?v=4RmpSvQ2LL0

Blind men and an elephant
https://en.wikipedia.org/wiki/Blind_men_and_an_elephant

Geometric Deep Learning From Learning ODE Dynamics towards Graph Neural Diffusion [Brune]

https://bathicmsworkshop.github.io/ChristophBrune.pdf
The Road to Reality: A Complete Guide to the Laws of the Universe

https://www.amazon.co.uk/Road-Reality-Complete-Guide-Universe/dp/0099440687

Lenia - Mathematical Life Forms [Cellula Automata]
https://www.youtube.com/watch?v=iE46jKYcI4Y

Graph Neural Networks - a perspective from the ground up [Alex Foo]
https://www.youtube.com/watch?v=GXhBEj1ZtE8

SEMI-SUPERVISED CLASSIFICATION WITH GRAPH CONVOLUTIONAL NETWORKS [Kipf]
https://arxiv.org/pdf/1609.02907.pdf

Convolution theorem
https://en.wikipedia.org/wiki/Convolution_theorem
https://en.wikipedia.org/wiki/Graph_Fourier_transform

Wavelets on Graphs via Spectral Graph Theory [Hammond]
https://arxiv.org/pdf/0912.3848.pdf

Growing Neural Cellular Automata
https://distill.pub/2020/growing-ca/

Rediscovering the power of pairwise interactions [William Bialek]
https://www.princeton.edu/~wbialek/rome/refs/bialek+ranganathan_07.pdf

UNDERSTANDING OVER-SQUASHING AND BOTTLENECKS ON GRAPHS VIA CURVATURE [Topping12, inc Bronstein]
https://arxiv.org/pdf/2111.14522.pdf
https://towardsdatascience.com/over-squashing-bottlenecks-and-graph-ricci-curvature-c238b7169e16

GRAND: Graph Neural Diffusion [Chamberlain, inc Bronstein]
http://proceedings.mlr.press/v139/chamberlain21a/chamberlain21a.pdf

https://blog.twitter.com/engineering/en_us/topics/insights/2021/graph-neural-networks-as-neural-diffusion-pdes

Dr. Daniele Grattarola
https://danielegrattarola.github.io/

ON THE UNREASONABLE EFFECTIVENESS OF FEATURE PROPAGATION IN LEARNING ON GRAPHS WITH MISSING NODE FEATURES [Rossi + Bronstein et al]
https://arxiv.org/pdf/2111.12128.pdf

A Spline Theory of Deep Learning [_**Balestriero**_]
https://proceedings.mlr.press/v80/balestriero18b.html

COMBINING LABEL PROPAGATION AND SIMPLE MODELS OUT-PERFORMS GRAPH NEURAL NETWORK (Correct and smooth) [Huang]
https://arxiv.org/pdf/2010.13993.pdf

Review: Deep Learning on Sets [Fuchs]
https://fabianfuchsml.github.io/learningonsets/

Transformers are Graph Neural Networks
https://thegradient.pub/transformers-are-graph-neural-networks/

The Weisfeiler-Lehman Isomorphism Test
https://davidbieber.com/post/2019-05-10-weisfeiler-lehman-isomorphism-test/

How Powerful are Graph Neural Networks? [Xu, Stefanie Jegelka]
https://arxiv.org/abs/1810.00826

Видео #71 - ZAK JOST (Graph Neural Networks + Geometric DL) [UNPLUGGED] канала Machine Learning Street Talk