SIREN in PyTorch

In this video I implement the SIREN. It was proposed in the paper "Implicit Neural Representations with Periodic Activation Functions". It is a feedforward neural network that is using the sine function as activations. This video focuses on a case where the input features are 2D and they represent coordinates of an image. I discuss multiple relevant topics like the weight initialization and higher order derivatives.

Amazing high-level presentation: https://youtu.be/Q2fLWGBeaiI
The official implementation: https://github.com/vsitzmann/siren
Implementation from the video: https://github.com/jankrepl/mildlyoverfitted/tree/master/github_adventures/siren

00:00 Intro
00:27 Problem definition
01:58 Initialization scheme
03:11 Sine layer
04:55 Siren network
07:16 Activations analysis
11:25 Dataset creation + utils
16:25 DataLoader behavior
18:00 Higher order derivatives
19:32 Gradient utilities
21:18 Training script
27:21 Results
28:49 Outro

If you have any video suggestions or you just wanna chat feel free to join the discord server: https://discord.gg/a8Va9tZsG5

Credits logo animation
Title: Conjungation · Author: Uncle Milk · Source: https://soundcloud.com/unclemilk · License: https://creativecommons.org/licenses/... · Download (9MB): https://auboutdufil.com/?id=600

Видео SIREN in PyTorch канала mildlyoverfitted