Implementing a transformer with candle + Rust - (Part11, Back-prop implementation)

This series is about implementing a plain vanilla transformer in Rust using the candle framework. We cover the following -

0:00:00 a quick intro and recap
00:01:54 a simplified overview of back-prop in neural networks
00:07:55 how does candle perform back-prop
00:08:50 candle's Tensor and Op type(s)
00:12:00 topological ordering, impl and demo
00:22:00 problems with current impl of topological ordering
00:24:15 a quick summary
00:24:45 bonus - How do we compute the actual gradients (auto-diff)
00:27:40 wrap-up

In Part12 - lets look at dataset preparation, training and validation

#apple #rust #gpu #candle #framework

Видео Implementing a transformer with candle + Rust - (Part11, Back-prop implementation) канала regionaltantrums