Bin Bao at Modular Hack Weekend: From PyTorch Models to Standalone Binaries

Bin Bao, Software Engineer and Tech Lead on torch.compile at Meta, shares how PyTorch models can now be compiled into standalone binaries. No Python, no libtorch, and no virtual environment required. Learn how ahead-of-time compilation in PyTorch 2 makes deployment faster and easier, with examples like an 8MB Llama 3.1 binary.

This talk was part of Modular's GPU Programming Workshop, which kicked off Modular Hack Weekend: 3 days of hands-on hacking, community building, and deep dives into building with GPUs, Mojo, and MAX.

Watch the highlight reel: https://www.youtube.com/watch?v=cEekC9xOTXQ
Check out the winning projects: https://forum.modular.com/t/modular-hack-weekend-winners-announced/1848
Follow our calendar page to stay up-to-date on future events: https://lu.ma/modular-ai

00:00 Intro
00:51 How fast can you deploy a PyTorch model?
2:42 Deploy PyTorch model as standalone binary
3:45 Ahead-of-time compilation flow in PyTorch 2
6:21 Example 1
6:49 Example 2 (TORCH_LOGS=output_code)
10:41 TorchNative standalone

Join our community 🤝
Forum - https://forum.modular.com/
GitHub - https://github.com/modular
X (aka Twitter) - https://x.com/modular
LinkedIn - https://www.linkedin.com/company/modular-ai
Reddit - https://www.reddit.com/r/ModularAI/

#gpu #programming #ml

Видео Bin Bao at Modular Hack Weekend: From PyTorch Models to Standalone Binaries канала Modular