Fast Recovery in ML Training with FLOR - Talk by Rolando Garcia Sanchez (UC Berkeley)

There’s lots to log in model training: time series of metrics, tensor histograms, embeddings, activations, and more. Surprisingly though, most of the time, model developers only log parameters (e.g. batch size and learning rate) and time series metrics (e.g. loss and accuracy). What do they do when they need more data than the little bit they logged? They add more logging statements, and they re-run training. We call this practice “Hindsight Logging” because of its retrospective quality. In this talk, I will discuss efficient hindsight logging using Fast Low-Overhead Recovery (FLOR), a record-replay system first presented at VLDB ’21. During my talk, I will give a demo of FLOR using VSCode and a Jupyter Notebook, and show how I can use it, together with integrated query support, to understand what alternatives my colleague has tried to fit an NLP model for a Kaggle competition.

Видео Fast Recovery in ML Training with FLOR - Talk by Rolando Garcia Sanchez (UC Berkeley) автора JavaScript: основы и применение