Laura Lorenz | How I learned to time travel, or, data pipelining and scheduling with Airflow

PyData DC 2016

Slides: http://www.slideshare.net/PyData/how-i-learned-to-time-travel-or-data-pipelining-and-scheduling-with-airflow-67650418

Data warehousing and analytics projects can, like ours, start out small - and fragile. With an organically growing mess of scripts glued together and triggered by cron jobs hiding on different servers, we needed better plumbing. After perusing the data pipelining landscape, we landed on Airflow, an Apache incubating batch processing pipelining and scheduler tool from Airbnb.

The open source project is at https://www.github.com/industrydive/fileflow 00:00 Welcome!
00:10 Help us add time stamps or captions to this video! See the description for details.

Want to help add timestamps to our YouTube videos to help with discoverability? Find out more here: https://github.com/numfocus/YouTubeVideoTimestamps

Видео Laura Lorenz | How I learned to time travel, or, data pipelining and scheduling with Airflow канала PyData