Argo: Kubernetes Native Workflows and Pipelines | Canva
Get the slides: https://www.datacouncil.ai/talks/argo-kubernetes-native-workflows-and-pipelines?hsLang=en
ABOUT THE TALK
Data orchestration and DAGs are something that most data teams need. There are many commercial and open source options available. Examples include Airflow, Luigi, Oozie and many others.
Airflow is very popular at the moment and rightly so; it is a very useful tool and is the backbone of very productive data teams. Argo is a relatively new challenger. It is a Kubernetes native workflow engine.
At Canva, we evaluated both Airflow and Argo and chose Argo as our primary data orchestration system. In this talk I’ll briefly compare Airflow and Argo, talk about the evaluation process we undertook and how we came to our decision. Finally, I’ll talk about our experience using it so far, the things that have been good and the things that have been not so good.
ABOUT THE SPEAKER
Greg is the Data Engineering Lead at Canva. He founded the team and quickly grew it to build out scalable systems that enable advanced analytics and machine learning features inside the Canva product.
Previously he was the Co-founder and CTO of AirHelp, a Y Combinator backed startup, where he built systems to process flight and booking data in real-time.
Greg is passionate about technology and is frequently involved with meetups, such as the Sydney Data Engineering meetup.
ABOUT DATA COUNCIL:
Data Council (https://www.datacouncil.ai/) is a community and conference series that provides data professionals with the learning and networking opportunities they need to grow their careers. Make sure to subscribe to our channel for more videos, including DC_THURS, our series of live online interviews with leading data professionals from top open source projects and startups.
FOLLOW DATA COUNCIL:
Twitter: https://twitter.com/DataCouncilAI
LinkedIn: https://www.linkedin.com/company/datacouncil-ai
Facebook: https://www.facebook.com/datacouncilai
Eventbrite: https://www.eventbrite.com/o/data-council-30357384520
Видео Argo: Kubernetes Native Workflows and Pipelines | Canva канала Data Council
ABOUT THE TALK
Data orchestration and DAGs are something that most data teams need. There are many commercial and open source options available. Examples include Airflow, Luigi, Oozie and many others.
Airflow is very popular at the moment and rightly so; it is a very useful tool and is the backbone of very productive data teams. Argo is a relatively new challenger. It is a Kubernetes native workflow engine.
At Canva, we evaluated both Airflow and Argo and chose Argo as our primary data orchestration system. In this talk I’ll briefly compare Airflow and Argo, talk about the evaluation process we undertook and how we came to our decision. Finally, I’ll talk about our experience using it so far, the things that have been good and the things that have been not so good.
ABOUT THE SPEAKER
Greg is the Data Engineering Lead at Canva. He founded the team and quickly grew it to build out scalable systems that enable advanced analytics and machine learning features inside the Canva product.
Previously he was the Co-founder and CTO of AirHelp, a Y Combinator backed startup, where he built systems to process flight and booking data in real-time.
Greg is passionate about technology and is frequently involved with meetups, such as the Sydney Data Engineering meetup.
ABOUT DATA COUNCIL:
Data Council (https://www.datacouncil.ai/) is a community and conference series that provides data professionals with the learning and networking opportunities they need to grow their careers. Make sure to subscribe to our channel for more videos, including DC_THURS, our series of live online interviews with leading data professionals from top open source projects and startups.
FOLLOW DATA COUNCIL:
Twitter: https://twitter.com/DataCouncilAI
LinkedIn: https://www.linkedin.com/company/datacouncil-ai
Facebook: https://www.facebook.com/datacouncilai
Eventbrite: https://www.eventbrite.com/o/data-council-30357384520
Видео Argo: Kubernetes Native Workflows and Pipelines | Canva канала Data Council
Показать
Комментарии отсутствуют
Информация о видео
Другие видео канала
Kubernetes Webinar Series - Building CI CD Pipelines with Jenkins and KubernetesAWS Sydney Summit: Snowflake: Your Data. No Limits.Argo Workflows 101 Workshop 22 Sep 2020Kubernetes-Native Workflow Orchestration with Argo | SkillshareData Pipeline Frameworks: The Dream and the Reality | BeeswaxLISA19 - Deep Dive into Kubernetes Internals for Builders and OperatorsAWS EKS - Create Kubernetes cluster on Amazon EKS | the easy wayRook-ceph on Raspberry Pi Kubernetes cluster using cheap flashdrivesOrganizing the YAML mess with Kustomize - Florian AssmusAirflow on Kubernetes - Scaling DAG Workflows | Daniel Imberman, Seth Edwards @ PyBay2018Introduction to Argo CD : Kubernetes DevOps CI/CDArgo CD - Applying GitOps Principles To Manage Production Environment In KubernetesAn Introduction to Systems & Service Monitoring with Prometheus • Julius Volz • GOTO 2019Leveraging CircleCI for DigitalOcean Kubernetes DeploymentKubernetes Tutorial for Beginners [FULL COURSE in 4 Hours]Kelsey, Kubernetes, and GitOps - GitHub Universe 2020Accelerating Machine Learning App Development with Kubeflow Pipelines (Cloud Next '19)Advanced Celery Tricks - Itamar Hartstein - PyCon Israel 2018AWS re:Invent 2019: More data science with less engineering: ML infrastructure at Netflix (NFX201)