Data Lineage with OpenLineage and Airflow
If one out of your hundreds of DAGs fails, how do you know which downstream datasets have become out-of-date? The answer is data lineage. Data lineage is the complex set of relationships between your jobs and datasets. In this webinar, you'll learn how to use OpenLineage to collect lineage metadata from Airflow and assemble a lineage graph - a picture of your pipeline worth way more than a thousand words.
Видео Data Lineage with OpenLineage and Airflow канала Astronomer
Видео Data Lineage with OpenLineage and Airflow канала Astronomer
Показать
Комментарии отсутствуют
Информация о видео
Другие видео канала
LIVE with Astronomer: Set-up/Tear-down Tasks Deep Dive with Daniel Standish!LIVE with Astronomer: Committing to AirflowWhat's new in Airflow 2.8How to use DuckDB with AirflowThe SQL Column Check OperatorLIVE with Astronomer: Airflow and Databricks, Better Together!Reliable Data Orchestration for AI ApplicationsLIVE With Astronomer: Airflow and Astro API OverviewUsing Airflow with Tensorflow and MLFlowAirflow Summit Invitation by Marc Lamberti, Head of Customer Training at AstronomerHow to Use the TaskFlow API and Traditional Operators Together to Create More Efficient DAGs!The Airflow Brief - 2.0 Release Day Interview with Ash & KaxilAirflow Summit Kaxil NaikLIVE with Astronomer: How Chartboost Uses Airflow PluginsAirflow 2.7 Feature Update with Committer Utkarsh SharmaLIVE with Astronomer - ML & worker queuesAirflow 2.7 UI Updates with Brent BovenziUsing the Snowflake Deferrable OperatorLIVE with Astronomer: How to Use MLFlow and Airflow for Machine Learning Orchestration!Writing Functional DAGs with DecoratorsOpenLineage and Airflow: A Deeper Dive