Advancing Spark - Rethinking ETL with Databricks Autoloader

Figuring out what data to load can be tricky. but Databricks have the answer! The Databricks Autoloader function in Azure uses Event Grid too automatically ingest files as they land - rather than building out file-watching, polling functionality manually.

Join Simon as he runs through the setup of autoloader and looks at some different ways of using the tool.

For more information about Autoloader - see: https://docs.microsoft.com/en-us/azure/databricks/spark/latest/structured-streaming/auto-loader

Otherwise, don't forget to subscribe for more Sparky goodness, or check out our blogs at: https://www.advancinganalytics.co.uk/blog

Видео Advancing Spark - Rethinking ETL with Databricks Autoloader канала Advancing Analytics