Advancing Spark - Databricks Photon First Look

Since the initial announcement of the new execution engine, there's been a lot of buzz around Databricks Photon but not too much tangible information. We saw some deep dive technical talks at the last Data & AI Summit which tells us the kind of performance tuning & query types the engine aims to improve, and we can test out how the engine works with these different scenarios.

In this video, Simon walks through a couple of example queries run on the same size cluster both with and without Photon enabled. We'll see the types of execution pattern it's best at, and how to start on the performance tuning journey with the new engine.

During the video, we call out a couple of tech talks, we highly recommend checking out the details. Firstly with Alex Behm's technical deep dive - https://databricks.com/session_eu20/photon-technical-deep-dive-how-to-think-vectorized

Then with Todd Greenstein's Delta & Photon higher level overview - https://databricks.com/session_eu20/radical-speed-for-your-sql-queries-with-delta-engine

As always - let us know your thoughts in the comments, and don't forget to like & subscribe!

Видео Advancing Spark - Databricks Photon First Look канала Advancing Analytics