Optimising Snowflake Data Storage for Speed and Efficiency

📩 DOWNLOAD THE SLIDES FROM THIS SESSION: https://www.data-community.org/meetups-1/template-driven-data-vault:-a-code-centric-approach-to-master-complexity

🔍 Optimising Snowflake Data Storage for Speed and Efficiency
🎙️ Presented by Will Riley | Data Community Meetup

Will Riley breaks down how Snowflake’s storage engine really works and shows how understanding micro‑partitions, metadata, and natural clustering can dramatically improve query performance and reduce compute costs. He explains why features like Time Travel, immutable micro‑partitions, and column‑level statistics shape the way Snowflake retrieves data, and how simple changes to ingest patterns, ordering, and table design can have a major impact on pruning efficiency.

Will walks through practical techniques for analysing clustering depth, identifying high‑value predicates, and spotting tables where overlapping ranges cause unnecessary scanning. Through live demonstrations, he compares unordered and ordered tables, showing how loading data in the right sequence can transform query times, shrink warehouse consumption, and create more predictable performance. The session also highlights best practices for data types, insert‑only patterns, and avoiding functions that block pruning, giving teams a pragmatic toolkit for building faster and more cost‑efficient Snowflake workloads.

Speaker Bio
Will Riley is a Solutions Architect at Snowflake (EMEA). Over seven years working with the platform, he has helped customers evolve from “just make the warehouse bigger” to data‑first designs that exploit Snowflake’s storage, pruning, and clustering behavior for faster analytics at lower cost.

⏱️ Timestamps
00:00 Introduction and why “design for pruning” matters
02:40 Snowflake storage primer: micro‑partitions, immutability, Time Travel
07:10 Natural clustering from ingest order and why it drives pruning efficiency
12:30 Query pruning explained: min/max metadata, overlap, and depth
18:20 Measuring clustering with system clustering information and selectivity tests
24:50 Demo: unordered vs ordered tables, partitions scanned, and runtime impact
33:00 When to add clustering keys, sort‑on‑load, or rebuild tables for common predicates
40:20 Practical tips: data types, avoiding predicate functions, Gen2 warehouse gains
47:10 Cost and governance considerations: replication, DML churn, insert‑only patterns
53:30 Q&A and next steps for deeper optimization labs

🔗 Resources & Links:
👉 Join our FREE Q&A Forum: https://forum.data-community.org/invites/ceu8YUyDa5
👉 Connect with the Data Community: https://www.linkedin.com/company/data-community-bt/
👉 https://www.snowflake.com/en/

📌 Don’t forget to like, comment, and subscribe for more expert-led talks on data engineering, modelling, and architecture!

Видео Optimising Snowflake Data Storage for Speed and Efficiency канала Business Thinking