Data and Engineering @ Exa

Hubert Yuan, Software Engineer from Exa, shares the philosophy and engineering behind building a search engine to surpass Google. He walks through Exa's internal data platform — code-named Exa-D — which powers their web-scale indexing pipeline.

The talk covers how Exa uses Lance as a storage layer (leveraging its manifest for O(1) state tracking and zero-cost column evolution) and Ray for distributed compute (enabling CPU/GPU pipeline parallelism). Hubert argues that in the AI era, the best engineers focus on architecture and system design while offloading empirical/debugging work to agents.

Blog: https://exa.ai/blog/exa-d

🔗 Connect with us:
Website: https://lancedb.com
GitHub: https://github.com/lancedb/lancedb
Discord: https://discord.gg/qjGtt4JYCu
LinkedIn: https://www.linkedin.com/company/lancedb
Twitter: https://x.com/lancedb

Видео Data and Engineering @ Exa канала LanceDB