Bringing Software Engineering Practices to Data and Artificial Intelligence — Denny Lee, Databricks

In the rush to store everything and parallelize data processing, the art and rigor of building reliable data systems have been lost. Denny will begin this session with a focus on the fundamentals of reliable data: providing both a retrospective of the database and data warehousing age and how those concepts differ and resonate in the age of data.

Storing data without regard for why, what, and how it is being stored can create a new set of problems. Denny suggests that these problems can be avoided by applying rigorous software engineering practices to data and data engineering.

In this talk, Denny will explore how the distribution of data, microservices, and shifting to the cloud has solved many problems but has also introduced new ones. He’ll then review data engineering best practices and the machine learning life cycle. The key theme is the merging (finally) of data engineering and software engineering practices.

https://www.ischool.berkeley.edu/events/2021/bringing-software-engineering-practices-data-and-artificial-intelligence

Видео Bringing Software Engineering Practices to Data and Artificial Intelligence — Denny Lee, Databricks канала Berkeley School of Information