Geospatial Options in Apache Spark
Geospatial data appears to be simple right up until the part when it becomes intractable. There are many gotcha moments with geospatial data in spark and we will break those down in our talk. Users who are new to geospatial analysis in spark will find this portion useful as projections, geometry types, indices, and geometry storage can cause issues. We will begin by discussing the basics of geospatial data and why it can be so challenging. This will be brief and will be in the context of how geospatial data can cause scaling problems in spark. Critically, we will show how we have approached these issues to limit errors and reduce cost. There are many geospatial packages available within Spark. We have tried many of them and will discuss the pros and cons of each using common examples across libraries. New users will benefit from this discussion as each library has advantages in specific scenarios. Lastly, we will discuss how we migrate geospatial data. This will include our best practices for ingesting geospatial data as well as how we store it for long term use. Users may be specifically interested in our evaluation of spatial indexing for rapid retrieval of records.
About:
Databricks provides a unified data analytics platform, powered by Apache Spark™, that accelerates innovation by unifying data science, engineering and business.
Read more here: https://databricks.com/product/unifie...
Connect with us:
Website: https://databricks.com
Facebook: https://www.facebook.com/databricksinc
Twitter: https://twitter.com/databricks
LinkedIn: https://www.linkedin.com/company/data...Instagram: https://www.instagram.com/databricksinc/ Databricks is proud to announce that Gartner has named us a Leader in both the 2021 Magic Quadrant for Cloud Database Management Systems and the 2021 Magic Quadrant for Data Science and Machine Learning Platforms. Download the reports here. https://databricks.com/databricks-named-leader-by-gartner
Видео Geospatial Options in Apache Spark канала Databricks
About:
Databricks provides a unified data analytics platform, powered by Apache Spark™, that accelerates innovation by unifying data science, engineering and business.
Read more here: https://databricks.com/product/unifie...
Connect with us:
Website: https://databricks.com
Facebook: https://www.facebook.com/databricksinc
Twitter: https://twitter.com/databricks
LinkedIn: https://www.linkedin.com/company/data...Instagram: https://www.instagram.com/databricksinc/ Databricks is proud to announce that Gartner has named us a Leader in both the 2021 Magic Quadrant for Cloud Database Management Systems and the 2021 Magic Quadrant for Data Science and Machine Learning Platforms. Download the reports here. https://databricks.com/databricks-named-leader-by-gartner
Видео Geospatial Options in Apache Spark канала Databricks
Показать
Комментарии отсутствуют
Информация о видео
Другие видео канала
Low-Code Apache SparkComcast makes home entertainment accessible to everyone with voice, data and AINBA Analytics | Data Brew | Season 4 Episode 2Data+AI Summit 2022 HighlightsAccelerating the Pace of Autism Diagnosis with Machine Learning ModelsMagnet Shuffle Service: Push-based Shuffle at LinkedInSciplay enables real-time customer insights with the Databricks Data Intelligence PlatformDemo Video: Connect to Power BI Desktop from DatabricksRay and Its Growing EcosystemGain 3 Benefits with Delta SharingPower to the (SQL) People: Python UDFs in DBSQLAutomating Data Quality Processes at ReckittLLM Module 3 - Multi-stage Reasoning | 3.7.3 Notebook Demo Part 3Modern Architecture of a Cloud-Enabled Data and Analytics PlatformLLM Module 4: Fine-tuning and Evaluating LLMs | 4.13.2 Notebook Demo Part 2LLM Module 2 - Embeddings, Vector Databases, and Search | 2.7 SummaryProtecting PII/PHI Data in Data Lake via Column Level EncryptionState-of-the-Art Natural Language Processing with Apache Spark NLPRun Your Queries Instantly in One of the Most Optimized EnvironmentsGrab leverages data + AI to create economic opportunities in Southeast AsiaMoving to the Lakehouse: Fast & Efficient Ingestion with Auto Loader