A talk by Ayodeji Ogunlami and Luke Menzies from Advancing Analytics.
In the rapidly evolving field of Geographic Information Systems (GIS), the integration of Databricks offers groundbreaking opportunities for processing and analysing spatial data at scale. This talk aims to explore the innovative application of Databricks in GIS, highlighting its capabilities to perform geospatial analysis, data integration, and real-time processing in a distributed manner.
We begin by introducing the fundamentals of GIS and the unique challenges presented by geospatial data, such as its complexity and voluminous nature. Traditional GIS tools like ESRI ArcGIS have long been at the forefront of managing and analysing spatial data. However, the advent of big data and the need for more scalable solutions have led to the exploration of new platforms such as Databricks. We shall also introduce the concept of the H3 index, a unique grid-based approach that overcomes many of the issues encountered with traditional geospatial operations. The talk will then delve into how Databricks, a cloud-based big data analytics platform, addresses these challenges using a library known a Mosaic. Key features of Mosaic will be discussed showing how they can be used for advanced analytics, and machine learning capabilities within Databricks.
Furthermore, the talk will cover the technical aspects of integrating GIS with Databricks. This includes discussions on data formats, libraries for GIS and the use of APIs for connecting GIS software with Databricks environments.
This talk is aimed at beginner data engineers, data scientists, and anyone interested in the intersection of geospatial technology and big data analytics. No prior experience with Databricks is required, but a basic understanding of GIS concepts will be beneficial.
Technical Level: Technical practitioner
This session was part of the Data Science Festival MayDay event 2024. Find out more at datasciencefestival.com/event...
The Data Science Festival is the place for data-driven people to come together, share cutting-edge ideas, and solve real-world problems. We run monthly events, meet-ups, and the biggest free-to-attend data festivals in the UK. Join the community at datasciencefestival.com/