Big Data In The Browser

So why would anyone want to put alot of data into a browser? Well, for a lot of the same reasons that edge computing and distributed computing have become so popular.

You get the data a lot closer to the user and you don’t have to pay for the compute ;) 

… this sounds great but as I found out during this conversation it's not as easy as it might seem! 

There are a lot of trade-offs that need to be evaluated when moving data and analytics to the client.

 

Nick Rabinowitz  Senior Staff Software Engineer at Foursquare has a ton of experience with this so he volunteered his time to help us understand more about it.

https://location.foursquare.com/

https://studio.foursquare.com/home

If you are not familiar with the Arrow data format it might be worth checking out

 

Apache Arrow defines a language-independent columnar memory format for flat and hierarchical data, organized for efficient analytic operations on modern hardware like CPUs and GPUs. The Arrow memory format also supports zero-copy reads for lightning-fast data access without serialization overhead

 

Related podcast episodes that you might find interesting include

H3 grid system

https://mapscaping.com/podcast/h3-geospatial-indexing-system/

The H3 geospatial indexing system is a discrete global grid system consisting of a multi-precision hexagonal tiling of the sphere with hierarchical indexes. H3 is a really interesting approach to tiling data that was developed by UBER and has been open-sourced. 

Hex Tiles

https://mapscaping.com/podcast/hex-tiles/

If you have not heard of the H3 grid system before listen to that episode first before listening to this one it will add a lot of useful context!

Spatial Knowledge Graphs

https://mapscaping.com/podcast/spatial-knowledge-graphs/

Foursquare is moving away from spatial joins and focusing on building a knowledge graph. If you are not familiar with graphs this might be a good place to start, also its interesting to hear the reasons for the move from spatial joins to another data structure.

 

Distribution Geospatial Data

https://mapscaping.com/podcast/distributing-geospatial-data/

This is interesting if you want to understand more about distributed databases and some of the strategies for doing this. It sounds complicated but this episode is a really good introduction! 

 

Cloud Native Geospatial

https://mapscaping.com/podcast/cloud-native-geospatial/

This episode give a solid overview of what cloud-native means and some of the current geospatial cloud native formats out there today

 

I am constantly thinking about how I can make this podcast better for you so if you have any ideas or suggestions please let me know! 

Also, I am thinking of recording a behind-the-scenes episode, is that something you might be interested in? if so what questions do you have? 

 

Some more episodes you might enjoy

 

ESRI, GIS careers, Geospatial Data Science 

QGIS, Geospatial Python, ArcGIS Pro

Google Maps, Geomatics, Cartography

Location Intelligence, Mapping