Access to Cloud Raster Data Using GDAL, MRF and LERC

Abstract:

One of the main obstacles to a fully enabled web GIS platform is the access to raster data. This is because the rasters used in GIS tend to be much larger than the common image rasters. Compressed raster formats that support tiling, as well as web access to tiled data are common technologies that address these issues. The NASA originated open source OnEarth tile server and the Meta Raster Format (MRF) are steps in the right direction, a basis for both local and web based rasters. Esri has taken this technology even further, making MRF a low level raster caching format and adding a very fast data compression mechanism called Limited Error Raster Compression (LERC). The on-demand Landsat 8 and NAIP mosaic services have served as real world implementation prototypes and have shown the value of this approach to the GIS community. While continuously in service for more than two years, the Landsat 8 service has been re-implemented multiple times, adapting to the changing cloudscape. The initial architecture converted every incoming scene from GeoTiff to MRF with LERC, reducing the average response time from about eight seconds to less than two. Recently, it was re-implemented to use the Landsat on Amazon Web Services (AWS), and to use MRF with LERC as a local dynamic cache. Recently, a python application called OptimizeRasters has been developed, which can convert and copy the data to and from cloud storage, while at the same time applying these technologies.