How to utilize STAC to load satellite images faster in web applications?

How to utilize STAC (SpatioTemporal Asset Catalog) to load satellite images faster in web application(2)

Spatineo was given an opportunity to participate in a project for Finnish Meteorological Institute (FMI) aiming to build a prototype catalog for Sentinel satellite images: FMI Sentinel catalog. The idea was to build a web based catalogue that would allow efficient and fast querying and downloading satellite images. In this case, the imagery is stored an AWS S3 compatible service. 

The catalogue was built using Radian Earth Foundation’s STAC (SpatioTemporal Asset Catalog) specification. STAC contains specification, which aims to “increase the interoperability of searching for satellite imagery” and other geospatial assets. STAC allows data providers to produce a simple machine readable catalogues of imagery in a flexible and light data format. Data providers are able to add additional metadata and additional properties to the assets as they wish.

STAC implementation is formed of a STAC catalog and STAC items. A STAC item is a GeoJSON Feature containing additional fields including links to the assets. STAC items contains the geometry of the asset and can be used for example for selecting the correct satellite image from a map. STAC catalog is simple json that contains links to child STAC catalogs and/or STAC items. There are also STAC collections, which can be used to describe for example common properties of items without having to repeat the metadata on every item.

FMI Sentinel Catalogues Spatineo
FMI Sentinel Catalogue in action

In its simplest form, a STAC implementation can be a static catalog, which is basically set of json files on a server that link to one another and are therefore crawlable. The assets themselves can be on the same server or in another location, such as AWS S3 bucket or other HTTP server. The more tuned version is a catalog API, which is a RESTful API that is designed to mirror the static catalog. 

You can read more about STAC specification here.

In this project the catalog was developed to follow STAC version 0.7.0 and it was built to be static catalog. In addition to specification, child link fields contain additional field: dimension. Dimension is used to specify whether the child catalog specifies certain geographic location (geohash) or date (time). This makes it easy for applications to understand the structure of the catalogue and to find sub-catalogues that contain items for the current view and time span.

FMI Sentinel catalog contains catalogs for Sentinel satellite images processed by FMI.  The somewhat simplified hierarchy of the catalog is:

root.json – root STAC catalog
  • dataset-S1.json – catalog for S1 images
      • dataset-S1-location-ug.json – catalog for S1 images in geohash ug
          • dataset-S1-location-ug-time-2017-08-01.json – catalog for S1 images for geohash ug and date 2017-08-01
              • 1_processed_20170801_15[…].json – STAC item for image
                  • VH-asset.tif – Cloud optimized GeoTIFF asset (VH polarisation)
                  • VV-asset.tif – Cloud optimized GeoTIFF asset (VV polarisation)
      • dataset-S1-location-uu.json
    • dataset-S3.json

In order to allow fast loading of remote sensing images in a web map framework, such as OpenLayers, the regular GeoTIFF images were processed to be cloud optimized GeoTIFFs (COGs). COG takes advantage of GeoTIFFs ability to organize pixels in particular ways and HTTP GET range requests that allow data to fetched within certain byte ranges. This fast loading of image in a certain zoom level and geographic extent.

    • With GDAL version above 3.1, you can do it whit
      gdalwarp src1.tif src2.tif out.tif -of COG
    • With lower GDAL versions, use
      gdal_translate src.tif out.tif -co TILED=YES -co COPY_SRC_OVERVIEWS=YES -co COMPRESS=LZW

If you are unsure whether your geoTIFF is cloud optimized or not, use this method to find out

By combining the simplicity and crawlability of STAC and speed of COGs it is possible to build easy-to-use (and -maintain) view and download services for satellite images. Go ahead and test FMI Sentinel catalog and other implementations of STAC catalogs. The code that produces the catalogue and the map application are also open source and available of github. In case you have further questions, don’t hesitate to shoot us a comment or an email!

Want to stay ahead of the game, and get latest news from us? Subscribe now to our newsletter!

GIS Expo: Laying the foundations for Interoperable Geospatial data in Finland

Finnish GIS Expo has been thriving for 31 years now, and the event still brings the local geospatial community together annually. Climate Change and Interoperability were the big topics this year and Spatineo had a special part in this years expo!

Laying the foundations for Interoperable Geospatial

As most people in Finland already know, the geospatial sector has been shifting towards more interoperable dataflow and usage within the last few years. Finnish Geospatial Platform and Geospatial Policy Report (Paikkatietopoliittinen selonteko) have been few of the drivers towards effective usage of geospatial data. Several different presentations were held on these two, such as Sakari Jäppinen’s “Towards Digital Decision Making in Land Usage” and “Work together or Die” presentation from Antti Jakobsson.You can see some of the recordings from here.

We were also on the stage several times during the conference. First our very own Riitta Vaniala had a presentation about the Impact of geospatial data. The topic has been hot lately, and we have been working quite hard here at Spatineo to showcase the impact of open data. You can read and download the Impact whitepapers, of which Riitta’s presentation was based upon, from our Impact page.

Riitta is also hosting a webinar on the topic of Impact Assessment. You can register to the webinar for free.

Besides Riitta presenting at GIS Expo, Jaana Mäkelä took a strong stand at moderating panel discussions.

She was the main moderator in two major panel discussions. First being the “Future of Geospatial industry” on monday. Participants were Teemu Virtanen from Sitowise, Ville Alasalmi from Aalto University and Researcher Pyry Kettunen from NLS of Finland . Panel recording can be viewed from here.

One of the most important program numbers was Geospatial Grill, in which Jaana was the moderator and valued guests were literally grilled on the premise of how they are going to make geospatial data better and more available. 

The panelists were Antti Vertanen from the Ministry of Agriculture and Forestry, Sami Suomalainen from Emergency Response Centre Agency of Finland, Päivi Nerg from Ministry of Finance, Tommi Karttaavi from Association of Finnish Local and Regional Authorities, and Vesa Ilmarinen from Platform of Trust. 

We will not go down to the details on the discussions here, but summarizing everyone on the panel agreed that data should be even more available, and easily reachable. Private companies must be backing up the public sector, and innovation has to be formed in both private and public organisations.

One of the most delightful things in GIS Expo was seeing how people meet their old colleagues and friends. Most people have worked in several companies within the industry, and cooperation between organisations is common.

Huge thanks to everyone who made GIS Expo possible this year, and shoutout to everyone who visited our stand! We had great conversations with people there, and in case some of you want to keep those discussions going, don’t hesitate to contact us at your convenience.

Want to stay ahead of the game, and get latest news from us? Subscribe now to our newsletter!

Inspire Helsinki 2019 was a success – what is the future of data challenges?

Inspire Helsinki 2019 Epicenter

After last year’s INSPIRE conference in Antwerp, a lot of the regular visitors to the conference asked themselves where will the next INSPIRE conference be held. The answer turned out to be Helsinki, but it was not going to be the typical yearly affair. Inspire Helsinki 2019 was a change to the long running tradition of yearly INSPIRE conferences. This time the focus was more on technology and data users and based on the feedback, the event was well received. So what was different and what did we learn from Inspire Helsinki 2019?

Data Challenges and technological approach

The event was organised by the National Land Survey of Finland, Ministry of Agriculture and Forestry along with the Joint Research Centre of the European Commission. Spatineo was part of the local organisation committee and lead the organisation of the data challenges. 

Usually topics and presentations in INSPIRE conferences have been about policy related matters, information and updates on technical guidance, and news about progress in implementation. While technology has always been a part of the conferences, this time it was decided the event should focus on both technology and data users.

The event hosted a number of workshops and data challenges. Challenges were hosted by our challenge partners who designed challenges where data was to be used to solve a problem or help people in their day to day lives. There were four challenges ranging from testing INSPIRE infrastructure to improving day to day commuting:

More information and the webinars related to the challenges can be found from the links above.

Why Data Challenges?

INSPIRE Conferences have been hosted by and for the people working on INSPIRE policy and implementation. This format has worked well, and was especially important in the early days of INSPIRE implementation, but as more and more data has been published, more and more people have been asking about presentations on how the data is actually used. In the Inspire Helsinki 2019 event we specifically set  out to address this issue.

Five teams who had produced submissions to the challenges presented at the event and they showed their solutions. Their works ranged from identifying landslide hazards to bike route planning while avoiding bad weather to analyzing best vacation areas using machine learning methods. You can read more about all five submissions on the challenge website.

The room where the teams presented their solutions was packed with standing room only and feedback from the audience sent a clear message that there is a need in the community to reach out and hear back from developers using open data. I feel there are two important factors here: 1) seeing how data is used is a real motivator for service providers – it is difficult to get excited about opening data if you don’t see how that work can impact society and people, 2) the developers using data benefit from a connection to the service providers – being able to ask questions and provide feedback, to have a dialogue between the provider and user, is crucial for developing a true ecosystem.

The winners were announced on the last day of the event and JRC sponsors one member of each team who presented at the event to attend the Dubrovnik INSPIRE Conference in 2020!

From Helsinki to Dubrovnik

As the dust settles in Helsinki, it is a good time to start thinking about how we can as a community, to make future INSPIRE Conferences even better. INSPIRE, while not complete, is ready for use and as such, should reach out to data users and developers. This requires dialogue: meeting people and getting the word out of the confines of the comfortable INSPIRE implementation bubble. The Helsinki Data Challenges were well received and I feel we were able to start the dialogue between data providers and users. We really hope that this good spirit and communication is carried over and continued in upcoming INSPIRE conferences as well!

Spatineo will be once again attending the upcoming event in Dubrovnik, and we’re thrilled to see how INSPIRE events can evolve over time towards a better more inclusive ecosystem.