Spatineo Service Map visualises Spatial Web Service growth across Europe

Spatineo has launched a freely accessible visualisation of the availability of spatial web services organized by country and administrative area. This tool provides never before seen insight into the development of the European Spatial Data Infrastructure (SDI) and INSPIRE, particularly how the scope and availability of these services have developed over time. Spatineo offers SaaS tools, such as Spatineo Monitor, for the analytics and quality assurance of spatial web services.

Service Availability is vital for SDIs

The vision and goal of the INSPIRE legislation is to simultaneously open more data and increase its use. We at Spatineo believe it is crucial to show that organisations are working hard to fulfil their obligations. This transparency is necessary to inspire the private sector to discover and trust the spatial web services that can enable companies to both innovate and build new businesses that utilise the open spatial data.

Accurate and timely information on service quality lets data providers react faster and steer development resources more efficiently. Spatineo has helped numerous organisations improve the quality of their services by providing them with indispensable information on both what the actual service level of their services is and what factors influence their services availability. Until now, no one has even attempted to assess this on the SDI level. For instance, the advancement of INSPIRE has thus far only been tracked through manually compiled annual reports, which have not been publicly visualised or aggregated to form a big picture.

The technical quality of a web service is most often expressed in terms of its availability – generally, the percentage of time a specific service has been operational. Clients and suppliers often use availability as a benchmark in service level agreements, and European INSPIRE legislation compliance requires a level of 99% availability. There are many other factors underlying the quality of a spatial web service, such as spatial accuracy, comprehensiveness and consistency, but if a given service is inaccessible, no other qualitative measures apply since the data is altogether unavailable to users. Moreover, availability can also be compared across a wide range of different service types and classes; therefore it is ideal for evaluating a wide variety of services at once.

Collecting the data

Spatineo monitors tens of thousands of spatial web services globally. Quality assurance is the heart of our business and we pride ourselves in being a trustworthy neutral party who provides unbiased results. We have also participated in the development of both Open Geospatial Consortium (OGC) specifications and INSPIRE technical guidance documents. This puts us in the unique position of being able to build a tool that efficiently visualises all the hard work of the GIS community in Europe.

Spatineo harvests available spatial web services from service catalogues and search engines to keep its registry up to date. For the purposes of the map, services are broadly defined as any service endpoint that is described by a single service description document of a particular service type. For example, each WMS Capabilities document describes a single service. All services within our catalogue are continuously monitored. This monitoring procedure is compliant with the INSPIRE normalized testing procedure for availability and has provided us with data spanning back to 2012. To construct the map, availability results for each service are continuously tested month-by-month against the 99% availability threshold (not counting pre-announced maintenance windows) consistent with INSPIRE requirements.

Our service registry is enriched by additional information on the organisations providing these services. Service availability monitoring data and organisation information is then aggregated based on the sphere of influence of each organisation to provide summaries for each country.

Map visualisation

The Service Map is accessible through an easy to use website (see http://servicemap.spatineo.com/) that lets users view both the current status of European SDI development and overall trends. Users can examine Europe as a whole or focus on specific countries to compare different administrative areas.

The shading of each country indicates the percentage of high-availability services while the bars on the bottom show the number of both total services and high-availability services over time. Users can choose to inspect either the current month or any previous month.

The shading of each country indicates the percentage of high-availability services while the bars on the bottom show the number of both total services and high-availability services over time. Users can choose to inspect either the current month or any previous month.

 

Not only does this tool aggregate data by geographic areas, it also provides insight into the top spatial data providers in Europe. Top organisations are listed according to their average service availability and the number of services they each provide. Additionally, data providers that have recently increased the availability or quantity of their services are highlighted.

The high level SDI status data shown on the map immediately reveals that there is healthy growth in this sector. The quantity of spatial web services has continued to grow steadily over the past two years and the number of compliant services available is very promising. We highly recommend that you visit servicemap.spatineo.com to explore the map yourself. The data used for our visualisation is constantly improved as new organisations and services are identified.

The technology behind this map can also be applied to more specialized cases. For instance the European Location Framework project (ELF) uses another map based on the same software. Their project amasses services from multiple countries and multiple service providers into a single harmonized map interface to guide decision making, drive eServices development and empower citizens. Our Service Map provides a high level view on the services involved and helps assess the strategic success of their project.

Moving forward

We have not yet seen wide adoption of the spatial web services provided by the public sector. Achieving widespread use requires not only good quality data and services, but also that the existence of these services be communicated and advertised to companies and private citizens. Our Service Map promotes openness, which should increase public scrutiny of current service quality.

It is also important to remember that the whole is the sum of its parts. The Service Map is merely a visualisation and the quality of the actual services is key. For the actual service quality to improve, data providers should look for tools to monitor and analyse the quality of their services, tools such as Spatineo Monitor.

Monitoring interruption on Apr 23rd 2015

Our monitoring network experienced a service interruption on Apr 23rd 2015 from 9:20 to 10:40 UTC. No monitoring data was collected during this time. During this interruption Spatineo Monitor and Spatineo Performance were accessible and usable, but users were unable to configure services or meters and it was not possible to add new services for monitoring. This interruption was caused by malfunction in crucial services provided to us by Amazon Web Services. We noticed this malfunction right as it begun and reacted to the developing situation as quickly as possible. Monitoring was resumed immediately after the services were fixed.

Monitoring is now operating nominally and we are keeping a close eye on the situation. We apologize for any inconvenience caused by this interruption.

Identifying requests from Spatineo

Spatineo monitors a large amount of web services, and most of these services are publicly available over the internet. Monitoring works in a completely automated fashion in typical cases. In some cases though, our customers are interested in identifying which requests originate from our monitoring. For example some services may require authorization using HTTP authentication, User-Agent filtering or IP address white lists.

HTTP authentication

HTTP Basic Authentication is widely used in OGC as well as with other web service protocols. As the name says, it is very basic and it is not ideal or the most secure authentication method available, but it is supported by almost all HTTP clients and servers and, despite the valid criticism, it still has its uses. If you combine Basic Authentication with a properly secured HTTPS transport, it is still a relatively good authentication method. Even without HTTPS, it does give the service provider more control over their service for example, to block offenders if irregularities arise.

Users of Spatineo Monitor may set up the basic authentication parameters used when we send requests to their service via service parameter configuration. These parameters are used for retrieving service descriptions, monitoring requests and performance tests.

In addition to HTTP Basic Authentication, our customers can configure extra query parameters or HTTP request headers that will be sent on every request.

User Agent

All monitoring requests use the HTTP User-Agent request header to tell the server that the request is coming from Spatineo. This is the most precise way of knowing which requests are sent by us. For monitoring requests, the User Agent also contains a link to the Spatineo Directory page of the monitored service. Please keep in mind however that the User-Agent identifiers we use are well known and as such, are unreliable for authentication.

The User-Agent for querying for new services or updating previously known services is:

Spatineo Monitor Controller (http://www.spatineo.com/spatineo-monitor/)

The following User-Agent is used for monitoring requests (where NNN is the ID for this particular service):

Spatineo Monitor GetMapBot (http://www.spatineo.com/spatineo-monitor; MonitoredService http://directory.spatineo.com/service/NNN)

Spatineo Performance uses this User-Agent for requests sent during performance tests:

Spatineo Performance (http://www.spatineo.com/spatineo-performance/)

User-Agent information is also helpful if you wish to allow or block monitoring from us by using the robots.txt mechanism.

Source IP

The Spatineo monitoring network is comprised of multiple servers running in multiple data centers. Monitoring requests, performance tests and service discovery all come from different IP addresses. We keep an up to date list of the IP addresses used by monitoring and service discovery which is available at http://directory.spatineo.com/public/monitoring-ip-list.txt. The list is automatically updated when IP addresses are added or removed from the list.

Please keep in mind that this list does not include the IP addresses that Spatineo Performance tests originate from.