Opening the INSPIRE conformance testing

Our CTO Ilkka Rinne was selected as the facilitator of the workshop on INSPIRE validation and conformity held in Ispra, Italy in early December 2014. In this post he describes some of the issues they were (and still are) working with, how he sees the future of the INSPIRE conformance testing based on Open Source code and community involvement.

lake_view_from_hotel_lido

A misty view from the Hotel Lido in Angera over the Lago Maggiore. At this time of year the breakfast was the time to take photos, after the workshop day it was already too dark.

INSPIRE conformance, what does that mean exactly?

EU INSPIRE directive is at the centre the multi-national activity for building a Europe-wide Spatial Data Infrastructure (SDI): a network of online Spatial Web Services providing spatial data collected by public authorities in all EU member states. The aim is to be able to find and fetch data from mineral resources to up-to-date weather conditions, and administrative borders to animal species distribution on-demand from directly from the authorities collecting them.

The INSPIRE directive mandates the public authorities to publish their data sets in harmonised formats using standard web services. The data users including other public authorities, research institutes, private companies and citizens are able to use same software products for viewing and downloading the INSPIRE data sets from any member state. According the INSPIRE vision it should be easy to compare these data sets and use them as up-to-date input data for all kinds of applications and decision making processes.

The seamless INSPIRE data-sharing vision does not turn into reality without formal regulation and technical guidance on the data content, metadata, formats and service interfaces to use within the technical INSPIRE infrastructure. Fortunately there are already good general data and service standards in the field of Spatial Data that can be adopted for the INSPIRE use provided by well-known standardisation organisations like ISO and the OGC. These standards provide a solid base for technical interoperability on which the INSPIRE SDI can be built on.

The INSPIRE specific requirements for datasets and services, as well as the metadata records describing them, are given in INSPIRE Implementing Rules (IR) and their corresponding Technical Guidance documents (TG). The IRs are published as part of the EU legal regulation (directives), to be implemented as national laws in the EU member states. The TGs are not legal text, but provide a help and working examples for technical solutions the member states can use to abide to the IR legal requirements.

An EU member state may conform to the INSPIRE regulation using one of the options outlined by the Technical Guidance documents or in any other means, provided that it can show that this alternative method still fulfils the IR legal requirements. The later approach is generally not recommended because it’s likely to cause technical interoperability problems between the data exchange between the member states. However, it would also be problematic to bind a particular technical solution to the inevitably slow process of the EU-wide legal regulation system: It’s a lot simpler to keep a technical document up-to-date if every change does not have to be ratified in the parliaments of 28 countries.

The downside of having two levels of requirements is that the term “INSPIRE conformance” is bit tricky to define: Does that mean that the IR (legal) rules are fulfilled? Or that the requirements in a particular Technical Guidance are fulfilled? These questions and harmonising the results existing INSPIRE validator software solutions used in different member states are in core focus of the INSPIRE Maintenance and Implementation Group (MIG) activity MIWP-5: Validation and conformity testing.

A workshop with a clear focus

One part of the MIWP-5 activities is analysing the requirements written down in the current TG documents and trying to formalise tests for validating services, datasets and metadata records against these requirements. I’m the facilitator of the small and active group of people currently working with this subtask. Last week on 2nd and 3rd December 2014 we had a two-day workshop on this matter kindly hosted by the JRC Institute for Environment and Sustainability which is part of the European Commission’s research facilities in Ispra, Italy.

From the total of 208 requirements in the Technical Guidance documents for INSPIRE Metadata, View and Download Services we chose to focus on the ones considering metadata, view service implemented using the OGC Web Map Service and the file-based download service option implemented using Atom feeds. Even with this limited focus we had almost 160 requirements on our plates – more than enough for two workshop days.

The main task as to define Abstract Test Suites (ATS) for each of the implementation options. These sets of natural language test case descriptions will then act as functional requirements for validation testing programs, or Executable Test Suites (ETS), which form the core of the validator software implementations for INSPIRE conformance testing. The Technical Guidance documents also contain a number of requirements for which it is very difficult, if not impossible to formulate a mechanical pass/fail test: Sometimes it takes a human expert or external knowledge to assert if the requirements are fulfilled or not. This is completely ok, but for making automatic validation tools it’s important to separate the two. Passing a mechanical validator can never guarantee a 100% conformance, but a good validator can help the work considerably by helping in spotting many common errors.

Schematic on relations of INSPIRE IR and TG requirements and the Abstract and Executable tests. CC stands for Conformance Class.

Schematic on relations of INSPIRE IR and TG requirements and the Abstract and Executable tests. “CC” stands for conformance class.

 

I’m glad to say that we made good progress with the work in these two days: We agreed on the format for the Abstract Tests, had very good discussions around the challenges in some of the TG requirements and the ways to work around them, and decided on the next steps and dead-lines for finishing the work.

Increased transparency with Open Source validation code

Validation rules and techniques used for evaluating the official INSPIRE conformance must the be transparent. If a validation fails for some input it must be possible to check how the validator code has been implemented. As in any software product there will be bugs and certain inputs that will result on false validation statements. In order to spot these weaknesses and make the validation code better in time, we strongly support the idea of publishing both the Abstract Test Suites and at least the officially endorsed validator implementations as Open Source projects.

I believe that the fact that code and it’s functional requirements are openly available and easily extendable, is a key to the INSPIRE conformance testing success. Only by being open and getting the developer community involved in the process can we get keep the testing codebase continuously up-to-date. By carefully choosing the licenses used we can create living and continuously improving shared validation codebase and still make it feasible for companies like us at Spatineo to create additional value for our users by focusing on thing like usability and integration with our other products.

As any successful Open Source project the INSPIRE validation projects also need clear governance and change management processes. It must be clear which versions, forks and branches are officially endorsed by the project leads. For INSPIRE the obvious authority for approving the ATS and validator library code changes would be the MIG. In practise the work of filtering and consolidating the code change proposals, preparing the changes for MIG approval, and creating the official releases would have to be done by some kind of expert board, the members preferably chosen amongst the top contributors but also having some technically oriented MIG representatives to guide the direction of the work.

As one result of the workshop we have now have set up public ATS repositories at Github under organisation inspire-eu-validation. While writing this the ATS drafting is still in it’s early stages, but the content will add up as we approach the February 2015 dead-line. The ATS texts are published in Public Domain using Creative Commons CC0 1.0 Universal license. The license used for the executable validation code has not yet been decided, but I’m pretty confident that it will allow the code to be used as a software library in both open source and closed source applications.

Spatineo will be contributing both design time and our expertise in the INSPIRE & OGC validation to these ATS and ETS projects. You may ask what is Spatineo getting out of participating in this work? The answer is that with Open Sourcing of the validation logic we can focus our development resources for keeping the usability of INSPIRE validation tools top-notch while still ensuring that we can always provide the officially endorsed validation results to our customers. Making Spatial Data Infrastructures “tick” is what we do at Spatineo. Being able to build more reliable tools for validation is an important part of that work.

INSPIRE Workshop: Practical Quality Assurance of Spatial Web Services

We’re doing a lot in in this year’s INSPIRE conference in Aalborg, Denmark. Our R&D Director Sampo Savolainen is talking about “Performance Testing of INSPIRE and OGC Services” in “Quality and Testing” parallel session at 9:15 on Thursday in “Room: 5 Radiosalen” and our Service Director Jaana Mäkelä about “Open Data and INSPIRE Web Services Are Available – Are Users Ready to Utilize Them?” in parallel session “INSPIRE Benefits Usage and Users” at 14:30 also on Thursday in “Room: 3 Latinestuen”.

We’re proud to be a bronze sponsor for the conference and also have a Spatineo stand in the conference centre during the actual conference days from Wednesday to Friday. Come visit and learn more about how we can supercharge your SDI.

See the conference venues on a map

My biggest personal effort for the conference this year is a workshop on Tuesday morning. You’re most welcome to join if you’re coming to the INSPIRE Conference 2014 in Aalborg.

Practical Quality Assurance of Spatial Web Services

INSPIRE Conference 17th June 2014 at 9:00 – 13:00 (two sessions),

Aalborg University, Badstuestræde 9/auditorium 1

Hosted by CTO Ilkka Rinne & Managing Director Kristian Jaakkola from Spatineo Inc.

aalborg legoland By flicrk user Alan Lam https://www.flickr.com/photos/alanandanders/

aalborg legoland By flicrk user Alan Lam https://www.flickr.com/photos/alanandanders/

Preliminary workshop schedule:

Session 1: INSPIRE Quality of Service

Presentation: Overview of the INSPIRE QoS requirements for Network Services

Group discussions, topics:

  • INSPIRE QoS requirements: Difficult or easy to achieve? Useful or not?
  • How to make improving QoS worthwhile and simple for the data providers?
  • Application developer perspective: What QoS indicators would be most useful for the data users? How they should be advertised for INSPIRE services?
  • Planning for QoS: How to integrate QoS measuring and improvements to a daily / monthly / yearly development plans & practices?

Demo: Evaluating the Spatial Web Service availability and continuous performance with Spatineo Monitor

Session 2: Hands-on Spatineo QoS Tools

Hands-on training: Spatineo Monitor

  • Finding and monitoring new services
  • Checking availability, alerts & maintenance breaks
  • Usage analytics & reporting

Live capacity testing session using Spatineo Performance

Workshop wrap-up & discussion

Update: the workshop slides are now available in Slideshare.

Kickstart for automating the INSPIRE Monitoring

What we typically mean by monitoring at Spatineo is quite different from the INSPIRE monitoring. Our monitoring is continuous technical surveillance of the spatial web services to evaluate their availability and responsiveness. In the INSPIRE sense monitoring usually means the yearly process of collecting and reporting a set of numerical indicators for following the progress of the INSPIRE directive in EU member states. Both activities are important for building a reliable European Spatial Data Infrastructure, and we’re happy to be able to help our customers in calculating some of the most laborious indictors such as the usage statistics by services and data sets.

Making the INSPIRE monitoring easier and more useful is one of problems to be tackled within the work of the INSPIRE Maintenance and Implementation Group (MIG), and specifically in it’s subgroup called MIWP-16 in which I’m one of the 30 members. The group had it’s first face-to-face meeting in a very nice Italian town Arona by the beautiful Lago Maggiore on Friday 11th April. In this blog post I’m giving you my personal view of the work we’re doing and how we could reach the goals set for the group’s work.

Carousel on the Arona town square

Carousel on the Arona town square

In many EU member states the collection data for the monitoring indicators requires a lot of manual work both for the government and local authorities. For example a great part of INSPIRE data providers currently lack the technical means for automatically creating precise usage statistics for the services they are providing. In many cases calculating these yearly figures takes a considerable amount of time, and with the decreasing governmental organisation budgets, it’s in many cases considered infeasible altogether. The obviously results in decrease in the quality and coverage of submitted indicator data, which in turn limits the usability of this data in further analysis and strategic planning at EU level.

The main objective of the MIG work package MIWP-16 is to reduce the amount of manual work for collecting, analysing and submitting the monitoring indicator data required for the INSPIRE monitoring reports sent by each EU member state to the European Commission each year. These indicators are designed to reflecting the progress and successfulness of INSPIRE implementation in EU member states, such as the numbers of available INSPIRE compliant data sets, web services for accessing and usage statistics for those services and data sets. These indicators have been listed in the INSPIRE Implementation Rule for Monitoring and Reporting, and thus each member state is legally mandated to collect them. In addition to streamlining the data collection process, a web dashboard tool with the outlook for the collected monitoring indicators and their yearly trends is also in the to-do list for this work package.

Scene along the shoreline towards the Arona town square

Scene along the Lago Maggiore shoreline towards the Arona town square

The working group of voluntaries consist of both people working directly with the INSPIRE monitoring in the member states and other INSPIRE experts interested in the monitoring and reporting. We are tasked to come up with ideas and design of technical tools for making the monitoring indicator collection more automatic for the member states. One of the essential ideas is to reuse the information already available in the so called discovery services containing machine-readable metadata records for each the INSPIRE data sets and services. As each member state is mandated to provide and maintain these metadata web services, it’s a natural idea to use the information there as an input in automating the calculation of the monitoring indicators.

For some indicators such as the amounts and names of the available INSPIRE data sets and services the process of information extraction is relatively simple, because of the standards-based querying capabilities of these services. The Catalog Service for Web (CSW) interface standardised by the Open Geospatial Consortium (OGC) allows filtering of the returned metadata records based on the properties they contain, such as the type the the described (data set or service) and the declared compliance of those resources against the INSPIRE regulations. It should be noted that the metadata-extracted numbers can only show numbers for the resource that have INSPIRE-compliant metadata records available via a CSW interface. This is not yet the case for all available INSPIRE data sets and services in all member states. For other indicators, such as the usage statistics, this approach cannot be used, as the information for calculating them is not available in the metadata records.

Yes, we did some work too...

Yes, we did some work too…

As of mid-April 2014 our process of designing the mentioned automating tools is still in it’s very early phases. As I mentioned before, this meeting in Arona was the first face-to-face meeting of the MIWP-16 group after a series if bi-monthly web/teleconferences since setting up the group in late December 2013. The group has executed a web-based questionnaire targeted to the people working with the INSPIRE monitoring and reporting on the national level. The received answers from the 14 member states provide a good guidance for the perceived importance for the different indicators from the member state’s perspective and their desires in having those indicators included in the monitoring dashboard. After this meeting we are in the phase of drafting the functionality of the dashboard and the technical architecture of the data collection system providing the data for the dashboard application.

Flying over the Alps

Flying over the Alps

During my flights home over the snow covered Alps and cloudy Central-Europe I draw a high-level draft of a diagram for the information processing system required for the monitoring indicator automation tool based on the group discussion. The system should allow complementing the information retrieved from the metadata records from external sources, like dataset/service validators, usage statistics calculators etc. One option for providing this information would be an authenticated API with a reasonable simple, general enough data model that these external systems could use to submit the complimentary data. The automatically calculated monitoring indicators would then be reviewed by the reporting authorities and manually corrected if necessary. The output of the tool could be published in the monitoring dashboard and/or exported as the pre-filled spreadsheet or XML document to be used in the official INSPIRE monitoring.

Overview of the indicator automation process

It seems to be commonly agreed that the resulting software should not be created having only the EU level monitoring in mind. It should be possible to install the tool set also by member state authorities for helping the INSPIRE monitoring at national and local levels. Technically this would mean ability to connect to any CSW service for extracting INSPIRE metadata records, not just the INSPIRE Geoportal. It would also be natural to build the system flexible enough to allow adding and changing the set of indicators for each reporting period to fit the slightly diversifying needs of monitoring at all levels for years to come. Licensing the software as Open Source would also seem to me like a good fit for making the continuous non-centralised development possible. I should remind the readers that these ideas at least at this stage are my own, and not a commonly agreed position of the MIWP-16 group.

Coming from the background of user software design, I must admit that the goals of the work package seem quite ambitious, especially considering the given time frame (a prototype of the dashboard by the INSPIRE conference in June 2014, about 2 months from now, and the final results by the end of the year). It does not exactly help that the most of the working group members are able to dedicate only a fraction of the work time for the project and no other resources (for example for UI design and implementation) are currently foreseen. Personally I’m already quite occupied with making the first release of Spatineo Performance happen before summer as well as in helping Finnish Meteorological Institute in releasing more interesting meteorological information as open data. Nevertheless I’m confident that with good project leadership it’s possible to achieve good results in drafting the required functionalities based on the most important use cases for the dashboard and the report automation even with the limited time and resources.

Borromea castle in Angera as seen from Arona side in the lake

Borromea castle in Angera as seen from Arona side in the lake

Following the principles of openness and transparency of the INSPIRE maintenance and implementation work, the workspace of the MIWP-16 group is publicly available as a Redmine site hosted by the JRC. Most of the group material, including the accepted minutes of the web and face-to-face meetings are provided for anyone. Internal discussion of the group members and tasking is kept private to make the internal communication as efficient as possible.

If you have any comments or ideas about this work, please comment on this post or send me an email. I’d be glad to pass them on to the working group members for discussion.

Further information: