INSPIRE validation on the right track

Wow, the time really flies: it’s been over 1.5 years since my last post about the INSPIRE validation and conformance testing in December 2014! Since then the INSPIRE MIG subgroup MIWP-5 has worked hard to get the Abstract Test Suites for INSPIRE Network Services defined and the requirements listed for the commonly agreed INSPIRE validator reference implementation. In the August 2016 the first fruits of this work will be ready for picking as the first release of the INSPIRE Test Framework open source project is scheduled to released.

INSPIRE validation workshop

INSPIRE validation workshop

I recently attended to a validation workshop of the INSPIRE Knowledge Exchange Network (INSPIRE KEN) in IGN’s school of Geomatics (ENSG), Marne-la-Vallee, France. The idea of the workshop was to bring together people working with the INSPIRE specifications, validation tools and data providers, and to share experiences and ideas about INSPIRE metadata, data and service validation. The workshop was a success with more than 60 participants from 38 organisations in 17 European countries. If you missed the workshop, the slides and videos of the 28 presentations are available at the workshop website, including my presentation about Spatineo tools (video of the presentation in YouTube).

Test Framework for consistent INSPIRE validation

One of the most interesting topics was the status and plans of the INSPIRE Test Framework project. The main goal of this project is to create an open source reference application to verify the compliance of the tested objects against the requirements of the INSPIRE Technical Guidance documents. This work is part of the JRC ARE3NA project and continues until mid 2017. PwC and Interactive Instruments have been contracted to plan and run this software project, which makes me pretty confident about the results.

The main outcome of this work is a reusable, open source software package capable of executing reference implementations of the validation tests for INSPIRE metadata, datasets and services. This software will be hosted by the JRC as a web application, but it will also be possible to install it locally and to integrate the tests in the internal data provision workflows. The project will also define solid data models for the validation testing concepts, enabling persistent test data storage and remote execution and test data exchange using APIs. Needless to say this sounds very promising: A common, well-governed test codebase and a solid software framework is a key in helping the INSPIRE data and service providers to reach the interoperability goals of INSPIRE with considerably less effort.

inspire-test-framework_technical-context-view

The initial design of the Test Framework as presented by Clemens Portele of Interactive Instruments at the workshop assumes that the existing ETF tool will be used as the base of the test framework including service testing capabilities using SoapUI and large XML file validation leveraging BaseX. Some of the key extensions for ELF include a properly documented and comprehensive API, multilingual test reports, and test driver for TeamEngine enabling reuse of the existing CITE conformance tests for the OGC standards.

At Spatineo we have been actively involved with the INSPIRE validation work as members of the MIWP-5 group from the beginning. We are very excited to start integrating the Test Framework into Spatineo Monitor to complement our INSPIRE Network Service testing capabilities, which already cover the availability, performance and capacity testing as well as service metadata validation for WMS and WMTS services. I also encourage everyone interesting in this work to take a look at the Design Report of the INSPIRE Test Framework v0.3 containing the initial design and commenting to the authors of something does not look right.

IORL0495

Validate early, provide user-friendly results

At the end of the INSPIRE KEN workshop, Michael Lutz from JRC emphasised the importance of using the validation tools already quite early in the work data provision and service publishing processes: If the validation is only done as the final acceptance test step, this probably results in both extended initial release time and double work as the found errors need to fixed. Other concluding point was the importance of the usability of the validation results: clear error messages, and instructions on how to fixed the particular errors.

No matter how good the INSPIRE validation tools are, using them is not enough to create a successful European Spatial Data Infrastructure. The strength of a good validation tool is guiding the data and service providers to avoid decisions endangering interoperability, and doing that as early as possible in order to save time for more important things.

IORL0464

Helsinki Region Transport hackathon’s winning project targets special needs group

Helsinki Region Transport (HSL) has recently organized a hackathon for further improving their Journey Planner. The hackathon featured 9 teams with very different approaches to solving the given task. The winning project targets customers from different special needs groups (e.g. visually impaired), as well as children and the elderly.

The hackathon was supported by HSL’s partners: Liikennevirasto, Helsinki Think Company, CGI, Netlight, Nitor, Palmu and Visma. We’ve visited the hackathon and interviewed some of the participants about their projects and the problems they were trying to solve with their ideas. The event was positively evaluated by all the interviewed participants. They claimed that the organizers were very helpful and the teams enjoyed spending time together and working on their project in a friendly and dynamic environment.

DSC_0343

Team “STOP 2.0”

Henna Kalliokoski, a member of the winning team “STOP 2.0” describes their idea: “The main purpose of “STOP 2.0” is to allow the traveler to inform the driver about their wish to stop the bus. The application uses small iBeacon Bluetooth transmitters for location. Both the buses and the stops have iBeacon transmitters, so the user is able to identify the beacon and ask the driver to stop at the right place: either from the bus stop for the approaching vehicle or from inside the bus. “STOP 2.0” makes use of the new driver’s devices of Helsinki Region Transport. The driver’s device is a computer that supplies real-time information about the traffic and locations of the other vehicles on the same route. “STOP 2.0” is integrated into the driver’s device to notify the driver about the need to stop at a certain bus stop.

The idea came from one of our team members Ronja Oja. We all wanted to do something that actually benefits people, and she had this problem with public transportation because she is blind. We also considered a couple of other ideas, but this one provided the most additional value for the effort and investment it would require. The existing public transport doesn’t pay enough attention to different special needs groups such as the visually impaired, children or the elderly. We want Helsinki to be the city where the people can use public transport smoothly regardless of age or capability.”
Team members: Pihla Toivanen, Tintti Rahikainen, Ronja Oja and Jenny Tyrväinen. All the team members are computer science students living in the capital area.

Team "Gaidi"

Team “Gaidi”

Miranda Kastemaa from team “Gaidi” presents their fun experiment: “Our project is an experiment in generative art that transforms real-time tram location data into a spooky audiovisual experience. Trams are represented by skeletons jumping around in a desolate world, consisting of drum machine pads that trigger different sounds. Being passionate about graphics and audio programming, it seemed like the obvious thing to do when I heard about the real-time location API offered by HSL. We started out with some basic wireframe cubes moving around, and the finer details of the concept emerged over the weekend. The skeleton theme may have been inspired by Skull Trumpet and Dark Souls.

We wanted to bring something a bit different to the event with a more aesthetics-oriented project. Judging from the response we got, I think we succeeded!”

Team members: Miranda Kastemaa, software engineer and electronic musician, and Wolf Wikgren, master’s student at Aalto ARTS and web developer.

Team "Just in Time!"

Team “Just in Time!”

Team “Just in time!” created an app, aimed at saving commute time for the users. Rashmi Kasat tells: “Our app uses traveler’s personal history data of commute (usual starting point, usual destination, usual schedule etc.) and combines it with the real-time transport data to create personalized alerts for the user. It notifies the user of the precise time they need to leave for their stop, so that they are at the stop just in time (not too early and not too late). The user is also notified if he/she misses the bus, or there are disruptions on the desired route and proposes the time they should leave for the next bus.

The idea originated from the fact that several times I have had to spend a long time waiting for the bus when I missed the bus by a couple of minutes, sometimes even seconds, or when the bus is cancelled or delayed. In the winters particularly this is very tiring. I wanted to have this kind of notifications personalized to my behavioral pattern taking into account e.g. number of steps from starting point until the stop, wait time in the elevators, some other time delay etc. I also wanted to have a notification if I had a high likelihood to miss the bus based on the lead time calculated from above data.”

Team members: Rashmi Kasat, Director and Head of Digital Services Practice at Capgemni, and Mika Majakorpi. Senior Software Architect at Nitor.

Team "Tram Challenge"

Team “Tram Challenge”

Team “Tram Challenge” created a fun project to help citizens explore the city. Stephen Sykes explains: “The goal of the Tram Challenge is to visit all the tram stops on the Helsinki tram system as quickly as possible. It was inspired by the London Tube Challenge, and also the desire to get people exploring Helsinki and making full use of public transport. We thought it would make a fun day out this summer. You can register and time your attempt, get tram line and stop information, and you can track the trams as they move around so you can optimize your route. Using the TramChallenge app will be the only way to get onto the official leaderboard.”

Team members: Stephen Sykes, Matias Korhonen, Joao Cardoso, Krister Kari.

Water Map Popularity in Finland Proved the Importance of Public Maps’ Performance Testing

Finnish Environment Institute (SYKE) has recently launched a very handy map of water quality in Finland, which allows anyone to check the ecological status of Finnish lakes and rivers.  After a news article featuring the online water map was published at the very popular news website YLE, the Finnish Broadcasting Company on 1st of May 2016, the map gained unprecedented popularity up to 85 000 visitors per day wanted to see whether the lake by their summer cottage is ready for after-sauna swim. While being an inspiring use case of open data of SYKE, the unexpected success of the application also resulted in some technical problems for the group running the background services.

We had a chat with Mikko Hynninen, Development Engineer at the Finnish Environment Institute, to find out how they were able to react to the problems and what were the key takeaways from the situation for their team.

The map’s popularity peak happened on a national holiday (Vappu) and the capacity of the map was not initially sufficient for accommodating that many users”, Mr Hynninen told us. “One of the problems was that the exact timing of the media announcement of the map launch was not known, so the capacity of the services was not increased beforehand. Also the particular overloaded services were not technically monitored at the time, which prevented us noticing the traffic increase immediately. Customer feedback about the application problems made us notice pretty soon that there was something wrong.”


SYKE vesikartta

How was the capacity issue solved?

Finnish Environment Institute team was able to react to the service disruption in a timely manner. “As soon as the capacity problem was noticed, our GIS-group increased the service capacity as much as possible”, Mr. Hynninen adds. “The amount of users peaked on Sunday, and by Monday evening we managed to more than double the service capacity. On Monday night we further increased the capacity of our servers. The number of parallel processes in the services were adjusted based on statistics from ArcGIS Server. Our internal systems are isolated from the Internet services, so neither the national environmental administration nor our research activities were not affected by the traffic spike.”

“Successful communication and timing is crucial in making a popular service stand to the expectations”, Mikko summarizes the lessons learned. “After the news article about the water map was published, the number of requests multiplied by 250. Services intended for wide use of the citizens should only be provided using tiled and cacheable maps. We did know this beforehand, but had made an intentional decision not to pre-calculate these map tiles, since the application also required dynamically retrieved data.”

Ilkka Rinne, CTO of Spatineo, a leading European company providing availability monitoring, usage analytics and performance testing tools for Spatial Web Services, comments: “Open data and public web services like the ones driving the Water map of the Finnish Environment Institute are very important for the society and it is great to see how well this map was received by the Finnish citizens. This kind of cases where particular services suddenly become very popular for a short while are becoming commonplace as the online news sites take advantage of the tremendous possibilities in creating new and interesting dynamic maps that combine data from several open services. We are glad to be able to help our customers, such as the Finnish Environment Institute, in preparing for events like this and improving the already high quality of their Spatial Web Services.”