Crop Monitoring with Artificial Intelligence – How does it work with Sentinel-2 Satellite Imagery?

Joona Laine from Spatineo chose this as subject of his master’s thesis (Crop identification with Sentinel-2 satellite imagery in Finland). What kind of problems he solved and what were the steps to ensure that the results were high quality? How does crop identification work with neural network and what are cloud pixels?

Crop Monitoring with Artificial Intelligence – How does it work with Sentinel-2 Satellite Imagery Spatineo

The need for Field Monitoring and Crop Identification

There are 22 million farmers and agricultural workers in the European Union (EU) agri-food sector. To ensure a decent standard of living to the farmers, EU is supporting them with the common agricultural policy and In order to ensure that the subsidies are divided equally and without misuse, the CAP subsidies must be controlled by the local authorities within Integrated Administration and Control System (IACS). At least 5% of the agricultural parcels, that the subsidies are applied for, have to be monitored for each year.

This monitoring has been done traditionally by inspectors actually going on site, to check the field and making sure that all information gives in the CAP application is valid. Checking 5% of all the fields manually is a huge effort for local authorities, and this is where satellite monitoring comes is as a “easier” solution.

Satellite imagery, and specifically Sentinel-2 imagery, was optimal choice because the cost effectiveness was unparalleled, the spatial resolution (pixel size) was suitable for crop monitoring and high revisit frequency was well suited for continuous monitoring. Other options for the source material would’ve been aerial orthoimagery and drone imagery.

From all possible satellite imagery options Sentinel-2 was chosen because the frequency the images are taken. Images taken in 2-3 day intervals gave chance to glance through clouds, as in Finland there can be over 200 cloudy days annually. European Space Agency (ESA) also has made the Sentinel imagery highly and openly available.

How did a machine learn to identify the crops?

Classification was based on pre-learned material, which the algorithms based their assumptions on when evaluating the imagery. Algorithms were “shown” different features consisting of multiple images representing the parcel in hand, and told what was the crop in question in each feature. First fhe algorithm were just shown feature of the parcel and they had to “guess” which crop was in it. Eventually as the algorithms were told each time if they were right or wrong, they were able to evolve better and better in guessing the crops.

Machine learning was used to classify different crop from each other. During the first stages of the project, it was still unclear with machine learning classification algorithm would perform best. Support Vector Machines (SVM), Random Forest (RF), Multi-layer Perceptron (MLP) and Convolutional Recurrent Neural Networks (ConvRNN) were the algorithms tested during the project and most of them were capable of basic classification of the largest classes of crops. Eventually it became clear that SVM, ConvRNN and MLP were most suited for the classification, and hence they were used for most part of the study.

SVM and RF represent more of a traditional way of machine learning classification, while MLP and ConvRNN utilize neural structure of deep learning. This might be the reason why MLP and ConvRNN were more prominent than other methods.

convRNN Crop Monitoring AI Sentinel-2

How to Improve Crop Identification Success Percentage Rate?

During the work it was clear that there were great amount of variables that made harder to identify crops. Some of these variables were just minor hurdles, but some needed hard work to be overcome. Down below there are the issues that required most effort to be fixed during the project.

Cloud interference

Cloud were the hardest problem from the beginning, since they made most of the source images technically useless without modification. After some research it was clear that cloud masks would have to be used, in order to make source material usable. Eventually two cloud masks, one snow mask and one cloud shadow mask were utilized. Although ConvRNN was able to detect clouds automatically.

Appearance dependent of growth cycle

Plants and crops have different appearances dependant on their growth period. This basically means that one image per field wasn’t enough, as some of the crops might have had almost identical appearances at the beginning of their growth. Time series of images from each phenological state (stage of the growth) was used to differentiate the crops from each other.

Imbalance of class distribution of the crops

In the training material for the algorithms there were two classes of crops that had the overwhelming majority. This uneven class distribution made is hard for the algorithms to identify accurately those crops that didn’t below to the two dominant categories. There were two possible fixes for this problem: either delete some part of the dominant categories or increase the number of lesser categories from the training material.

Both ways just adjust the ratio between the classes, and after some testing the increasing of the lesser categories proved to be more effective method.

Crop Monitoring Sentinel 2 Parcel Count
Take note that the scale in this chart is logarithmic.

Results of crop identification

After all the major hurdles were overcome the accuracy of the algorithms rose up to 92%. The goal for the whole project was at least 95%, since 95% is considered to be the limit where the automatic identification could be efficient alternative for the manual inspections done nowadays.

These identification methods proved to be effective and operational in handling crop identification process. In order to make it even more viable, more research and testing would be recommendable, but viability of the method has now been proved. Analysing satellite imagery will be a very viable alternative to traditional inspectors visiting the fields.

Want to stay ahead of the game, and get latest news from us? Subscribe now to our newsletter!

Who are we? – Part XV: Sanna Hautala

Sanna Hautala Spatineo GIS Junior Consultant

1.) Name, title, how long have you worked at Spatineo?

My name is Sanna Hautala and I just started working at Spatineo as a Junior GIS Consultant. My background is more in the field of chemistry (Master of Science at Helsinki University) but the last few years I have solely been working with GIS related services and applications.    

2.) What do you do at Spatineo in practice? Tell us about your typical work day!

I haven’t been here that long to have a typical work day yet but I know versatile is a word to describe it. It is a mixture of scripting (bash/python/js), databases,  GIS application interfaces and standards spiced with learning. Shortly described my typical day consists of anything related to GIS including customer consulting.

3.) What is the best thing about your work?

Solving problems, learning new things and the “light bulb” -moments thrives me forward at my work. The feeling of satisfaction when you finally figure out how to solve a problem is priceless. Working environment here at Spatineo inspires me to learn more about the whole industry.

4.) If you are a tourist in a foreign city, which map service do you use? Why?

My main map service is Google maps and especially the geocoding and routing functions of it but I have learned not to trust it completely. It is always a good idea to check the destination and route beforehand just to make sure that it is correct, with pseudonyms “Make U-turn” and “This road cannot get any worse”.

Our taste of JIIDE 2018 in Menorca

JIIDE2018 Menorca

Fabio Bittencourt from Spatineo attended JIIDE conference (Jornadas Ibéricas de las Infraestructuras de los Datos Espaciales) for the third time – the first time was in Barcelona 2016, and apparently the event was top notch this year! This is what he had to sau about JIIDE2018!

 

“The Iberian Journeys for SDIs – JIIDE 2018 main topic this year was “Improving the interchange of spatial data to protect the biosphere” and there was indeed a technical session and round-table discussion exclusively devoted to environmental SDI applications.

Obviously, quite much was also discussed about the regional and local SDIs as well as the distinct local scenarios and realities facing INSPIRE implementation.There were also sessions about new technologies and innovations, and even one session with presentations for discussing the new frontiers of SDIs, like the one made by Antonio Rodriguez, from CNIG named “All you need is open data”.

Coming from the private sector, there were also some very relevant presentations like those made by Tracasa (regarding the news on the SDI Viewer in Navarra) and Geograma with 3 presentations, including one about the implementation of INSPIRE model for Álava. Spatineo could show solutions for quality assurance of spatial services and discuss with some major players in the region. Over 50 new spatial web services were found in our directory during the event, which is a clear sign that despite the almost 100 thousand spatial web services we are already monitoring in our database, every day new spatial services are published and catalogued.”

JIIDE2018 Spatineo Conference
Picture from www.jiide.org

This year’s JIIDE was hosted in a very special place, an island called Lazareto, part of Menorca island. The place has an incredibly interesting historical background as it was used as a hospital for those suspecting having any contagious diseases coming from long boat trips. Menorca organisers deserve all the credit for such peculiar site choice and the successful event. I am already looking forward to see where the next JIIDE will be, and I strongly recommend you in Portugal or Spain not to miss it!

Our main take away from it? Having had some high level discussions with major coordinating organisations both at country-level and regional levels, we understand that they all face very similar challenges, which makes events like JIIDE, where the cooperation among organisations is promoted and encouraged, even more important to the affected societies and its citizens.”


 Fabio Bittencourt is Spatineo’s Area Sales Manager – Register now to his “Importance of Assuring the Robustness of Services in Your Spatial Data Infrastructure” webinar also!