Eo-learn , as the PerceptiveSentinel main platform is called, aims at providing a set of tools to make prototyping of complex EO workflows as easy, fast, and accessible as possible. Eo-learn has grown into a remarkable piece of open-source software, ready to be put to use by anyone who is curious about EO data, to actually using them for data science and machine learning.
On PyPI, where eo-learn is available for download, the project has over 31.000 downloads. More than 182 developers have created their own branch of eo-learn, which people do when they want to do local modifications. These numbers do not include people, who use eo-learn as a library installed like any other Python package. The eo-learn repository has more than 500 downloads per week.
The Sentinel Hub forum thread dedicated to our open-source libraries such as eo-learn has over 120 threads, which were viewed more than 40 thousand times. There are numerous Sentinel Hub users, Enterprise level subscribers, for whom we know they are using it due to eo-learn and they generate more than 80% of the volume of the processed requests, more than quarter of a billion requests per month. The Sentinel Hub blog has 100.000 views of blog posts related to eo-learn, e.g. Introducing eo-learn, Land Cover Classification with eo-learn, Land Cover Monitoring System.
During the first reporting period the initial focus of the project was on providing data analysis, data collection and data verification. We have uploaded all the EO-data as well as non-EO data to the project FTP server, so that they can be used et masse. The data are accessible using eo-learn platform as well. For verification of EO and non-EO data purposes, data from field trials studying N application rates in winter wheat were obtained. The Algorithm Theoretic Baseline Document was developed and contains an overview of to be image processing algorithms that are going to be implemented and summarizes necessary vocabulary on data, processing levels, algorithms and remote sensing as well as pre-processing methods. Selected features are mainly spatial or local parameters such as texture values, shape and geometric descriptors. Spatial and radiometric solution were selected from the state of the art by its characteristics of performance and low computing time, in order to be extracted systematically by the eo - learn platform for deriving higher level information. We have prepared working prototypes and pushed the feature extractors into the PreceptiveSentinel eo-learn platform. We have produced and refined the top-level architectural design of the eo-learn platform, i.e. the top-level structure and software components.
During the second reporting period from April 2019 to June 2020 the focus of the project was the development of PerceptiveSentinel core libraries, which have been upgraded with the introduction of pre-processing algorithms incorporated into eo-learn library. The library illustrates high-level architecture and main components of PerceptiveSentinel. It is exposed as a rich variety of pre-processing and processing algorithms used for data pre-processing or creation of custom EO-processing chains such as data pre-processing, deep learning algorithms incorporated into processing library. These are integrated with EO-QMiner to support streaming machine learning, new data gateways and cases showing how to use eo-learn library with external services.
Software components eo-learn library, QMiner, pre-processing library and Jupyter Notebook have been integrated with PerceptiveSentinel platform (eo-learn platform). EO-QMiner integration layer has been developed to support seamless communication with machine-learning engine. Its main tasks are to transform data into existing QMiner data formats for the ingestion by EO-QMiner and to provide labelled data. Workflow designer and workflow engine have been upgraded with the addition of new input modules to support new data gateways and new processing modules to support streaming machine learning. Platform provides learning and interpretation data; external services provide interpreted data.
The EOTasks are part of the public eo-learn GitHub repository. As such anyone can use them, copy them, or modify them. However, certain tasks are considered as IPR and its authors or owners would like to keep them protected and not shared. Such IPR-protected value-added-services can still be used with eo-learn as long as they are wrapped as an EOTask. Then these services can be used as a standalone EOWorkflow or as a Task within EOWorkflow consisting of several tasks.
Cultivated areas detection was required early in the crop growing period and the service would not have been able to use long time series. Therefore, a new deep learning approach has been applied, using a single date and implementation of deep learning algorithms for “cultivated areas” extraction and “crop type” detection on Sentinel-2 data inside the eo learn framework.
A Jupyter-Notebook for the automatic classification of crop types with Sentinel-2 data has been developed. LPIS data is used as a source of in-situ data. It was originally developed for regions in Austria but has also been applied to test regions in Denmark. A multitemporal crop damage algorithm was developed using the eo-learn library and implemented as a REST API web service. Algorithm for the detection of soil moisture was developed and the resulting data for three distinct regions and two consecutive years 2018 and 2019 was modelled (Lower Austria, Mura region in Slovenia and middle Jutland in Denmark, excluding areas close to the ocean). A Jupyter notebook is provided which can be used to access the data.
For the crop cycle the same approach was taken, using data of Sentinel 2 satellites trying to illustrate the crop development at regional level and compared the crop development of certain smaller area of interest with the regional average of crop development. We illustrate this functionality through the Jupiter Notebook. The Crop Cycle Service can be implemented in the same way also in other European countries when using results from Crop Type Service as an input data. We prepared Jupiter Notebook using Slovenian LPIS data from 2019. Crop Type service demonstrated full functionality with crop type identification for most crops with over 90% accuracy so it serves well as input data for Crop Cycle Service.
Stream-learning validation report presents a variety of achievements within the PerceptiveSentinel. Perhaps the biggest achievement of this work is the FASTENER FeAture SelecTion ENabled by EntRopy algorithm, which extends the current state of the art in Earth Observation and is a strong candidate even for other use cases. The algorithm is based on genetic approach and exploits information theory measures based on entropy to steer the feature selection process. The algorithm has been tested in 25 general feature selection scenarios, where – in terms of speed and accuracy – it performs better than the current state-of-the-art.
In Crop Yield, a model predicting winter wheat yield within the field during the growth season has been developed, thereby giving the farmer the opportunity to regulate fertilizer strategy. The prediction model has been implemented in the web-based management tool Crop Manager used by Danish farmers. The integration into Crop Manager was used for demonstrating external chaining capabilities of the PerceptiveSentinel platform. Crop Manager serves as an external front-office consuming PerceptiveSentinel Services, upgrading them with specific agricultural services and providing a data link with Mark Online, thus delivering high added value to Danish farmers. The winter wheat prediction model provides the farmer with a valuable tool to improve the management of his farm, both in the field and in budgeting.
A key resource for the success of eo-learn is, of course, the community, both of remote sensing and machine learning experts. We are constantly improving on new functionalities, stability, and efficiency on tasks and workflows as the library grows. We therefore have invited anyone with interests in developing large-scale remote sensing applications using spatial-temporal satellite imagery to try eo-learn out, give us feedback, and possibly contribute to it . The project partners have successfully planned, coordinated and facilitated several communication activities tailored to target specific audience throughout the project. Through communication activities project partners have promoted the project to specific target groups, attracted potential intermediate users and end users and established the circumstances for successful market entry.
In the project Jozef Stefan Institute has developed a stream learning C++ library exposed in Python called ml-rapids. It is easily used in python and with eo-learn and can be installed using pip. It could be used for EO scenarios, for fast and still accurate classification. The ml-rapids library was used also in other environmentally oriented projects at Jozef Stefan Institute. One of the usages is in the creation of a COVID-19 decision support system. Another developed algorithm is FASTENER, which is a general feature selection algorithm. It was originally developed for feature selection in EO use cases but was tested and can be used on multiple more general data sets. The algorithm is competitive with other state-of-the-art feature selection algorithms.
During the project period, L&F has disseminated the project results both domestically and abroad to rise visibility and awareness about project results, with the end users as our primary target group. The main focus was on teaching farmers and farm advisors to use the new capabilities developed throughout the project and has disseminated the newly developed tool for precision agriculture through presentations and demonstrations at conferences, webinars, articles, manuals and through courses for farm advisors. Nearly all Danish farmers have an affiliated crop production advisor, and thus training of advisors is an effective strategy for implementing new technology in Danish farming. The newly developed tool for precision agriculture has been well received among the farmers and farm advisors.
The eo-learn library was identified for Geoville as an excellent teaching and development tool and was used to create a master thesis as part of the UNIGIS Master programmer of the University of Salzburg.
Eo-learn was presented to many collaborators by Magellium. This allowed eo-learn to be considered and used for other applications. The main use for the framework in Magellium is prototypes and visualizations as it allows quick manipulation of satellite data which is decisive in every data science projects.
All deliverables were submitted to the EC (see Table).