LifeWatch ERIC Metadata Catalogue
AI-based Framework for Land Use/Land Cover abundance estimation using (Sentinel-2 high-resolution RGB images + Environmental ancillary data)
This workflow employs a deep learning model for blind spectral unmixing, avoiding the need for expensive hyperspectral data. The model processes 224x224 pixel RGB images and associated environmental data to generate CSV files detailing LULC abundance at two levels of detail (N1 and N2). The aim is to provide an efficient tool for LULC monitoring, answering the question: Can LULC abundance be estimated from RGB images and environmental data? This framework supports environmental monitoring and land cover analysis.
Background
Land Use and Land Cover (LULC) represents earth surface biophysical properties of natural or human origin, such as forests, water bodies, agricultural fields, or urban areas. Often, different LULC types are mixed together in the same analyzed area. Nowadays, spectral imaging sensors allow us to capture these mixed LULC types (i.e., endmembers) together as different spectral data signals. LULC types identification within a spectral mixture (i.e., endmembers identification) and their quantitative abundance assessment (i.e., endmembers abundances estimation) play a key role in understanding earth surface transformations and climate change effects. These two tasks are carried out through spectral unmixing algorithms by which the measured spectrum of a mixed image is decomposed into a collection of constituents (i.e., spectra, or endmembers), and a set of fractions indicating their abundances.
Introduction
Early research on spectral unmixing dates back more than three decades. First attempts, referred to as linear unmixing, assumed that the spectral response recorded for an LULC mixture is simply an additive function of the spectral response of each class weighted by its proportional coverage. Notably, some authors used linear regression and similar linear mixture-based techniques in order to relate the spectral response to its class composition. Afterwards, other authors claimed the necessity of overcoming this assumption by proposing non-linear unmixing methods. However, non-linear methods require endmember spectra extraction for each LULC class, which has been found difficult in several works. Moreover, some studies indicated that it is unlikely that the spectra could be derived directly from the remotely sensed data since the majority of image pixels may be mixed. To overcome these limitations, several works introduced what is called blind spectral unmixing as an alternative method to avoid the need to derive any endmember spectra or making any prior assumption about their mixing nature. However, the majority of works that adopted blind spectral unmixing used deep learning-based models trained with expensive and hard-to-process hyperspectral or multispectral images. Therefore, many researchers during the last decade pointed out that more effort should be dedicated towards the usage of more affordable remote sensing data with few bands in spectral unmixing. They justified this need by two important factors: (1) In real situations, we might have access to images with only a few bands because of their availability, cost-effectiveness, and acquisition time-efficiency in comparison to imagery gathered with multi-band devices that require more processing effort and expenses; (2) In some cases, we do not really need a huge number of bands, as they can be used as a fundamental dataset from which we determine optimal wavebands for a particular application. In parallel, high-quality research in artificial intelligence application to remote sensing imagery, such as computer vision-based techniques and especially DL, is continuously achieving new breakthroughs that encourage researchers to entrust remote sensing imagery analysis tasks to these models and be confident about their performance.
Aims
The objective of this work is to present what is to our knowledge the first study that explores a multi-task deep learning approach for blind spectral unmixing using only 224x224 pixels RGB images derived from Sentinel-2 and enriched with their corresponding environmental ancillary data (topographic and climatic ancillary data) without the need to use any expensive and complex hyperspectral or multispectral data. The proposed deep learning model used in this study is trained in a multi-task learning approach (MTL) as it constitutes the most adequate machine learning method that aims to combine several information from different tasks to improve the performance of the model in each specific task, motivated by the idea that different tasks can share common feature representations. Thus, the provided model in this workflow was optimized for elaborating endmembers abundance estimation task that aims to quantify the spatial percentage covered by each LULC type within the analyzed RGB image, while being trained for other spectral unmixing related tasks that improves its accuracy in the main targeted task which is endmembers abundance estimation. The provided model here is able to give for each input (RGB image + ancillary data) the contained endmembers abundances values inside its area summarized in an output CSV file. The results can be computed for two different levels N1 and N2. These two levels reflect two land use/cover levels definitions in SIPNA land use/cover mapping campaign (Sistema de Información sobre el Patrimonio Natural de Andalucía) which aims to build an information system on the natural heritage of Andalusia in Spain (https://www.juntadeandalucia.es/medioambiente/portal/landing-page-%C3%ADndice/-/asset_publisher/zX2ouZa4r1Rf/content/sistema-de-informaci-c3-b3n-sobre-el-patrimonio-natural-de-andaluc-c3-ada-sipna-/20151). The first level "N1" contains four high-level LULC classes, whereas the second level "N2" contains ten finer level LULC classes. Thus, this model was mainly trained and validated on the region of Andalusia in Spain.
Scientific Questions
Through the development of this workflow, we aim at addressing the following main scientific question:
- Can we estimate the abundance of each land use/land cover type inside an RGB satellite image using only the RGB image and the environmental ancillary data corresponding to the area covered by this image?
Default
Identification
- Date ( Publication )
- 2023-12-31T00:00:00
- Status
- Under development / Pre operational
- Version
- 1.0
- Keywords
- AI-based Framework
- Keywords
- Land Use/Land Cover (LULC)
- Keywords
- Sentinel-2
- Keywords
- RGB Images
- Keywords
- Environmental Data
- Keywords
- Deep Learning
- Keywords
- Spectral Unmixing
- Keywords
- Abundance Estimation
- Keywords
- SIPNA
- Keywords
- Remote Sensing
- Keywords
- LULC Monitoring
- Keywords
- Environmental Monitoring
- Access constraints
- Copyright
- Other constraints
- To be confirmed
- Protocol
- DOI
- Service Name
- Abundance estimation AI-model
- Service Description
- An AI-powered script with a multi-input multi-task deep learning model able to automatically estimate the abundance of each land use/land cover type at both levels N1 and N2 inside the provided RGB satellite using these latter as inputs along with their corresponding environmental ancillary data
- Service Reference (id)
- https://github.com/yassirbenhammou/Spectral_Unmixing_RGB_Deep_learning/blob/f7 592fe94cc7398d5cec056a270f3a6140ec9f9c/script.py
- Workflow Helpdesk
- https://helpdesk.lifewatch.eu
