Skip to main content
Background Image for header that show a weather storm

Description of the project

DESCRIPTION OF THE PROJECT

Mission
and Vision

Climate change is leading countries in the EU to experience increasingly frequent and damaging adverse climatic events, such as large fires and flooding. The impact of severe weather events is expected to make EU increasingly vulnerable due to the magnitude and frequency with which they will occur in the coming years.

Natural Disaster Management (NDM), in order to support disaster prevention and preparedness, can be greatly improved by the adoption of technologically advanced tools capable of analyzing and processing large volumes of data from different sources to provide predictions on the evolution of phenomena in (near-)real-time and to be useful to stakeholders for recommendations and guidelines to adopt in dealing with a complex emergency.

In light of such urgency, and under the advancements in science and technology that have been achieved in recent years, the TEMA will greatly improve NDM by automating precise semantic 3D mapping and disaster evolution prediction to achieve NDM goals in near-real-time. It will analyze and fuse many heterogeneous extreme data sources: smart drone and in-situ sensors, remote sensing data, topographical data, meteorological data/predictions and geosocial media data (text, image and videos).

TEMA will focus on the extreme nature of the data, due to their varying resolution and quality, very large volume and update rate, different spatiotemporal resolutions and acquisition frequencies, real-time needs and multilingualism. It will develop an integrated, groundbreaking NDM platform, focusing on real-time semantic extraction from multiple heterogeneous data modalities and sources, on-the-fly construction of a meaningful semantically annotated 3D disaster area map, prediction of disaster evolution and improved communication between service providers and end-users, through automated process triggering and response recommendations. Semantic analysis computations will be distributed across the edge-to-cloud continuum, in a federated manner, to minimize latency. Extreme data analytics will be performed in a trustworthy and transparent way, by greatly advancing state-of-the-art AI and XAI approaches. The constantly updated 3D map and the disaster evolution predictions will form the basis for an advanced, interactive, Extended Reality (XR) interface, where the current situation will be visualized and different response strategies will be dynamically evaluated through simulation by NDM personnel. The innovative, scalable and efficient TEMA platform will provide precise NDM support, based on extreme data analytics.

TEMA joins together a pan-European, fully qualified and multidisciplinary consortium, composed of 19 Partners experts in all relevant specific subfields (data analytics, AI/machine learning, remote sensing/Earth Observation, federated and cloud computing, fire/flood modeling, geovisual analytics, AR, NDM), able to properly face the challenge. Partners come from leading research laboratories with significant international R&D experience and relevant end-users from across Europe.

DESCRIPTION OF THE PROJECT

Key
Objectives

The aim of project TEMA is to improve NDM by providing a state-of-the-art disaster management support system, dynamically exploiting multiple data sources and AI technologies for providing an accurate assessment of an evolving crisis situation.

Specifically, TEMA will pursue the following objectives over the next 4 years:

  • Improve Natural Disaster Management (NDM) using new digital technologies and extreme data analytics.
  • Improve and accelerate extreme data analytics, by increasing trustworthiness, accuracy and responsiveness of extreme data analysis algorithms;
  • Improve and accelerate emergency phenomenon modeling, evolution predictions, simulation and interactive visualization.

Background Image: a forest in the night with black trees

Key Results

  • Trustworthy AI

    TEMA advancements in XAI will integrate novel methods for the extreme data scenario (both generic ones for multimodal analysis for heterogeneous sources and more specific ones for vision), as well as innovative Out-of-distribution (OOD) detection algorithms and geometric training regularizers for DNN robustness. In this sense, TEMA will contribute in progressing XAI for multimodal analysis and increase AI robustness.

  • Real-time semantic visual analysis

    Given the low maturity level of visual analysis for real-time extreme data analysis in emergencies, TEMA will improve current SoA by integrating novel, fast DNN-based real-time semantic visual analysis methods for the project use-cases, and novel methods for handling data scarcity during DNN training. In order to improve the generalization ability of trained DNNs so that they are applicable to different imaging sensors and acquisition platforms across varying atmospheric conditions and scene properties, TEMA will provide domain adaptation strategies for better DNN generalization across different geographical areas when analyzing satellite data, multimodal exploitation of visual and non-visual measurements for better 3D smoke reconstructions in real-time and novel DNN-based visual privacy preservation methods.

  • Geosocial media, news and text analysis

    DNNs are the SoA in sentiment analysis, however taking into account complex texts in social media posts, accuracy of semantic analysis tends to drop significantly and becomes difficult. TEMA will integrate fast semantic social media/news posts analysis methods which will automatically perform accurate sentiment analysis in complex text (e.g., using figurative language), accurately identify the topic while taking into account accompanying images, and assign relevance score to the post/article. The estimated sentiment, topic and relevance will then be exploited by the TEMA platform for emergency management, while multiple languages will be inherently supported.

  • Federated analytics

    TEMA solution will provide smart management of federated data for NDM, as analytics will be running on a novel edge-to-cloud continuum, supporting dynamical and transparent distribution of AI/DNN inference workload for a large afflicted area, federated computing and swarm-based organization of edge nodes. This continuum will analyze data coming from heterogeneous sources in real-time.

  • Near-real time phenomenon modeling

    TEMA will accelerate existing forest fire and flood modeling engines to near-real-time using parallelization, ghost cells and other approaches, improving data collation from an unprecedented range of heterogeneous sources/modalities (including early warnings). TEMA will also improve continuous incorporation of up-to-date fused information on-the-fly, exploiting visually-derived information to derive boundary or initial conditions for models, and developing a calibration model of forecasted high-resolution weather data with in-situ weather data obtained from TEMA supported sensors, which will predict accurate smoke 3D concentration distributions (in forest fires) by exploiting measurements from multiple dispersed TEMA sensors and model-based AI approaches. Notably, the information fed to the modeling engines will be derived from an extremely wide range of heterogeneous, georeferenced data sources which will be fused on-the-fly and in near-real-time: smoke, gas and wind measurements, RGB videos and thermal images from drone- and ground unit-mounted sensors, satellite/SAR data, social media/news posts, WWW warnings, etc.

  • Decision support for remote sensing

    TEMA will overcome current limitations of using remote-sensing aids for operational and decisional processes, that still require mostly manual steps, by providing an innovative decision support service relying on: fully automated processing of public WWW data (e.g., warning, forecasts, etc.) and of TEMA modeling outputs, and automated retrieval of satellite position and acquisition data. This service will inform human operators about imminent emergencies much faster than the current SoA and in a much less labor-intensive manner, and moreover will be able to automatically request collection of relevant social media data for further processing by the Analytics module.

  • Response planning and recommendations for optimal sensor placement

    In order to improve good semantic mapping and increase situational awareness, critical elements in disaster management, TEMA will provide near-real-time recommendations for mission/path planning regarding supported drones and ground units, using a cloud-based engine tuned for optimal sensor placement, that exploits NDM domain knowledge, a priori available physics constraints, the TEMA semantic map and TEMA model predictions. The innovative TEMA fleet of autonomous drones will be fully able to automatically receive and execute the recommendations on-the-fly, as an optional step.

  • XR based interactive visualization

    TEMA aims at improving the utilization of Digital Twins in NDM and overcoming limitations of existing geovisual analytics. To this end, TEMA will merge a georeferenced Digital Twin and a geospatial map constructed on-the-fly and in real-time during the emergency, and will automatically annotate the resulting 3D area map with semantic annotations, predictions, decision proposals and recommendations derived automatically from the other TEMA components. It will also allow real-time interactive visualization of the annotated, multi-view, high-resolution 3D map via both an AR interface and a complementary desktop GUI, thus leading to a combined XR interface, it will permit interactive exploration of contingent response alternatives in this map by keeping it connected to always up-to-date copies of the (flood, fire) modeling engines, finally ensuring optimal user experience for the human operator, via feedback and design input from HCI experts.