Publications about the project
Multimodal Geo-Information Extraction from Social Media for Supporting Decision-Making in Disaster Management
Effective decision-making in natural disaster management relies heavily on a comprehensive understanding of the situation in affected areas. Social media has been established as a tool to monitor human response and damage assessment. Given the vast amounts of data available, computational methods such as topic modelling are typically employed to reduce information complexity. However, these methods mostly neglect aspects such as geographic location and emotional response, which frequently results in sequential workflows of initial semantic filtering and subsequent spatial or spatio-temporal analysis. This study presents a novel approach for multimodal information extraction from geo-social media data for aiding decision support in disaster management. The method leverages a spatial, temporal, semantic, and sentiment-based clustering approach of social media posts to extract clusters that provide insights into disaster-related content. A case study in the Ahr Valley region in Germany demonstrates the method’s effectiveness in providing actionable insights for disaster response and management. The approach offers a tool for the quick assessment of disaster-related information from social media, potentially aiding timely and informed decision-making.
Active Learning for Identifying Disaster-Related Tweets: A Comparison with Keyword Filtering and Generic Fine-Tuning
Information from social media can provide essential information for emergency response during natural disasters in near real-time. However, it is a difficult task to identify the disaster-related posts among the large amount of unstructured data available. Previous methods often use keyword filtering, topic modelling or classification-based techniques to identify such posts. Active Learning (AL) presents a promising sub-field of Machine Learning (ML) that has not been used much in the field of text classification of social media content. This study therefore investigates the potential of AL for identifying disaster-related Tweets. We compare a keyword filtering approach, a RoBERTa model fine-tuned with generic data from CrisisLex, a base RoBERTa model trained with AL and a fine-tuned RoBERTa model trained with AL regarding classification performance. For testing, data from CrisisLex and manually labelled data from the 2021 flood in Germany and the 2023 Chile forest fires were considered. The results show that generic fine-tuning combined with 10 rounds of AL outperformed all other approaches. Consequently, a broadly applicable model for the identification of disaster-related Tweets could be trained with very little labelling effort. The model can be applied to use cases beyond this study and provides a useful tool for further research in social media analysis.
Assessing the spatial accuracy of geocoding flood-related imagery using Vision Language Models
While the capabilities of large language models and visual language models for various classification tasks have advanced significantly, their potential for location inference remains largely underexplored. Therefore, this study evaluates the performance of four prominent models — BLIP-2, LLaVA1.6, OpenFlamingo, and GPT-4o — for geocoding flood-related images from Flickr. Model inferences are compared against the original photo locations and human-labelled assessments. Our findings reveal that GPT-4o achieves the highest spatial accuracy (median deviation of 89.12 km). OpenFlamingo geocodes the highest number of images (90.7%), albeit with fluctuating quality (median 408.35 km), while still outperforming the human annotators. LLaVA1.6 geocodes only 18.9% of all images, while BLIP-2 exhibits the highest median deviation (1,781 km). We observe a spatial bias in our results, with inferences being most accurate in Central Europe. Additionally, model results improve when images feature recognisable landmarks. The proposed workflow could significantly increase the amount of geocoded web-based data available for disaster management, though further research is required to enhance accuracy across diverse geographic contexts.
More than just Tweets: the potential of alternative geo-social media data for disaster management
Natural disasters are increasingly prevalent worldwide, necessitating the utilisation of diverse datasets for effective disaster response. While geo-social media data represents a valuable resource in this context, the recent restrictions to Twitter data have significantly impacted its availability for disaster research. Alternative social media platforms to Twitter remain underexplored, leading to limited understanding of their potential. To address this gap, we collected posts for a specific use case, Hurricane Ian, from four social media platforms (Mastodon, Reddit, Telegram, and TikTok), and subsequently geoparsed each post. We then computed spatial and temporal patterns and evaluated their correlations to investigate the potential applicability of other data sources for disaster response efforts. While none of the platforms can fully substitute Twitter’s role in disaster management, the findings demonstrated that substantial amounts of potentially valuable data can be sourced from other platforms. Despite consistent overall patterns, subtle differences in temporal activity and spatial distribution suggest that each platform offers unique insights that enhance situational awareness. However, a significant challenge in using these platforms for disaster response is the low spatial accuracy achievable through geoparsing.
Bluesky as a social media data source for disaster management: investigating spatio-temporal, semantic and emotional patterns for floods and wildfires
Social media has become a key data source for near-real-time disaster monitoring and response, with Twitter playing a central role for over a decade. However, recent Application Programming Interface (API) changes on Twitter (now: X) have restricted academic data access, creating an urgent need to identify viable alternatives. This study investigates the suitability of the decentralised social media platform Bluesky for disaster-related geo-social media analysis, aiming to evaluate whether it can serve as a viable alternative microblogging platform for spatio-temporal disaster monitoring. Using a keyword-based crawling pipeline, we collected 676,337 posts related to two major natural disasters: the September 2024 Central Europe floods and the January 2025 Southern California wildfires. We applied a multilingual analysis pipeline covering semantic, emotional, geospatial, and temporal modalities. It includes disaster-relatedness classification, emotion detection, geoparsing and subsequent spatio-temporal aggregation. Our results show that disaster-related content on Bluesky surged in direct response to the disasters, peaking at up to 80% of daily posts during the main impact phases. Emotional expressions, particularly fear and anger, rose sharply alongside event progression. Geospatial analysis of the geoparsed data revealed heightened disaster-related posting activity in affected areas, demonstrating the platform’s utility for geographic disaster monitoring. However, large differences between urban and rural regions, as well as between different countries, were identified. Furthermore, we demonstrate current platform limitations such as user penetration, API constraints and sensitivity to keyword selection.
Few-Shot Learning for Relevance Classification of Textual Social Media Posts in Disaster Response
STRUCTURED EFFICIENT SELF-ATTENTION SHOWCASED ON DETR-BASED DETECTORS
© 2025 N. Militsis, V. Mygdalis, I. Pitas. This is the authors' version of the work. It is posted here for your personal use. Not for redistribution
The Multi-Head Self-Attention (MHSA) mechanism stands as the cornerstone of Transformer architectures, endowing them with unparalleled expressive capabilities. The main learnable parameters in a transformer self-attention block include matrices that project the input features into subspaces, where similarity metrics are thereby calculated. In this paper, we argue that we could use less learnable parameters for achieving good projections. We propose the Structured Efficient Self-Attention (SESA) module, a generic paradigm inspired by the Johnson-Lindenstrauss (JL) lemma, that employs an Adaptive Fast JL Transform (A-FJLT) parameterised by a single learnable vector for each projection. This allows us to eliminate a substantial 75% of the learnable parameters of the legacy MHSA, with very slight sacrifices to accuracy. SESA properties are showcased on the demanding task of object detection at the COCO dataset, achieving comparable performance with its computationally intensive counterparts.
These Maps Are Made by Propagation: Adapting Deep Stereo Networks to Road Scenarios with Decisive Disparity Diffusion
Stereo matching has emerged as a cost-effective solution for road surface 3D reconstruction, garnering significant attention towards improving both computational efficiency and accuracy. This article introduces decisive disparity diffusion (D3Stereo), marking the first exploration of dense deep feature matching that adapts pre-trained deep convolutional neural networks (DCNNs) to previously unseen road scenarios. A pyramid of cost volumes is initially created using various levels of learned representations. Subsequently, a novel recursive bilateral filtering algorithm is employed to aggregate these costs. A key innovation of D3Stereo lies in its alternating decisive disparity diffusion strategy, wherein intra-scale diffusion is employed to complete sparse disparity images, while inter-scale inheritance provides valuable prior information for higher resolutions. Extensive experiments conducted on our created UDTIRI-Stereo and Stereo-Road datasets underscore the effectiveness of D3Stereo strategy in adapting pre-trained DCNNs and its superior performance compared to all other explicit programming-based algorithms designed specifically for road surface 3D reconstruction. Additional experiments conducted on the Middlebury dataset with backbone DCNNs pre-trained on the ImageNet database further validate the versatility of D3Stereo strategy in tackling general stereo matching problems.
3D-Flood Dataset
General description of the dataset
The dataset will be used for the construction of a 3D model regarding the district of Agios Thomas in Larisa, Greece, after the flood events of 2023. It is comprised of 795 UAV video frames, taken from four publicly available videos.
Dataset Structure
Within the dataset, a dedicated folder contains four CSV files. Each file provides the link to the original video and specifies the video frames that were utilized from each source.
Details on acquiring the dataset can be found here.
Flood Master Dataset
General description of the dataset
This dataset comprises flood-related images collected from several publicly available datasets, as well as frames extracted and annotated from related videos. The training and validation sets were constructed using the following sources: “Flood Area Segmentation,” “Water Dataset,” and “Roadway Flooding Image Dataset.” For the first two sources, the corresponding binary masks were normalized to values in {0,1}, while the third source was retained in its original normalized form. The test set consists of video frames extracted from real flooding scenarios in Greece and Italy, which were annotated for segmentation purposes. Specifically, the Greek and Italian videos contributed 567 and 1,406 frames, respectively. If one uses any part of these datasets in his/her work, he/she is kindly asked to cite the following papers:
- P. Mentesidis, V. Mygdalis and I.Pitas, "Improve Real-time flood segmentation by encoding and distilling foreground information", IEEE International Conference on Image Processing (ICIP), Anchorage, Alaska, USA, 13-17 September, 2025.
- A. Gerontopoulos, D. Papaioannou, C. Papaioannidis and I.Pitas, "Real-Time Flood Water Segmentation with Deep Neural Networks", IEEE 25th International Symposium on Cluster, Cloud and Internet Computing Workshops (CCGridW), Tromsø, Norway, pp. 85-91, 2025
Dataset Structure
The dataset is organized into separate folders for the training, validation, and test sets, each containing the corresponding annotations and a CSV file specifying the image paths, annotation paths, and the source of each image. The training–validation split was performed using a 3:1 ratio. Finally, the URLs of the original data sources are provided in the “sources.csv” file.
Details on acquiring the dataset can be found here.