In October 2025, Katarzyna Kaczmarek-Majer and Marcin Ostrowski participated in the European Conference on Artificial Intelligence (ECAI 2025), held in Bologna, Italy.
Our principal investigator, Katarzyna Kaczmarek-Majer presented a talk entitled „Pilot Assessment of Transparency of LLM-based Systems to Support Emergency Rooms”. This is a joint work with Marek Reformat from the University of Alberta, Edmonton (Canada) and medical doctors from the Poznań University of Medical Sciences. One of the main challenges when developing medical decision support systems for the emergency room is adequately filtering the most relevant information. Physicians working with patients need specific details on their clinical situation. High workload, stress, and the necessity for urgent decisions require precise answers to the questions posed. Although LLM-based systems can provide abundant information, physicians need concise and relevant data in clinical settings. In this study, we perform a pilot assessment of the transparency of selected LLM-based systems. The comparative analysis includes ChatGPT o1 model, which was asked to produce responses with varying temperatures and a pilot graph-based RAG specializing in cardiovascular diseases. A survey was conducted among 33 clinicians regarding the amount of information contained in the provided prompts. Physicians favored the most readable, specific, and helpful answers in emergency department conditions. Reliable medical data and the form in which answers are delivered are crucial for physicians working in the emergency room. We conclude that physicians have preferences for LLM responses at a specific temperature. Further research should be expanded to enable tailoring responses not only to the clinical situation but also to the experience of the asking physician. The article is available HERE.
Marcin Ostrowski presented paper “Beyond Static Importance: Quantifying Stability and Distribution Drift”. The conference provided an excellent platform to discuss research on the temporal stability of machine learning explanations and to receive valuable feedback from the international AI community. Feature importance is a cornerstone of explainable machine learning. In temporal settings, where data accumulates sequentially, the relevance of features may evolve, introducing challenges for interpretation. While temporal variation in feature importance is increasingly relevant for applications such as clinical monitoring and time-series prediction, it remains underexplored in the literature. In this paper, we propose a novel methodology for quantifying the temporal stability of local feature attributions. Our approach combines exponentially weighted moving average (EWMA) model with performance metrics. The goal is to compute a feature-wise stability metric that reflects how consistently a feature contributes to model predictions over time. To complement this, we introduce a distributional drift score based on the Wasserstein distance, capturing shifts in the underlying feature distributions. Together, these two signals form a diagnostic framework that distinguishes between shifts due to data dynamics and those arising from model behavior. We evaluate our approach on a simulated dataset reflecting mental health monitoring scenario, as well as a publicly available benchmark time-series dataset. In both cases, the proposed metrics uncover nuanced patterns of feature behavior, enabling practitioners to identify features that are not only important but also temporally reliable. Our results demonstrate that assessing both the stability of explanations and the drift of features provides a more robust foundation for trustworthy model interpretation in dynamic environments. The article is available HERE.
ExplainMe team co-organized the EXPLIMED workshop. Proceedings are available under this link: CEUR-WS.org/Vol-4059 – Second Workshop on Explainable Artificial Intelligence for the Medical Domain (EXPLIMED 2025)
Beyond the presentation, ECAI 2025 offered a unique opportunity to engage with researchers from around the world, including both early-career scientists and established experts in artificial intelligence and machine learning. Numerous discussions, both formal and informal, enabled a fruitful exchange of ideas and perspectives, which proved inspiring and contributed to the further development of my research directions. The conference was also an enriching cultural experience, taking place in the historic city of Bologna, where the welcoming atmosphere and renowned Italian cuisine created an ideal setting for networking and collaboration.
