Deliverables

Lessons Learned and Practical Guidelines from a Trustworthiness Assessment: Co-design of an Explainable Artificial Intelligence System for Voice-based Mental State Monitoring

The growing use of artificial intelligence (AI) in mental healthcare raises important challenges regarding transparency, trustworthiness, ethical governance, and clinical usability. This paper presents lessons learned and practical guidelines from the co-design of ExplainMe: an Explainable Artificial Intelligence (XAI) system for voice-based monitoring of mental state in patients with bipolar disorder and major depressive disorder. An interdisciplinary team of 32 experts in computer science, psychiatry, medicine, ethics, social sciences, and law participated in a participatory process following the Z-Inspection® methodology in accordance with the European Commission High-Level Expert Group guidelines for Trustworthy AI. The trustworthiness assessment focused on ethical, legal, technical, and medical dimensions of the considered socio-technical scenarios, including transparency, accountability, privacy, explainability, human oversight, and potential risks associated with psychiatric decision support systems. Trustworthiness assessment process aimed at the co-design of a clinically meaningful and socially responsible AI capable of explaining predictions derived from acoustic speech features. During the process, the interdisciplinary team of experts identified key tensions, e.g., predictive performance vs. explainability, privacy protection vs. clinical applicability. In this work, we summarize main steps of this process and formulate recommendations and practical guidelines for designing trustworthy XAI systems for voice-based mental health monitoring. The findings highlight the importance of interdisciplinary collaboration, participatory design involving clinical stakeholders, context-aware explainability mechanisms, and continuous ethical evaluation throughout the AI lifecycle. The presented lessons learned may support future development of trustworthy AI systems in psychiatry and other sensitive healthcare domains.