The “ExplainMe” project aims to develop innovative IT tools that enable explaining the way of speaking, thus supporting diagnostics and monitoring of health.
There are applications on the market that support the monitoring of patients suffering from affective disorders, e.g. MONARCA, ReMind, SIMBA, BraPolar, Monarca II, UP!, AIoT, but none of them allows alerting the patient/doctors/relatives about affective states (e.g. depression, mania, etc.) based on advanced voice analysis based on artificial intelligence and does not explain how a person’s voice speaking has changed. The MoodMon system developed by the business partner for monitoring of patients with affective disorders u ses voice. Unfortunately, algorithms embedded in it they operate on the “black box” principle and users do not know what caused the alarm, which is a significant limitation during the implementation of the system and doctors accept it with reservation information about alarms generated by the system. Explainable algorithms developed in ExplainMe artificial intelligence can significantly increase trust among system users (doctors, patients, their loved ones). ExplainMe will be the first IT technology that is a set of methods computational intelligence enabling advanced reasoning based on voice and manner speaking and explaining affective states in natural language against the background of the way of speaking.
The originality of ExplainMe’s research work lies in the development of reliable intelligence methods computational support for analysis, monitoring and explanation of complex and incomplete medical data including both multivariate time series and expert knowledge.
The developed methods for complex medical data combine the theory and applications of intelligence computing (intelligent computing) with selected elements of statistics, analysis methods and time series forecasting and signal processing and large data streams. ExplainMe will provide innovative technologies because they will effectively combine statistical modeling with granular computing and computing with words words). ExplainMe will develop technology that will provide easy-to-interpret summaries linguistic explanations for complex data describing the way of speaking and incomplete health information. The novelty and originality of ExplainMe’s development work lies in the application of developed technologies in specific areas of health care and medicine, especially psychiatry. Although the way of speaking may signal the first symptoms of episodes schizophrenia, unipolar depression, and other affective disorders [Low, D et al. (2020). “Automated “assessment of psychiatric disorders using speech: A systematic review” ttps://pubmed.ncbi.nlm.nih.gov/32128436/], there is no reliable computer tool on the market supporting doctors and patients in monitoring psychiatric disorders and the development of these disorders and treatment effectiveness. Numerous studies show that voice analysis can help monitor mental illnesses [A. Antosik-Wójcińska et al. “Smartphone as a monitoring tool for bipolar disorder: a systematic review including data analysis, machine learning algorithms and predictive modeling”, https://www.sciencedirect.com/science/article/abs/pii/S1386505619312766]. For example, in the state In the depressed state, speech activity is reduced and the voice characteristics associated with pauses are intensified. Features acoustic, such as signal energy in different bands or melcepstral coefficients can be extract from the voice. Unfortunately, there is still a lack of effective IT tools that would enable effective explanation of changes in the way of speaking in natural language and also explanation of the voice against the background symptoms associated with mental disorders such as low mood, anxiety, agitation, etc.
During quarterly meetings, the IAB will monitor the achievement of the set goal and support prime contractor for: (1) developing explainable AI that meets standards ethical, legal; (2) commercialization strategies and implementations of specific applications in medicine and healthcare health; (3) promotion of the project results.