TY - GEN
T1 - Multimodal Creativity State Detection from Speech and Voice
AU - Kalateh, Sepideh
AU - Estrada-Jimenez, Luis A.
AU - Nikghadam-Hojjati, Sanaz
AU - Barata, José
N1 - info:eu-repo/grantAgreement/FCT/Concurso de avaliação no âmbito do Programa Plurianual de Financiamento de Unidades de I&D (2017%2F2018) - Financiamento Base/UIDB%2F00066%2F2020/PT#
Funding Information:
This work was funded by Fundação para a Ciência e Tecnologia through the program UIDB/00066/2020 and Center of Technology and Systems (CTS).
Publisher Copyright:
© IFIP International Federation for Information Processing 2024.
PY - 2024/6/28
Y1 - 2024/6/28
N2 - Modern society has been shaped and advanced by creativity, which has an impact on many facets of human expression and interaction. This research explores the intersection of technology and human interaction by proposing a computational multimodal creativity state identification from speech and voice. This study examines the relationship between language elements and emotion detection, as well as the emotional dimensions (arousal and valence), with a focus on the dynamic and process of identifying creative states through the analysis of speech and voice modalities. To achieve the creativity state detection, it uses a CNN model that was trained using four datasets: Crema-D, Ravdess, Savee, and Tess. A linguistic prompt creativity test is included in the study to provide external validation of the suggested model. The findings show a strong relationship between emotion and emotional dimension, alongside the machine-based detection methods.
AB - Modern society has been shaped and advanced by creativity, which has an impact on many facets of human expression and interaction. This research explores the intersection of technology and human interaction by proposing a computational multimodal creativity state identification from speech and voice. This study examines the relationship between language elements and emotion detection, as well as the emotional dimensions (arousal and valence), with a focus on the dynamic and process of identifying creative states through the analysis of speech and voice modalities. To achieve the creativity state detection, it uses a CNN model that was trained using four datasets: Crema-D, Ravdess, Savee, and Tess. A linguistic prompt creativity test is included in the study to provide external validation of the suggested model. The findings show a strong relationship between emotion and emotional dimension, alongside the machine-based detection methods.
KW - Affective Computing
KW - CNN
KW - Computational Creativity
KW - Creative State detection
KW - Emotion detection
KW - Speech and Voice
UR - https://www.scopus.com/pages/publications/85199656503
U2 - 10.1007/978-3-031-63851-0_8
DO - 10.1007/978-3-031-63851-0_8
M3 - Conference contribution
AN - SCOPUS:85199656503
SN - 9783031638503
T3 - IFIP Advances in Information and Communication Technology
SP - 123
EP - 136
BT - Technological Innovation for Human-Centric Systems - 15th IFIP WG 5.5/SOCOLNET Advanced Doctoral Conference on Computing, Electrical and Industrial Systems, DoCEIS 2024, Proceedings
A2 - Camarinha-Matos, Luis M.
A2 - Ferrada, Filipa
PB - Springer Science and Business Media Deutschland GmbH
T2 - 15th Advanced Doctoral Conference on Computing, Electrical and Industrial Systems, DoCEIS 2024
Y2 - 3 July 2024 through 5 July 2024
ER -