TY - GEN
T1 - Human and Machine Complementary Roles in Collaborative Evaluation of Creative Speech
AU - Kalateh, Sepideh
AU - Nikghadam-Hojjati, Sanaz
AU - Barata, José
N1 - info:eu-repo/grantAgreement/FCT/Concurso de avaliação no âmbito do Programa Plurianual de Financiamento de Unidades de I&D (2017%2F2018) - Financiamento Base/UIDB%2F00066%2F2020/PT#
Funding Information:
This work was supported by the Portuguese FCT program UIDB/00066/2020 (Center of Technology and Systems - CTS).
Publisher Copyright:
© IFIP International Federation for Information Processing 2024.
PY - 2024/9/12
Y1 - 2024/9/12
N2 - This paper explores the complementary roles of humans and machines in the collaborative evaluation of creative speech, aiming to enhance the accuracy, reliability, and efficiency of creativity assessments. Traditional methods of evaluating creative speech primarily rely on human judges, who bring intuition and contextual understanding but are limited by subjectivity and scalability challenges. Conversely, artificial intelligence offers computational power and consistency but lacks the complex judgment that human evaluators provide. This research aims to investigate the divergent and convergent aspects of each evaluation and highlight the importance of co-creative collaboration between humans and machines. We utilize the Torrance Tests of Creative Thinking to assess the 22 creative speech samples of 11 participants, comparing evaluations conducted by human experts, and an AI system (the GPT4). While the results demonstrate a certain degree of correlation between human and AI evaluation especially in evaluating Flexibility, they also reveal the differences in how humans and machines perceive Originality in text.
AB - This paper explores the complementary roles of humans and machines in the collaborative evaluation of creative speech, aiming to enhance the accuracy, reliability, and efficiency of creativity assessments. Traditional methods of evaluating creative speech primarily rely on human judges, who bring intuition and contextual understanding but are limited by subjectivity and scalability challenges. Conversely, artificial intelligence offers computational power and consistency but lacks the complex judgment that human evaluators provide. This research aims to investigate the divergent and convergent aspects of each evaluation and highlight the importance of co-creative collaboration between humans and machines. We utilize the Torrance Tests of Creative Thinking to assess the 22 creative speech samples of 11 participants, comparing evaluations conducted by human experts, and an AI system (the GPT4). While the results demonstrate a certain degree of correlation between human and AI evaluation especially in evaluating Flexibility, they also reveal the differences in how humans and machines perceive Originality in text.
KW - Computational Creativity
KW - Creative Speech
KW - GPT
KW - Human-Machine Collaboration
KW - Linguistic Computation
KW - LLM
UR - https://www.scopus.com/pages/publications/85205124966
U2 - 10.1007/978-3-031-71739-0_4
DO - 10.1007/978-3-031-71739-0_4
M3 - Conference contribution
AN - SCOPUS:85205124966
SN - 9783031717383
T3 - IFIP Advances in Information and Communication Technology
SP - 51
EP - 67
BT - Navigating Unpredictability
A2 - Camarinha-Matos, Luis M.
A2 - Ortiz, Angel
A2 - Boucher, Xavier
A2 - Barthe-Delanoë, Anne-Marie
PB - Springer Science and Business Media Deutschland GmbH
T2 - 25th IFIP WG 5.5 Working Conference on Virtual Enterprises, PRO-VE 2024
Y2 - 28 October 2024 through 30 October 2024
ER -