Rating Prediction in Conversational Task Assistants with Behavioral and Conversational-Flow Features

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

1 Downloads (Pure)

Abstract

Predicting the success of Conversational Task Assistants (CTA) can be critical to understand user behavior and act accordingly. In this paper, we propose TB-Rater, a Transformer model which combines conversational-flow features with user behavior features for predicting user ratings in a CTA scenario. In particular, we use real human-agent conversations and ratings collected in the Alexa TaskBot challenge, a novel multimodal and multi-turn conversational context. Our results show the advantages of modeling both the conversational-flow and behavioral aspects of the conversation in a single model for offline rating prediction. Additionally, an analysis of the CTA-specific behavioral features brings insights into this setting and can be used to bootstrap future systems.
Original languageEnglish
Title of host publicationSIGIR 2023
Subtitle of host publicationProceedings of the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval
Place of PublicationNew York
PublisherAssociation for Computing Machinery, Inc
Pages2314-2318
Number of pages5
ISBN (Print) 978-1-4503-9408-6
DOIs
Publication statusPublished - 19 Jul 2023
Event46th International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR 2023 - Taipei, Taiwan, Province of China
Duration: 23 Jul 202327 Jul 2023

Publication series

NameIR: Research and Development in Information Retrieval
PublisherAssociation for Computing Machinery

Conference

Conference46th International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR 2023
Country/TerritoryTaiwan, Province of China
CityTaipei
Period23/07/2327/07/23

Keywords

  • Conversational Task Assistants
  • NLP
  • Rating Prediction

Cite this