Crowdsourced Video Subtitling with Adaptation Based on User-Corrected Lattices

João Miranda, Ramón Astudillo, Ângela Costa, André Silva, Hugo Silva, João Graça, Bhiksha Raj

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

This paper investigates an approach for fast hybrid human and machine video subtitling based on lattice disambiguation and posterior model adaptation. The approach aims at correcting Automatic Speech Recognition (ASR) transcriptions requiring minimal effort from the user and facilitating user corrections from smart-phone devices. Our approach is based on three key concepts. Firstly, only a portion of the data is sent to the user for correction. Secondly, user action is limited to selecting from a fixed set of options extracted from the ASR word lattice. Thirdly, user feedback is used to update the ASR parameters and further enhance performance. To investigate the potential and limitations of this approach, we carry out experiments employing simulated and real user corrections of TED talks videos. Simulated corrections include both the true reference and the best combination of the options shown to the user. Real corrections are obtained from 30 editors through a special purpose web-interface displaying the options for small video segments. We analyze the fixed option approach and the trade-off between model adaptation and increasing the amount of corrected data.
Original languageEnglish
Title of host publicationInternational Conference on Advances in Speech and Language Technologies for Iberian Languages
EditorsAlberto Abad
Place of PublicationLisbon
PublisherSpringer
Pages138-147
Number of pages10
Volume10077
ISBN (Print)978-3-319-49168-4
DOIs
Publication statusPublished - 2016
EventInternational Conference on Advances in Speech and Language Technologies for Iberian Languages - Lisbon, Portugal
Duration: 23 Nov 201625 Nov 2016

Publication series

NameLecture Notes In Computer Science
PublisherSpringer
Volume10077

Conference

ConferenceInternational Conference on Advances in Speech and Language Technologies for Iberian Languages
Abbreviated titleIberSPEECH 2016
CountryPortugal
CityLisbon
Period23/11/1625/11/16

Keywords

  • Automatic Speech Recognition
  • Error Analysis

Fingerprint Dive into the research topics of 'Crowdsourced Video Subtitling with Adaptation Based on User-Corrected Lattices'. Together they form a unique fingerprint.

Cite this