Multimodal Web Based Video Annotator with Real-Time Human Pose Estimation

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

This paper presents a multi-platform Web-based video annotator to support multimodal annotation that can be applied to several working areas, such as dance rehearsals, among others. The CultureMoves’ “Motion-Notes” Annotator was designed to assist the creative and exploratory processes of both professional and amateur users, working with a digital device for personal annotations. This prototype is being developed for any device capable of running in a modern Web browser. It is a real-time multimodal video annotator based on keyboard, touch and voice inputs. Five different ways of adding annotations have been already implemented: voice, draw, text, web URL, and mark annotations. Pose estimation functionality uses machine learning techniques to identify a person skeleton in the video frames, which gives the user another resource to identify possible annotations.
Original languageEnglish
Title of host publicationIntelligent Data Engineering and Automated Learning: IDEAL 2019 - 20th International Conference, Proceedings
EditorsHujun Yin, Richard Allmendinger, David Camacho, Peter Tino, Antonio J. Tallón-Ballesteros, Ronaldo Menezes
PublisherSpringer
Pages23-30
Number of pages8
ISBN (Print)9783030336165
DOIs
Publication statusPublished - 2019
Event20th International Conference on Intelligent Data Engineering and Automated Learning, IDEAL 2019 - Manchester, United Kingdom
Duration: 14 Nov 201916 Nov 2019

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume11872 LNCS
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349

Conference

Conference20th International Conference on Intelligent Data Engineering and Automated Learning, IDEAL 2019
CountryUnited Kingdom
CityManchester
Period14/11/1916/11/19

Keywords

  • Machine learning for creativity
  • Multimodal video annotations
  • Real-time human pose estimation

Fingerprint Dive into the research topics of 'Multimodal Web Based Video Annotator with Real-Time Human Pose Estimation'. Together they form a unique fingerprint.

  • Cite this

    Rodrigues, R., Madeira, R. N., Correia, N., Fernandes, C., & Ribeiro, S. (2019). Multimodal Web Based Video Annotator with Real-Time Human Pose Estimation. In H. Yin, R. Allmendinger, D. Camacho, P. Tino, A. J. Tallón-Ballesteros, & R. Menezes (Eds.), Intelligent Data Engineering and Automated Learning: IDEAL 2019 - 20th International Conference, Proceedings (pp. 23-30). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 11872 LNCS). Springer. https://doi.org/10.1007/978-3-030-33617-2_3