A Brief Overview of Formal Body-Movement Annotations in Gesture Studies: From Schemes to Validation

Formal analyses of human movement are intended to be - by nature - objective. Despite the great interest in Gesture Studies of descibing gesture and other body movement forms, there is no one standardized guideline for the formal transcription nor annotation of body movementes. Thus, scienctists are left to adapt versions of pre-existing annotation schemes or develop their own. This paper aims at providing an overview of the gamut of annotation schemes used in the multimodal communication literatura in order to raise questions about how researchers define, treat and analyze body movements in their data. Differences of definitions cause problems when it comes to comparing research findings and are directly connected to the question of how body movement units are identified and classified by the research community. On the one hand, there is the problem of formal and functional labeling that are often collapsed in the adopted annotation scheme; on the other, definitional diversity affects humans raters' evaluation and judgmnt, not to mention differences in the annotation process when marking start- and end-points of a movement unit. Another issue is that human movement is often studied in a fragmented manner, where researchers (without blame) focus only on certain articulatores and not others. Whereas manual gestures are most studied within the the fiel, and some attempts have been undertaken for its standardization (inter alia Bressem, Ladewig, & Müller 2013; Lausberg & Sloetjes 2009), a structured annotation guideline for other articulators' expressions has yet to be reached (cf. "head-gestures annotation schemes": Kousidis 2013; Poggy 2010; Heylen 2008; Cerrato 20017; Allwood & Cerrato 2003). Consequently, a comprehensive annotation scheme containing all body articulators is unavailable to gesture researchers, although within the performing arts domain we find an example of that type (i.e. Laban movement analyses and notation). Besides the problems of definig the movement units and the segmentation issues, researchers also face obstacles in processing the data and the estimation of their reliability and validity. Already the widely used statistical coefficients for the measurement of inter-rater agreement (i.e. Fleiss' kappa, Krippendorff's alfa, Cohen's kappa), are problematic for this field (McHugh 2012) and are not aways included in the statistical evaluation exactly. Some researchers claim that a statistical calculation of agreement is not mandatory (e.g. Stelma & Cameron 2007). This presentation intends to provide more questions than answers, but at the same time provide suggestions to scientists tackling the questions of how to perform formal studies of human movements.