Describing movement learning using metric learning

Antoine Loriette (1) , Wanyu Liu (2, 3) , Frédéric Bevilacqua (1) , Baptiste Caramiaux (4, 3)

1 STMS - Sciences et Technologies de la Musique et du Son
2 LTCI - Laboratoire Traitement et Communication de l'Information
3 EX-SITU - Extreme Situated Interaction - LISN
4 ISIR - Institut des Systèmes Intelligents et de Robotique

Analysing movement learning can rely on human evaluation, e.g. annotating video recordings, or on computing means in applying metrics on behavioural data. However, it remains challenging to relate human perception of movement similarity to computational measures that aim at modelling such similarity. In this paper, we propose a metric learning method bridging the gap between human ratings of movement similarity in a motor learning task and computational metric evaluation on the same task. It applies metric learning on a Dynamic Time Warping algorithm to derive an optimal set of movement features that best explain human ratings. We evaluated this method on an existing movement dataset, which comprises videos of participants practising a complex gesture sequence toward a target template, as well as the collected data that describes the movements. We show that it is possible to establish a linear relationship between human ratings and our learned computational metric. This learned metric can be used to describe the most salient temporal moments implicitly used by annotators, as well as movement parameters that correlate with motor improvements in the dataset. We conclude with possibilities to generalise this method for designing computational tools dedicated to movement annotation and evaluation of skill learning.

Domaines: Informatique [cs]

Antoine Loriette, Wanyu Liu, Frédéric Bevilacqua, Baptiste Caramiaux. Describing movement learning using metric learning. PLoS ONE, 2023, Plos ONE, 18 (2), ⟨10.1371/journal.pone.0272509⟩. ⟨hal-03790829⟩