Variations in Word Usage for the Financial Domain

Syrielle Montariol (1, 2, 3) , Alexandre Allauzen (4) , Asanobu Kitamoto (2)

1 Société Générale
2 NII - National Institute of Informatics
3 ILES - Information, Langue Ecrite et Signée
4 ESPCI Paris - Ecole Superieure de Physique et de Chimie Industrielles de la Ville de Paris

Natural languages are dynamic systems; the way words are used vary depending on many factors, mirroring the divergences of various aspects of the society. Recent approaches to detect these variations through time rely on static word embedding. However the recent and fast emergence of contextualised models challenges the field and beyond. In this work, we propose to leverage the capacity of these new models to analyse financial texts along two axes of variation: the diachrony (temporal evolution), and synchrony (variation across sources and authors). Indeed, financial texts are characterised by many domain-specific terms and entities whose usage is subject to high variations, reflecting the disparity and evolution of the opinion and situation of financial actors. Starting from a corpus of annual company reports and central bank statements spanning 20 years, we explore in this paper the ability of the language model BERT to identify variations in word usage in the financial domain, and propose a method to interpret these variations.

Domaines: Informatique et langage [cs.CL]

Syrielle Montariol, Alexandre Allauzen, Asanobu Kitamoto. Variations in Word Usage for the Financial Domain. Second Workshop on Financial Technology and Natural Language Processing, Jan 2021, Virtual conference, France. ⟨hal-04421686⟩