Transformers in Automatic Speech Recognition
Résumé
The Transformer, a model relying entirely on the attention mechanism, brought significant improvements in performance on several natural language processing tasks. This chapter presents its impact on the speech processing domain and, more specifically, on the automatic speech recognition task. A short history of the evolution of automatic speech recognition systems is also given. A selection of important works making use of transformers are presented, as well as pretraining selfsupervised architectures.