hallucinations-nmt

[Verse] In this work, we study hallucinations in Neural Machine Translation, which lie at an extreme end on the spectrum of NMT pathologies. [Chorus] [aaa aaa hallucinations aaa aaa] [Chorus] Firstly, we connect the phenomenon of hallucinations under source perturbation to the Long-Tail theory of Feldman, and present an empirically validated hypothesis that explains hallucinations under source perturbation. [Chorus] [hmm hmm hallucinations hmm hmm] [Chorus] Secondly, we consider hallucinations under corpus-level noise, without any source perturbation and demonstrate that two prominent types of natural hallucinations -- detached and oscillatory outputs could be generated and explained through specific corpus-level noise patterns. [Chorus] [aaa aaa hallucinations aaa aaa] [Chorus] [Verse] Finally, we elucidate the phenomenon of hallucination amplification in popular data-generation processes such as Backtranslation and sequence-level Knowledge Distillation. [Chorus] [hmm hmm hallucinations hmm hmm] [Chorus]