LLM2Vec abstract

piano afro-rock

April 11th, 2024suno

Lyrics

[Bridge] [Verse 1] Large decoder-only language models LLMs are the state-of-the-art models on most of today’s NLP tasks [Verse 2] Yet, the community is only slowly adopting these models for text embedding tasks which require rich contextualized representations. [Verse 3] In this work, we introduce LLM2Vec, a simple unsupervised approach that can transform any decoder-only LLM into a strong text encoder. [Chorus] LLM2Vec consists of three simple steps: One. Enabling bidirectional attention Two. Masked next token prediction, and Three. unsupervised contrastive learning. [Verse 4] We demonstrate the effectiveness of LLM2Vec by applying it to three popular LLMs [Verse 5] We outperform encoder-only models by a large margin on word-level tasks and reach a new unsupervised state-of-the-art performance on the Massive Text Embeddings Benchmark MTEB. [Verse 6] Moreover, when combining LLM2Vec with supervised contrastive learning, we achieve state-of-the-art performance on MTEB [Chorus] LLM2Vec consists of three simple steps: One. Enabling bidirectional attention Two. Masked next token prediction, and Three. unsupervised contrastive learning. Three. unsupervised contrastive learning. []Verse 7] Our strong empirical results and extensive analysis demonstrate that LLMs can be effectively transformed [Verse 8] into universal text encoders in a parameter-efficient manner without the need for expensive adaptation or synthetic GPT-4 generated data. [Chorus] LLM2Vec consists of three simple steps: One. Enabling bidirectional attention Two. Masked next token prediction, and Three. unsupervised contrastive learning.

Recommended

Midnight Chase
Midnight Chase

industrial drum and bass edgy

บ้านสมานมิตร
บ้านสมานมิตร

marching ,orchestral , epic

Lady with the Phoenix Tatoo
Lady with the Phoenix Tatoo

Soft Rock, Acoustic Guitar, Bass, Acoustic Drums

Allein in der Nacht
Allein in der Nacht

ruhig hip hop trompete

يا زمن
يا زمن

emotional electropop, epic vocalist

Beachside Serenade
Beachside Serenade

rock,pop rock,alternative rock,power pop,energetic

The Return of the Epic Warrior
The Return of the Epic Warrior

epic folk black metal acoustic

Shattered Echoes
Shattered Echoes

duet pop fast-paced

Midnight Dance
Midnight Dance

pop upbeat electronic

Серце Моє
Серце Моє

Russian Glitch Hop, grunge-dramatic, ambient-pop, male vocals, epic, orchestral, acoustic,

Η γοργόνα
Η γοργόνα

Progressive House at 128 BPM, Male Vocals

jeeee
jeeee

rock

Veselá Postel
Veselá Postel

melodic pop

Au-Delà des Limites
Au-Delà des Limites

emotional techno

Ok vv
Ok vv

grunge