
LLM2Vec abstract
piano afro-rock
April 11th, 2024suno
Lyrics
[Bridge]
[Verse 1]
Large decoder-only language models
LLMs
are the state-of-the-art models
on most of today’s NLP tasks
[Verse 2]
Yet, the community is only slowly
adopting these models
for text embedding tasks
which require rich
contextualized representations.
[Verse 3]
In this work, we introduce LLM2Vec,
a simple unsupervised approach that
can transform any decoder-only LLM
into a strong text encoder.
[Chorus]
LLM2Vec consists of three simple steps:
One. Enabling bidirectional attention
Two. Masked next token prediction, and
Three. unsupervised contrastive learning.
[Verse 4]
We demonstrate the effectiveness of
LLM2Vec
by applying it to three popular LLMs
[Verse 5]
We outperform encoder-only models
by a large margin on word-level tasks
and reach a new unsupervised
state-of-the-art performance
on the Massive Text Embeddings Benchmark
MTEB.
[Verse 6]
Moreover, when combining
LLM2Vec
with supervised contrastive learning,
we achieve state-of-the-art performance
on MTEB
[Chorus]
LLM2Vec consists of three simple steps:
One. Enabling bidirectional attention
Two. Masked next token prediction, and
Three. unsupervised contrastive learning.
Three. unsupervised contrastive learning.
[]Verse 7]
Our strong empirical results
and extensive analysis
demonstrate that
LLMs can be effectively transformed
[Verse 8]
into universal text encoders
in a parameter-efficient manner
without the need
for expensive adaptation or
synthetic GPT-4 generated data.
[Chorus]
LLM2Vec consists of three simple steps:
One. Enabling bidirectional attention
Two. Masked next token prediction, and
Three. unsupervised contrastive learning.
Recommended

Зрелая женщина
D`n`B

Midnight Chase
industrial drum and bass edgy

บ้านสมานมิตร
marching ,orchestral , epic

Lady with the Phoenix Tatoo
Soft Rock, Acoustic Guitar, Bass, Acoustic Drums

Allein in der Nacht
ruhig hip hop trompete

يا زمن
emotional electropop, epic vocalist

Guerrero del conocimiento
Power metal
Beachside Serenade
rock,pop rock,alternative rock,power pop,energetic

The Return of the Epic Warrior
epic folk black metal acoustic

Shattered Echoes
duet pop fast-paced

Midnight Dance
pop upbeat electronic

Серце Моє
Russian Glitch Hop, grunge-dramatic, ambient-pop, male vocals, epic, orchestral, acoustic,

Η γοργόνα
Progressive House at 128 BPM, Male Vocals

jeeee
rock

Veselá Postel
melodic pop

Au-Delà des Limites
emotional techno

Ok vv
grunge