LLM2Vec abstract

piano afro-rock

April 11th, 2024suno

Lyrics

[Bridge] [Verse 1] Large decoder-only language models LLMs are the state-of-the-art models on most of today’s NLP tasks [Verse 2] Yet, the community is only slowly adopting these models for text embedding tasks which require rich contextualized representations. [Verse 3] In this work, we introduce LLM2Vec, a simple unsupervised approach that can transform any decoder-only LLM into a strong text encoder. [Chorus] LLM2Vec consists of three simple steps: One. Enabling bidirectional attention Two. Masked next token prediction, and Three. unsupervised contrastive learning. [Verse 4] We demonstrate the effectiveness of LLM2Vec by applying it to three popular LLMs [Verse 5] We outperform encoder-only models by a large margin on word-level tasks and reach a new unsupervised state-of-the-art performance on the Massive Text Embeddings Benchmark MTEB. [Verse 6] Moreover, when combining LLM2Vec with supervised contrastive learning, we achieve state-of-the-art performance on MTEB [Chorus] LLM2Vec consists of three simple steps: One. Enabling bidirectional attention Two. Masked next token prediction, and Three. unsupervised contrastive learning. Three. unsupervised contrastive learning. []Verse 7] Our strong empirical results and extensive analysis demonstrate that LLMs can be effectively transformed [Verse 8] into universal text encoders in a parameter-efficient manner without the need for expensive adaptation or synthetic GPT-4 generated data. [Chorus] LLM2Vec consists of three simple steps: One. Enabling bidirectional attention Two. Masked next token prediction, and Three. unsupervised contrastive learning.

Recommended

No Enemies
No Enemies

Classic

Midnight reverie
Midnight reverie

philly soul funk disco orchestra brass vibraphone Strings

"Jazz Java Jive"
"Jazz Java Jive"

Create a lo-fi bossa nova jazz track with saxophone and soft piano for a café vibe, BPM70.

Красный Фломастер
Красный Фломастер

акустический мелодичный поп

jk
jk

dubstep, rock, metal, guitar

Dancin' in the Rain
Dancin' in the Rain

groovy smooth mellow funk

流浪的小狗
流浪的小狗

acoustic pop emotional

multi
multi

hard rock, drum and bass, bass, pop, pop, electro, violin, synth, groovy, bass

Доброта
Доброта

deathmetal, shout, femalescreamer, deathcore, metalsinger, growl, aggretsuko, screaming shots, accordion, female singer

Rain of My Heart
Rain of My Heart

hard rock heavy metal classic rock

Hymn to the Fallen Star
Hymn to the Fallen Star

male vocalist,rock,metal,alternative metal,progressive metal,heavy,dark,passionate,energetic,heavy metal,aggressive

Жизнь Это Война
Жизнь Это Война

Bounce drop, futuristic

海参崴之恋
海参崴之恋

Narrative, Epic,ballad, piano, guitar,keyboard, heavy, tragic.

하루의 기적
하루의 기적

hip-hop happy pop

Nobody But You
Nobody But You

melodic pop