Kevin Lu (@kevinlu4588) 's Twitter Profile
Kevin Lu

@kevinlu4588

Diffusion Research @ Northeastern

@ CVPR2025

ID: 1774996041466777600

calendar_today02-04-2024 03:03:40

10 Tweet

33 Takipçi

68 Takip Edilen

Sheridan Feucht (@sheridan_feucht) 's Twitter Profile Photo
Sheridan Feucht
@sheridan_feucht

6 months ago

[📄] Are LLMs mindless token-shifters, or do they build meaningful representations of language? We study how LLMs copy text in-context, and physically separate out two types of induction heads: token heads, which copy literal tokens, and concept heads, which copy word meanings.

[📄] Are LLMs mindless token-shifters, or do they build meaningful representations of language? We study how LLMs copy text in-context, and physically separate out two types of induction heads: token heads, which copy literal tokens, and concept heads, which copy word meanings.