Michael C. Frank (@mcxfrank) 's Twitter Profile
Michael C. Frank

@mcxfrank

Cognitive scientist at Stanford. Open science advocate. @stanfordsymsys director. Bluegrass picker, slow runner, dad.

ID: 598904227

linkhttp://www.stanford.edu/~mcfrank/ calendar_today04-06-2012 03:42:10

8,8K Tweet

14,14K Takipçi

1,1K Takip Edilen

Michael C. Frank (@mcxfrank) 's Twitter Profile Photo
Michael C. Frank
@mcxfrank

2 years ago

People are testing large language models (LLMs) on their "cognitive" abilities - theory of mind, causality, syllogistic reasoning, etc. Many (most?) of these evaluations are deeply flawed. To evaluate LLMs effectively, we need some principles from experimental psychology.🧵

People are testing large language models (LLMs) on their "cognitive" abilities - theory of mind, causality, syllogistic reasoning, etc. Many (most?) of these evaluations are deeply flawed. To evaluate LLMs effectively, we need some principles from experimental psychology.🧵