Michal Golovanevsky
@michalgolov
CS PhD student @BrownCSDept | Multimodal Learning | Mechanistic Interpretability | Clinical Deep Learning.
ID: 1573399875278049280
https://github.com/michalg04 23-09-2022 19:52:33
22 Tweet
32 Followers
42 Following
If SOTA models fail to recognize simple shapes, should we be evaluating them on complex geometric tasks? Most MLLMs struggle with counting the number of sides of regular polygons and all MLLMs receive 0% on novel shapes. William Rudman Amir Bar Vedant Palit [1/6]
![If SOTA models fail to recognize simple shapes, should we be evaluating them on complex geometric tasks? Most MLLMs struggle with counting the number of sides of regular polygons and all MLLMs receive 0% on novel shapes. <a href="/WilliamRudmanjr/">William Rudman</a>
<a href="/_amirbar/">Amir Bar</a> <a href="/vedantpalit1008/">Vedant Palit</a> [1/6]](https://pbs.twimg.com/media/GlNaPBbXMAAYNOh.jpg "Michal Golovanevsky (@michalgolov) on Twitter photo If SOTA models fail to recognize simple shapes, should we be evaluating them on complex geometric tasks? Most MLLMs struggle with counting the number of sides of regular polygons and all MLLMs receive 0% on novel shapes. <a href=\"/WilliamRudmanjr/\">William Rudman</a>
<a href=\"/_amirbar/\">Amir Bar</a> <a href=\"/vedantpalit1008/\">Vedant Palit</a> [1/6]")