@tannguyen689 : In our recent paper accepted at NeurIPS 25024, my student, Rachel Si Yan Teo (@rachtsy_), uncovered a link between self-attention in transformers and kernel PCA. Specifically, Rachel demonstrates that self-attention projects its query vectors onto the principal component axes of • TwiCopy

Tan Minh Nguyen

@tannguyen689

+ Follow

Assistant Professor of Mathematics (Presidential Young Professor) at the National University of Singapore (@NUSingapore). #DeepLearning, #RobustAI, #ScalableAI

ID: 705500312778309632

linkhttps://tanmnguyen89.github.io/ calendar_today03-03-2016 21:09:16

253 Tweet

1,1K Followers

733 Following

Tan Minh Nguyen

@tannguyen689

7 months ago

In our recent paper accepted at NeurIPS 2024, my student, Rachel Si Yan Teo (rachtsy), uncovered a link between self-attention in transformers and kernel PCA. Specifically, Rachel demonstrates that self-attention projects its query vectors onto the principal component axes of

In our recent paper accepted at NeurIPS 2024, my student, Rachel Si Yan Teo (<a href="/rachtsy_/">rachtsy</a>), uncovered a link between self-attention in transformers and kernel PCA.

Specifically, Rachel demonstrates that self-attention projects its query vectors onto the principal component axes of

thumb_up_off_alt798

chat_bubble_outline6

repeat113

shareShare