profile-img
Shreya Shankar

@sh_reya

I study ML & AI engineers and try to make their lives a little better. PhD-ing in databases & HCI @Berkeley_EECS @UCBEPIC and MLOps-ing around town. She/they.

calendar_today11-01-2014 06:46:16

4,1K Tweets

39,6K Followers

597 Following

Shreya Shankar(@sh_reya) 's Twitter Profile Photo
Shreya Shankar
@sh_reya

3 weeks ago

got screwed by my evals today. lesson: if you rely on a separate labeled dataset for testing changes to your production LLM pipeline, and some prod LLM inputs may consist of errors like typos, bad grammar, different capitalization, etc. the test set needs to reflect this!!

account_circle