Have you ever wondered why Deep Learning does not publish formal algorithms for their models while other CS disciplines such as Reinforcement Learning do? After 5 years of Transformers, here's some hopefully useful pseudocode: arxiv.org/abs/2207.09238

As my internship at @MetaAI comes to an end, I want to say a big thank you to my host Adina Williams, as well as Dieuwke Hupkes and Shubham Toshniwal. It's been great having the opportunity to work with you and hopefully there will be chances for more collaboration in the future 😊

To what extent do neural networks learn compositional behaviour? Together with Naomi Saphra, Jon Rawski, @ryandcotterell and Adina Williams we take a lesson from formal language theory to answer this question. arxiv.org/abs/2208.08195

We need to stop conflating open/gated access and opensource. ChatGPT is *not* open source -- we don't know what model is under the hood, how it works, or any other tweaks/filters that are applied. (1/n)

As an AI researcher, I find the glorification of closed-source, proprietary models problematic. We should be emphasizing sharing and open-sourcing AI models, datasets and code, be it in conferences or in the press (4/4).

A recent work from Iddo Drori claimed GPT4 can score 100% on MIT's EECS curriculum with the right prompting. My friends and I were excited to read the analysis behind such a feat, but after digging deeper, what we found left us surprised and disappointed. dub.sh/gptsucksatmit 🧵

Update: we've started replicating their experiments directly with GPT4 calls, and somehow it only gets worse. We've finished running zero-shot GPT 4 on the dataset, and after hand grading the first 30% of the dataset, the results don't seem to match the paper. 🧵

How much does an LM depend on information provided in-context vs its prior knowledge? Check out how Vésteinn Snæbjarnarson, Niklas Stoehr, Jennifer White, Aaron Schein, @ryandcotterell + I answer this by measuring a *context's persuasiveness* and an *entity's susceptibility*🧵