Transformers as Soft Reasoners over Language "we explore whether transformers can similarly learn to reason (or emulate reasoning), but using rules expressed in language, thus bypassing a formal representation." arxiv.org/abs/2002.05867 Datasets and demo: rule-reasoning.apps.allenai.org

#ICLR2020 paper on Differentiable Reasoning over a Virtual Knowledge Base: Efficient, end-to-end differentiable framework for doing complex multi-hop QA over a large text corpus. arxiv.org/abs/2002.10640 w/t Dhingra, Zaheer, Balachandran, Graham Neubig , William Cohen

We will present a tutorial on Machine Learning for Drug Development at #IJCAI2020! Materials to follow on our website: zitniklab.hms.harvard.edu/drugml IJCAIconf #drugs #networks #AI

Happy to share MARGE, our new work on rethinking pre-training: given a document, we first retrieve related documents, and then paraphrase these to reconstruct the original. MARGE works well for generation and classification in many languages, sometimes without supervision. (1/6)

We would like to wish you all a restful winter break, and best wishes for the new year 🙂🎄

Introducing ChatArena 🏟 - a Python library of multi-agent language game environments that facilitates communication and collaboration between multiple large language models (LLMs)! 🌐🤖 Check out our GitHub repo: github.com/chatarena/chat… #ChatArena #NLP #AI #LLM 1/8 🧵

🚨💫We are delighted to have Shishir Patil at our UCL Computer Science NLP Meetup *Monday 1st Nov 6:30pm GMT* The event will be *hybrid* Due to room capacity, there are *two links* to sign up depending on whether you attend in person or online Details in: meetup.com/ucl-natural-la…

David Attenborough is now narrating my life Here's a GPT-4-vision + ElevenLabs python script so you can star in your own Planet Earth:

LLMs are now trained >1000x as much language data as a child, so what happens when you train a "BabyLM" on just 100M words? The proceedings of the BabyLM Challenge are now out along with our summary of key findings from 31 submissions: aclanthology.org/volumes/2023.c… Some highlights 🧵

Often prompt engineering focuses on the *content* of the prompt, but in reality *formatting* of the prompt can have an equal or larger effect, especially for less powerful models. This is a great deep dive into this phenomenon by Melanie Sclar et al.

Congrats on the launch! LFG Yuxiang (Jimmy) Wu Zhengyao Jiang

We're excited to announce AIDE has become the first human-level AI agent for data science! AIDE outperforms half of human data scientists on a wide range of Kaggle competitions, surpassing conventional AutoML, LangChain agents, and ChatGPT with human assistance. 🏆

🍷 FineWeb technical report is out and so is 📚 FineWeb-Edu, a 1.3 trillion tokens dataset that outperforms all other open web datasets, with remarkable improvements on educational benchmarks such as MMLU, ARC, and OpenBookQA. Technical report: hf.co/spaces/Hugging… Dataset:

They say a picture is worth a thousand words... but work led by ralphtang.eth finds words worth a thousand pictures! arxiv.org/abs/2406.08482

🎉🎉🏆

🚨 New Paper 🚨 Can LLMs perform latent multi-hop reasoning without exploiting shortcuts? We find the answer is yes – they can recall and compose facts not seen together in training or guessing the answer, but success greatly depends on the type of the bridge entity (80%+ for