I am looking for a student research assistant for "Continual Learning ♾ and resource-efficient 🐁learning in NLP 📰 (or vision)". 20h/week. Remote: applications from anywhere in Germany. Local: Berlin or Potsdam. For details follow the link. bit.ly/3j3b3Zf

Mark Riedl that's why very large pre-trained models should be called "chonk"

BSc: This is the answer to this problem. MSc: How do you answer this slightly different problem? PhD: I don't know man, people disagree about a lot of things. I've been wrong more than I've been right, man. I just wanna knit or something

𝐀𝐬 𝐚 𝐤𝐢𝐝: My parents to me: "Don't believe everything you read on the internet." 𝐀𝐬 𝐚𝐧 𝐚𝐝𝐮𝐥𝐭: Me to my parents: "Don't believe everything you read on the internet." -- meme found on the internet so it must be true

Does anybody know of research that shows that large language models can actually do "out of pretraining domain" zero-shot prediction? If we pretrain on diverse texts, would we not expect to always be "partially in domain". So does 'out of domain ZS prediction' exist?

Primer: Searching for Efficient Transformers for Language Modeling arxiv.org/abs/2109.08668 We use evolution to design a new Transformer variant, called Primer. Primer has a better scaling law, and is 3X to 4X faster for training than Transformer for language modeling.

Some LM pretraining tricks for the knowledge tool box. Primer-EZ: Use a better activation function (RelU**2) + a CNN layer in MHSA. Makes large language model pretraining more efficient and thus a bit more accessible. This is great news.

Wondering how to connect Causality & NLP? Check out Bernhard Schölkopf's keynote at #EMNLP2021 EMNLP 2025 Causal Inference & NLP Workshop: youtu.be/Zwt1jJxVSvg

This remains one of my favorite papers. Offers surprising insights into how sparsity impact on generalization is nuanced. While in-distribution, sparsity harms the long-tail and rare features, it can benefit OOD generalization by preventing memorization of irrelevant data.

The CFP for the third insights-workshop.github.io is out! Short papers & abstracts can be submitted directly to the workshop by Feb 28, or through ARR by March 21. Any NLP topic is welcome. /1

More fun publisher surveillance: Elsevier embeds a hash in the PDF metadata that is *unique for each time a PDF is downloaded*, this is a diff between metadata from two of the same paper. Combined with access timestamps, they can uniquely identify the source of any shared PDFs.

How gorgeous are these astrocytes?! They're from an iDISCO+ cleared mouse brain hemisphere. The colors represent depth - white is closest, blue farthest. Interested in how they're connected, and what they're saying? Stay tuned... #WomeninScience #Science #NeuroTwitter #STEM

📣 Looking for a PhD position? 10 PhD fellowships are available via the Danish Data Science Academy ddsa.dk/open-calls/ope… Caveat: applicants need to have graduated from a 2-year MSc programme by the time of application Deadline: 20 March 2022 Get in touch if you're interested!

35. Antiroutine: To create original output, consume unusual input. Avoid trending videos, NYT bestsellers, widely cited papers. Instead. read ignored texts, plumb the past for forgotten ideas. Step outside the zeitgeist so you can see it with fresh eyes. h/t: Michael

@ropeharz Yes, much of what we claim to be "mysterious properties of deep learning" can easily be replicated with the most basic models. We showed double descent (and how closely it's tracked by effective dimension) with a simple linear model as part of arxiv.org/abs/2003.02139.

🗣️I'm looking forward to being on the panel for this Legal Tech Research Talk series . - "When Research Goes Wrong: Deepfakes"! Join us this Thursday in person (cozy fireplace room) or online via Zoom!

Good practical insights on transformer arches.

A great piece in VentureBeat by Sharon Goldman about what we hope to achieve with Cohere For AI. venturebeat.com/2022/06/14/goo…

So, let me get this straight. I won not one, but two American Psychological Association awards. But, in order to accept these awards at the conference, I need to pay nearly $600 in registration fees. No one day pass. No fee wavier. No way to attend a 50 min ceremony. Conferences are a scam. I’m not going.

When your (model) representations did not turn out quite right. #model_understanding #XAI