I just wrote a blog post about the future of language models and expressing concerns.machinethoughts.wordpress.com/2022/07/06/quo…

I have long argued that grounding is not necessary for understanding. I laid out my case against grounding in a response to Browning and LeCun. Yann LeCun Ilya Sutskever Dan Jurafsky Jenny Irwin Percy Liang machinethoughts.wordpress.com/2022/09/11/the…

We're hiring postdocs to work on foundational issues in AI alignment at the University of Chicago. Advised by (any combo of) myself, David McAllester, Chenhao Tan. Get in touch if you'd like to do deep, ambitious work! docs.google.com/document/d/15v… Please RT :)

Michael Douglas and I have been playing with chain of thought prompting for a variant of semantic parsing. The results seem relevant to the grounding hypothesis and the nativism/empiricism debate. Yann LeCun Gary Marcus Percy Liang Christopher Manning Yejin Choi wordpress.com/view/machineth…

I've discovered the secret of general Artificial Intelligence. It just so happens to be answered by my own field decades ago, but it just needed to be synthesized. I see further than everyone else. Follow me and I'll tweet out tidbits of wisdom / trivia at regular intervals.

When I was in high school I read a book on information theory. It was obvious to me at that time that strong modeling of the distribution of language requires uncovering meaning. I continue to be frustrated that this seemingly obvious observation gets so little traction.

RL from human feedback seems to be the main tool for alignment. Given reward hacking and the falliability of humans, this strategy seems bound to produce agents that merely appear to be aligned, but are bad/wrong in subtle, inconspicuous ways. Is anyone else worried about this?

machinethoughts.wordpress.com/2023/02/19/the… A perspective on "sentience" in the context of AI safety and language models. Yann LeCun Noah A. Smith

arxiv.org/abs/2301.11108 A paper on the mathematics of diffusion models that explains the diffusion SDEs --- both forward and backward --- assuming only familiarity with Gaussians. It also gives some original non-variational likelihood formulas.

Just posted an analysis of Wittgenstein's mantra "the meaning is the use". Christopher Manning Yejin Choi machinethoughts.wordpress.com

New LLM in town: ***phi-1 achieves 51% on HumanEval w. only 1.3B parameters & 7B tokens training dataset*** Any other >50% HumanEval model is >1000x bigger (e.g., WizardCoder from last week is 10x in model size and 100x in dataset size). How? ***Textbooks Are All You Need***

Once AI systems become more intelligent than humans, humans we will *still* be the "apex species." Equating intelligence with dominance is the main fallacy of the whole debate about AI existential risk. It's just wrong. Even *within* the human species It's wrong: it's *not* the

I just wrote a blog post on advobots and AI safety. It contains speculations on the future of language model architectures and the relationship between architecture safety. machinethoughts.wordpress.com/2023/09/23/adv…

clem 🤗 It may seem like AI is a brave and different world but it seems to me the same forces we’ve seen in business, software, and open source still apply. Dominant monopolies with technical strength have no incentive to open source (think of Oracle or Microsoft years ago). But open

I just wrote a post on "Guidance and Art". I propose a "semantics" for guidance and predict that a system trained only on photographs will generate drawing-like images when self-guidance is applied. Jason Salavon machinethoughts.wordpress.com/2023/09/29/gui…