At ICANN 1993, I extended my 1991 unnormalised linear Transformer, introduced attention terminology for it, & published the 'self-referential weight matrix.' 3 decades later, they made me Chair of ICANN 2024 in Lugano. Call for papers (deadline March 25): e-nns.org/icann2024/call…

Our #GPTSwarm models Large Language Model Agents and swarms thereof as computational graphs reflecting the hierarchical nature of intelligence. Graph optimization automatically improves nodes and edges. arxiv.org/abs/2402.16823
github.com/metauto-ai/GPT…
gptswarm.org

In 2016, at an AI conference in NYC, I explained artificial consciousness, world models, predictive coding, and science as data compression in less than 10 minutes. I happened to be in town, walked in without being announced, and ended up on their panel. It was great fun.…

Numbers are lining up: one tweet per year of my life; one follower per LSTM citation 👍🙂

2010 foundations of recent $NVDA stock market frenzy: our simple but deep neural net on NVIDIA GPUs broke MNIST arxiv.org/abs/1003.0358. Things are changing fast. Just 7 months ago, I tweeted: compute is 100x cheaper, $NVDA 100x more valuable. Today, replace '100' by '250.'…

The GOAT of tennis Novak Djokovic said: '35 is the new 25.” I say: “60 is the new 35.” AI research has kept me strong and healthy. AI could work wonders for you, too!

Best paper award for 'Mindstorms in Natural Language-Based Societies of Mind' at #NeurIPS2023 WS Ro-FoMo. Up to 129 foundation models collectively solve practical problems by interviewing each other in monarchical or democratic societies arxiv.org/abs/2305.17066

How 3 Turing awardees republished key methods and ideas whose creators they failed to credit. More than a dozen concrete AI priority disputes under people.idsia.ch/~juergen/ai-pr…

So Yann LeCun: 'I've been advocating for deep learning architecture capable of planning since 2016' vs me: 'I've been publishing deep learning architectures capable of planning since 1990.' I guess in 2016 Yann LeCun also picked up the torch. (References attached)…

Q*? 2015: reinforcement learning prompt engineer in Sec. 5.3 of “Learning to Think...” arxiv.org/abs/1511.09249. A controller neural network C learns to send prompt sequences into a world model M (e.g., a foundation model) trained on, say, videos of actors. C also learns to…

Amazing that Jürgen Schmidhuber gave this talk back in 2012, months before AlexNet paper was published.

In 2012, many things he discussed, people just considered to be funny and a joke, but the same talk now would be considered at the center of AI debate and controversy.

Full talk:

AI boom v AI doom: since the 1970s, I have told AI doomers that in the end all will be good. E.g., 2012 TEDx talk: youtu.be/KQ35zNlyG-o: “Don’t think of us versus them: us, the humans, v these future super robots. Think of yourself, and humanity in general, as a small stepping…

Thanks Elon Musk for your generous hyperbole! 

Admittedly, however, I didn’t invent sliced bread, just  #GenerativeAI  and things like that: people.idsia.ch/~juergen/most-…

And of course my team is standing on the shoulders of giants: people.idsia.ch/~juergen/deep-…

Original tweet by Elon Musk:…

Silly AI regulation hype

One cannot regulate AI research, just like one cannot regulate math.

One can regulate applications of AI in finance, cars, healthcare. Such fields already have continually adapting regulatory frameworks in place.

Don’t stifle the open-source movement!…

2023: 20th anniversary of the Gödel Machine, a mathematically optimal, self-referential, meta-learning, universal problem solver making provably optimal self-improvements by rewriting its own computer code people.idsia.ch/~juergen/goede…

Unlike diffusion models, Bayesian Flow Networks operate on the parameters of data distributions, rather than on noisy versions of the data itself. I think this paper by Alex Graves et al. will be influential.

In 2010, we used Jensen Huang's NVIDIA GPUs to show that deep feedforward nets can be trained by plain backprop without any unsupervised pretraining. In 2011, our DanNet was the first superhuman CNN. Today, compute is 100+ times cheaper, and NVIDIA 100+ times more valuable.…

Meta used my 1991 ideas to train LLaMA 2, but made it insinuate that I “have been involved in harmful activities” and have not made “positive contributions to society, such as pioneers in their field.” Meta & LLaMA promoter Yann LeCun should correct this ASAP. See…

Interview with Jürgen Schmidhuber, ‘Father Of Modern AI’, who says his life’s work won’t lead to dystopia.

He goes quite in depth into his views about the future of AI and AGI. A refreshing view of things, compared to other ‘influential leaders’ in AI.

reddit.com/r/MachineLearn…

Instead of trying to defend his paper on OpenReview (where he posted it), Yann LeCun made misleading statements about me in popular science venues. I am debunking his recent allegations in the new Addendum III of my critique people.idsia.ch/~juergen/lecun…