There's a lot of work now on LLM watermarking. But can we extend this to transformers trained for autoregressive image generation? Yes, but it's not straightforward 🧵(1/10)

Thrilled to share a major step forward for AI for mathematical proof generation! We are releasing the Open Proof Corpus: the largest ever public collection of human-annotated LLM-generated math proofs, and a large-scale study over this dataset!

🚨 AI agents wrote 7% of all GitHub PRs in June. But can we trust their code? We built Agents in the Wild – a live dashboard tracking autonomous AI agents across GitHub to answer that question: insights.logicstar.ai Here’s what we learned from analyzing 10M+ PRs 👇 1/n 🧵

SRI Lab is proud to present 14 of our works on Privacy and AI Safety at #ICML2025 this year (9 main conference, 5 workshop). Check out our overview below as well as the individual posts for each. Looking forward to seeing you at the conference! Open for more ⬇️

Unfortunately I couldn't travel to ICML, but my amazing colleagues will be there to present our papers on security attacks and evaluations of LLMs. In the second poster session today, we show that injecting a 5-token comment can steer Copilot towards generating insecure code!

If you are at #ICML2025 today, catch my amazing coauthors to chat about the security risks of LLM quantization!

We (well, not me, I am stuck in ZH) are presenting BaxBench at #ICML2025 from 4:30PM to 7PM in East Exhibition Hall A-B #E-806 as a spotlight💡. Come by and say hi to Niels Mündler, Nikola Jovanović, Jingxuan He, Veselin Raychev, and Baxi, our security inspector beaver.🦫

We just released the evaluation of LLMs on the 2025 IMO on MathArena! Gemini scores best, but is still unlikely to achieve the bronze medal with its 31% score (13/42). 🧵(1/4)

With the main track of #ICML2025 behind us, it is time for the cutting-edge workshops! The SRI Lab together with the INSAIT Institute is proud to present two papers at the AI4Math workshop by the matharena.ai team! Details in the 🧵

Impressive performance of GPT OSS on MathArena, taking shared first place on the final-answer comps! **Very important** note: we ended up running the models locally, as APIs are unreliable at this time. Do not trust benchmark results ran with APIs 🧵

How can we force Diffusion LLMs to adhere to strict rules, like JSON schemas or C++ syntax? We present the first work able to guarantee syntactic correctness for diffusion model outputs for any Context-Free Language! ⛓️ 🤖 A thread 🧵

Introducing MathArena Apex: A set of curated final-answer problems from recent competitions that even best LLMs still can't solve. Top models are correct at most 5% of the time🧵 (1/8)

🚀 We are delighted to release MamayLMv1.0 - the first open and efficient multimodal LLM for Ukrainian that can handle both text and visual data! 📊 MamayLMv1.0 outperforms up to 5x larger open models on Ukrainian tests, maintains strong English skills  and surpasses proprietary

🚀Releasing MamayLM v1.0 🇺🇦 MamayLM can now see! 👀 The new v1.0 version now has visual and enhanced long context capabilities, showcasing even stronger performance on Ukrainian and English languages.

If you're curious about language model watermarking and diffusion language models, you should check out my new work 😌 We propose the first watermarking scheme tailored for diffusion while using the same Red-Green watermark detector 🧵

How LLM Pruning Methods can be Maliciously Exploited? In this work, we investigate for the first time whether pruning can be exploited by an adversary to covertly trigger malicious behavior. Specifically, we demonstrate that an adversary can construct a model that appears

🚨 Be careful when pruning an LLM! 🚨 Even when the model appears benign, it might start behaving maliciously (e.g., jailbroken) once you download and prune it. Here’s how our attack works 🧵 arxiv.org/abs/2510.07985

Pruning can make a normal looking LLM turn harmful only after users prune it. i.e. pruning itself can trigger hidden backdoors at deployment. Pruning zeros many small weights to save memory and speed, and vLLM makes that step easy for deployments. The attack estimates which

MathArena goes visual: We evaluated models such as GPT-5 on Math Kangaroo 2025, a recent contest for ages 6-19 where most tasks require visual reasoning. Models struggle the most with tasks for younger kids. For example, they get this task for 1st graders only 3% of the time 🧵

I have created a small website to help explain my latest work on watermarking diffusion models. There is also a satisfying Manim animation for visualization 😌 diffusionlm-watermark.ing