2 weeks ago I defended my PhD thesis „Self-improving, open-world robotic scene understanding“. Thank you Roland Siegwart Autonomous Systems Lab, ETH Zürich for amazing 4.5 years in such a great environment and so many possibilities to collaborate with others.

I decided to take this week (mostly) off Twitter just to see what Mastodon was like. I joined an infosec-focused server and I’ve spent the past few days trying to use it like Twitter. Here’s a quick review. 1/

Data capture brings you to many fun places!

If you missed Oliver Lemke @ ICRA 2024 's demo at International Conference on 3D Vision , it will also be presented at IEEE ICRA mobile manipulation workshop and we just released the preprint and made a nice video:

Is scaling data the answer to all semantic perception problems? If you are working on this or similar questions, submit your preliminary results or existing work to our RSS workshop on Open-Set Robot Perception in the Wild! open-set-robotics.github.io Stephany Berrio Robotics: Science and Systems

Today is #RobotXInnovationDay at ETH Zürich Come by our booth and check out our demo! 🦾 cc Hermann Blum Marc Pollefeys CVG

🚨 Spot needs your help!! Can you help us and take some pictures of light switches at home or work?? You can upload them here: forms.gle/Y7QLpuwDjBmoNM… Help out your friendly robot neighbor and send us your pictures! 🫶 cc Hermann Blum Marc Pollefeys @zurbrene CVG

I am looking forward to #RSS this week. Ping me if you want to meet up, or join our workshop on Open-Set Robotic Perception on Monday on-site or online: buff.ly/3zFU1Ln

Exciting Day! My lab‘s first robot got delivered.

Attending European Conference on Computer Vision #ECCV2026 next week? We will give a live demo of our language-guided mobile manipulation with spot on Thursday. If you want to chat or are interested to join my new lab as phd student, send me a message to meet up during the conference!

Our demo team is getting ready to show robotic scene interaction at European Conference on Computer Vision #ECCV2026 Demo starts at 14.30 cvg.github.io/spot-demos/

My tl;dr for this: Existing architectures + large-scale auto-labelled data ➡️ new SOTA on ScanNet200. learnings: - the current bottleneck is data, not architectures - we can overcome this with auto-labelling more on labelmaker.org, incl. 5000 densely labelled scenes

Turns out Project Aria @Meta glasses are a very useful tool to demonstrate actions to robots: Based on egocentric video, we track dynamic changes in a scene graph and use the representation to replay or plan interactions for robots. 🔗 behretj.github.io/LostAndFound/ 📄 arxiv.org/abs/2411.19162