New Meta R&D Reveals Shortcut to Ultra-wide Retina Resolution Holographic Displays for XR Headsets Read the report 👉 roadtovr.com/meta-reality-l… Nathan Matsuda Felix Heide Ethan Tseng

Researchers at Princeton Engineering have created a tiny optical device that makes holographic images larger and clearer. Small enough to fit on a pair of eyeglasses, the device could enable a new kind of immersive #VR display. bit.ly/4blIs9L

To solve autonomous trucking, constant re-calibration of perception stacks does not scale. For long-baseline multi-view perception, vibration and structural flex are common "in the wild". We tackle online calibration in the stereo correlation, revisiting rectification as a

Are you interested in neural fields and computational photography? Or just really like splines and have no one to talk to about splines? Come chat with me @CVPR2024 next week on our latest spliney neural fieldy burst photography project: youtu.be/bvho-mtfAcI?si…

I am looking forward to presenting 9 papers at #CVPR this week! One really fun one is #GatedFields, which makes neural rendering work at night and long distances. We exploit the intrinsic depth cues in the gated videos, allowing for novel illumination conditions, novel views, and

How can we generate interesting edge cases to test our autonomous vehicles in simulation? We propose CtRL-Sim, a novel framework for closed-loop behaviour simulation that enables fine-grained control over agent behaviours. 🧵 1/8 arxiv.org/abs/2403.19918

Looking forward to presenting #RadarFields later this week at #SIGGRAPH2024. We learn 3D geometry directly from 2D raw radar measurements in frequency space, instead of processed radar point clouds😀. Also, no volume rendering needed here. This allows us to access much higher

Split-Aperture 2-in-1 Cameras at #SIGGRAPH2024 for single-shot HDR, hyperspectral, and depth imaging! By splitting the camera aperture and using a dual-pixel sensor, we show that it is possible to capture an optically coded and conventional image simultaneously without increasing

Implicit Neural Light Spheres lets you turn panoramic captures into dynamic wide FOV renders (with real-time rendering!). Instead of generating panoramas with image stitching, we use neural light spheres to jointly estimate the camera path and a high-resolution scene

Multimodal BEV Fusion over Long Ranges and in Adverse Weather! Check out our work tomorrow at #ECCV2024. We fuse multimodal sensor data from lidar, radar, and stereo cameras through attentive, depth-based blending schemes, with learned refinement on the Bird’s Eye View (BEV)

Check out CtRL-Sim later this week at #CoRL2024 in Munich! How can we generate interesting edge cases to test autonomous vehicles in simulation? We propose CtRL-Sim for closed-loop behavior simulation that enables fine-grained control over agent behaviors. CtRL-Sim leverages

Evaluating Neural Networks at the Speed of Light (with Light!). See live optical inference in the video below. Excited to share recent academic work on optical neural networks as a collection of computing elements embedded in the camera lens! These elements perform computation

Our work on neural étendue expansion for holographic AR/VR displays was highlighted in Optics & Photonics News "Optics in 2024"! Read the summary at optica-opn.org/home/articles/… See the original Nature Communications paper here: nature.com/articles/s4146… Optica

Using algorithms to sense and understand the world is the focus of Felix Heide's work. Now, he has created a new kind of camera — one that has optical elements that function as a neural network and can identify images at the speed of light. 📸 bit.ly/3DkktfJ

.Felix Heide uses algorithms to make sense of the world. He has built a new kind of camera that uses optical elements as a neural network that identifies images at the speed of light. Learn more: engineering.princeton.edu/news/2025/02/2…

Scenario Dreamer has been accepted at #CVPR2025! Website: …ceton-computational-imaging.github.io/scenario-dream… We train a vectorized latent diffusion model to synthesize high-fidelity driving simulation environments (agents+map). Scenario Dreamer enables fully data-driven closed-loop generative simulation!

Vectorized Diffusion without Rasterized Encodings! Scenario Dreamer (CVPR 2025) directly operates on vectorized scene elements to generate novel unseen scenes - making fully data-driven closed-loop generative driving simulation possible. We trained a novel vectorized latent

Broadband Imaging with a 1cm Ultra-Flat Metalens! We overcome intrinsic spectral bandwidth limitations in new work in Nature Communications. See dynamic outdoor captures "in-the-wild" below. We find that computational design and probabilistic diffusion methods make it possible