This simple pytorch trick will cut in half your GPU memory use / double your batch size (for real). Instead of adding losses and then computing backward, it's better to compute the backward on each loss (which frees the computational graph). Results will be exactly identical

Eternally stuck between wanting to make games, research more ML or go more knee deep into Graphics development 😔

Gotto check their ranks too. There are really interesting beings playing Dota.

sorry if i sound cranky, i’ve been operating on 1 brain cell these last few days and barely getting 2h of sleep per night

horrors beyond comprehension

Look at these cows

Hiking over the hills

Currently stuck at Plan C

Me looking at the deadline the PM decided on

I have been training neural networks for 9 years now. Here are 16 ways I actively use to optimize model training:

let's go get this bread

"Cameras as Relative Positional Encoding" TLDR: comparison for conditioning transformers on cameras: token-level raymap, attention-level relative pose encodings, a (new) relative encoding Projective Positional Encoding -> camera frustums, (int|ext)insics for relative pos encoding

this is about to be my go to on plane flights. u don't need to pay for wifi since it's just a dns request which aren't gated behind a paywall

Today, we're announcing our first hosted infrastructure product: pyx, a Python-native package registry. We think of pyx as an optimized backend for uv: it’s a package registry, but it also solves problems that go beyond the scope of a traditional "package registry".

Wrinkle map shader test #b3d #shader #geometrynodes

"It is beautiful. It is elegant. Does it work well in practice? Not really. This is often the caveat we face in research: the things that are beautiful don't work and the things that work are not beautiful." – Daniel Cremers