Will work on this idea, and try to implement experiments in the coming 2 weeks. Floating-point arithmetic non-associativity causes GPU-based matrix multiplications to produce numerically different outputs, particularly when computation order varies (e.g., with changing batch

I literally became weak as fuck. I am gone soft after graduating from college. Even at work I fucking suck. While in college I was working on crazy good side projects and I just never stopped. I used to build, write abt it, go back again and build something. It all stopped

neural nets. >Building Turi Labs, where me and a couple of my frnds work on research related to llms, agents, rl. >Currently on a random exploration phase. >One of our latest research paper: On the Structure of Floating-Point Noise in Batch-Invariant GPU Matrix Multiplication. Performs

Week 2 of delta @_TheResidency Completed the theoretical part of the project and implemented it in python code. Nights and Weekends were dedicated to finish the project. Hit a roadblock but got back up. I really should have gone the pure math route. Having a good math

This will surely bring the needed attention of a lot of ppl to ground level rl scene. Very good for the community as well. At Turi Labs, we are trying to work on some niche rl problems. Time to contribute to PI mayank

A small typo which wasted a lot of my time

A dream of mine is to build a 2d game world where different models interact and learn and work towards a common goal. Need enough compute to pull it off. mayank lets do it someday

This image is everywhere on my tl this week. Was busy the whole week with work, but should be able to read it by the end of the week to understand the buzz arounf the paper.

How does one become a good researcher? >is it by the number of good papers >by a phd >open sourced projects >open source contributions OR is it the experience of few years (>2) in the field trying out different ideas, approaches, building sota architectures (or atleast trying

Another day to study rl and build envs! Recently I am drifting more towards building game environments. I started CS because I wanted to build games. And now I can combine my favorite field to gamdev. Lets build some game envs!

Btw attended yesterday's Lossfunk graduation day. I won their merch!! Anyways, seeing all the projects and research is helpful for someone like me who always stays inside my own cave. The interaction is much needed, as it invites serendipity into the system. Research is fun,

New Research on the floating point noise seen in gpu kernels due to lack of batch invariance. I got the idea for this study while reading Thinking Machines blog on Non determinism in LLMs. It made me wonder what the noise actually looks like under the hood. My study takes that

I definitely think I should do a pre doc program.

introducing runable the general ai agent for every task slides, websites, reports, podcasts, images, videos... everything

Excited for the stuff we will build with Tinker. Thank you Thinking Machines for the $5,000 credit research grant. Back to building

Chief GPU Enchantress