@lioronai : You can now run 1500B parameter models on your local CPU without GPUs. Microsoft finally open-sourced their 1-bit LLM inference framework called bitnet.cpp: &gt; 6.17x faster inference &gt; 82.2% less energy on CPUs &gt; Supports Llama3, Falcon3, and BitNet models &bull; TwiCopy

Lior⚡

@lioronai

+ Follow

Covering the latest in AI • ML engineer/researcher (2017) • Building @AlphaSignalAI into the largest media company in AI → At 230k readers, 200M impressions.

ID: 931470139

linkhttps://alphasignal.ai calendar_today07-11-2012 07:19:36

3,3K Tweet

96,96K Takipçi

1,1K Takip Edilen

Lior⚡

@lioronai

23 days ago

You can now run 100B parameter models on your local CPU without GPUs. Microsoft finally open-sourced their 1-bit LLM inference framework called bitnet.cpp: > 6.17x faster inference > 82.2% less energy on CPUs > Supports Llama3, Falcon3, and BitNet models

thumb_up_off_alt6,6K

chat_bubble_outline140

repeat809

shareShare