LLaMA-3 is a prime example of why training a good LLM is almost entirely about data quality…

TL;DR. Meta released LLaMA-3-8B/70B today and 95% of the technical info we have so far is related to data quality:

- 15T tokens of pretraining data
- More code during pretraining

Local Stable Cascade 1 Click Launcher

Just wrote a 1-click launcher for this Stable Cascade Gradio app. Works very well!

Works on all platforms: Windows, Mac, Linux

Ten months ago, we launched the Vesuvius Challenge to solve the ancient problem of the Herculaneum Papyri, a library of scrolls that were flash-fried by the eruption of Mount Vesuvius in 79 AD.

Today we are overjoyed to announce that our crazy project has succeeded. After 2000

Thinking about making a list of some of the most impactful writings about tech and venture. Figure I'd let elon pay for the hosting costs.

Really interesting that Unicorn came from a data gathering project. Mostly descriptive, some great insights.

techcrunch.com/2013/11/02/wel…

I am so excited to tell the world about what we've been working on for the last few months. World, say hello to Bestever AI — GenAI tool for image & video ads.

The easiest way to generate creatives for campaigns is here.

The FTC is investigating OpenAI and the document outlining their questions is fascinating.

🧵Some highlights:

Is there a single 'AI Agent' product that people are actually using?

You'll soon see lots of 'Llama just dethroned ChatGPT' or 'OpenAI is so done' posts on Twitter. Before your timeline gets flooded, I'll share my notes:

▸ Llama-2 likely costs $20M+ to train. Meta has done an incredible service to the community by releasing the model with a

The future of AI code assistants is open-source, private, secure, and on-device. That future starts today. We’re excited to release Rift, an open-source AI-native language server and VSCode extension for local copilots.

morph.so

nfx.com/post/ai-startu… Really great article, most of the tests were things we already were thinking of, and the ODD test is a Better way of articulating something we already knew was a hairy problem, which is something i look for when reading a new article.

I share for all of the dreamers and makers. How do we successfully neutralize one of these?

Create advanced apps without coding using @Bubble! 🤓 Connect to APIs like OpenAI, add features like text generation, and launch your app quickly.

🚀Future of UI dev🔮:
~10% fixed UIs built by hand like today
~40% replaced by conversational UIs
~50% long-tail, on-the-fly UIs generated for specific tasks, used once, then vanish

Combined with ChatGPT plugins to read/write from the world 🤯

youtube.com/watch?v=xgi1YX…

Amjad Masad ChatGPT is cheaper than Davinci mostly because model is smaller.
Training over more data helps, but training over probability-distribution (via RL/Gumbel-softmax/KnowledgeDistillation) are much better than noisy next word training. This has gone unnoticed.
x.com/_ayushkaushal/…

LLM companies made a mistake of marketing model size via parameter count. Similar to how PCs made it seem like megahertz = performance and them slowly backtracked, I suspect now that parameter count is *going down* nobody is revealing their model size.

GPT-4 and its ilk are awesome for rapid prototyping and one-offs, but at the end of the day, enterprises will deploy far smaller distilled models in production. Here's my contrarian take -

I’ll have a new issue of Artificial Fintelligence (finbarrtimbers.substack.com) out later today talking about why, exactly, LLaMa.cpp is possible, and some estimates about how much room there is to improve it.

techcrunch.com/2023/03/15/zer…

An example of how we are in a complicated new world: I alternate between posts advising how to use AI to achieve 30%-50% productivity improvements, and ones warning that AI also appears to be able to generate novel chemical compounds and actually place orders for them online.

LLaMA has been fine-tuned by stanford,

'We performed a blind pairwise comparison between text-davinci-003 and Alpaca 7B, and we found that these two models have very similar performance: Alpaca wins 90 versus 89 comparisons against text-davinci-003.'