GGML Flash Attention support merged into llama.cpp

github.com/ggerganov/llam…

Nice repo for evaluating LLMs Long-Context performance

OpenAI, Anthropic, Cohere are supported.

'NExT: Teaching Large Language Models to Reason about Code Execution'

The key problem this paper aims to solve is that, (LLMs) of code are typically trained on the surface textual form of programs, thus may lack a semantic understanding of how programs execute at run-time. So

The Perplexity of Quantized Llama 3 degrades quite a bit vs quantized Llama 2

But looking at the below Perplexity numbers - Llama 3 has a higher initial perplexity when not quantized vs Llama-2.

Possible explanation 👇

Degree of specialization of each model on the Wikitext

Google's new Med-Gemini surpasses the GPT-4 model family on every benchmark where a direct comparison could be made.

Achieves SoTA performance of 91.1% accuracy on MedQA (USMLE) benchmark, using a novel uncertainty-guided search strategy.

📌 A very significant advancements in

Meta is really taking the leadership position for OSS contributions.

Beyond Llama-3 - we have all the below from them

(and this not an exhaustive list)

- React
- PyTorch
- React Native
- GraphQL
- Jest
- Flow
- Yarn
- Hermes
- FBT
- Prophet
- Cassandra
- Mercurial (which is

Comparison of TensorRT-LLM on consumer hardware vs llama .cpp - by Jan

(blog link in 1st comment)
---

Findings - 'TensorRT-LLM was:

- 30-70% faster than llama.cpp on the same hardware

- Consumes less memory on consecutive runs and marginally more GPU VRAM

Really fantastic paper for a new understanding of In-context Learning in Transformers

'Transformers learn in-context'

In-context learning refers to the ability of Transformers to adapt their predictions based on the context provided in the input sequence, without the need for

Recently Microsoft announced 'ZeroQuant(4+2): Redefining LLMs Quantization with a New FP6-Centric Strategy for Diverse Generative Tasks' 🔥

📌 It investigates the limitations of existing 4-bit quantization methods like GPTQ for large language models (LLMs), which tend to overfit

Meta is really taking the leadership position for OSS contributions.

Beyond Llama-3 - we have all the below from them

(and this not an exhaustive list)

- React
- PyTorch
- React Native
- GraphQL
- Jest
- Flow
- Yarn
- Hermes
- FBT
- Prophet
- Cassandra
- Mercurial (which is

anton For searching a database? of course it does, context kills RAGs 300%.

Cache the dataset in the KV cache, it is better in any way shape or form.

RAG will only be used for explainability because it is very easy to explain (and 'blame').

(rag is also very hard to get to work..)

The Perplexity of Quantized Llama 3 degrades quite a bit vs quantized Llama 2

But looking at the below Perplexity numbers - Llama 3 has a higher initial perplexity when not quantized vs Llama-2.

Possible explanation 👇

Degree of specialization of each model on the Wikitext

NEW VIDEO - Rabbit R1: Barely Reviewable

youtu.be/ddTV12hErTc

This is the pinnacle of a trend that's been annoying for years: Delivering barely finished products to win a 'race' and then continuing to build them after charging full price. Games, phones, cars, now AI in a box

When MKBHD proves a point in one scene - Love his style. 💯

LLM Hallucinations ( oops it's LAM)