OpenGPT-4o: A demo that combines Mixtral by Mistral, Idefics by Hugging Face, and Streaming STT nemo by NVIDIA. All open-access models, all free, and created in an hour

Announcement: huggingface.co/posts/KingNish…
Demo: huggingface.co/spaces/KingNis…

why is everyone bragging about day-1 support for gpt-4o. my guy you swapped out one string for another string. do you want a lollipop 🍭

Did I miss something there or did @openai leave the biggest question - 'when can we use this stuff?' - unanswered?

⭕ Use GPT-4o from LangChain

Today OpenAI launched their newest 'omni' model, offering improved speed and pricing compared to turbo.

You can use the available multimodal capabilities of it in any of your LangChain applications today!

Give it a try:

Py:

GPT-4o is 2x faster than GPT-4 Turbo and half the price.

I am 80% sure openAI has extremely low latency low quality model get to pronounce first 4 words in <200ms and then continue with the gpt4o model

Just notice, most of the sentences start with
“Sure”
“Of course”
“Sounds amazing”
“Let’s do it”
“Hmm”

And then it continues with +

The downside of hyping your future V5 so much is that you have to release all of your new models under the V4 brand in order to avoid disappointment -- in perpetuity

A reminder that this is what OpenAI is willing to release right now, and they have much better models in development

really exciting demos from OpenAI establishing new expectations for AI.
Lot of work to do on the Llama train -- which isn't going to stop until we catch up!
My personal feeling with gpt-4o -- it feels attainable - contrasting to gpt-4 when released, it felt magically impossible.

i think people are misunderstanding gpt-4o. it isn't a text model with a voice or image attachment. it's a natively multimodal token in, multimodal token out model.
you want it to talk fast? just prompt it to. need to translate into whale noises? just use few shot examples.

Real-time conversational audio-visual chat hits differently. A new mode of interaction that immediately feels very natural and different from anything I have seen before. The demos do not fully convey the experience. I’m excited to get it to users. 2/4

x.com/OpenAI/status/…

why does gpt-4o sound kinda flirty 😳

there is only one way this can go, 1000 startups will be obliterated today or 1000 startups will be born

Introducing GPT-4o, our new model which can reason across text, audio, and video in real time.

It's extremely versatile, fun to play with, and is a step towards a much more natural form of human-computer interaction (and even human-computer-computer interaction):

GPT-4o is our new state-of-the-art frontier model. We’ve been testing a version on the LMSys arena as im-also-a-good-gpt2-chatbot 🙂. Here’s how it’s been doing.

what do you mean I am running out of storage?!

oh

A big issue I see with AI systems is that people aren't spending enough time evaluating their evaluation pipeline.

1. Most teams use more than one metrics (3-7 metrics in general) to evaluate their applications, which is a good practice. However, very few are measuring the

Llama-3 based SQLCoder 8b is out! Open weights with a commercially friendly cc-by-sa license. Probably the best <10B param model for Postgres text to SQL right now.

Slightly better than gpt-4-turbo and claude opus for 0-shot text to SQL generation. Also approaches their

An open LLM that can be used for LLM-as-a-Judge evaluation as strong as OpenAI GPT-4 or Anthropic Claude 3? 🤯 Yes, KAIST AI just published PROMETHEUS 2, an open LLM specialized in evaluating other LLMs highly correlating with human and GPT-4 judgments. 🔥

Implementation:

Exciting new blog -- What’s up with Llama-3?

Since Llama 3’s release, it has quickly jumped to top of the leaderboard. We dive into our data and answer below questions:

- What are users asking? When do users prefer Llama 3?
- How challenging are the prompts?
- Are certain users