PLLaVA

Parameter-free LLaVA Extension from Images to Videos for Video Dense Captioning

Vision-language pre-training has significantly elevated performance across a wide range of image-language applications. Yet, the pre-training process for video-related tasks demands

New video on fine-tuning LLaVA with your own data!

LLaVa is a powerful LLM + vision model that understands images but produces verbose GPT-style responses

This guide fine-tunes it to instead respond with succinct image tags

2024 is the year for multimodal. Let it rip 🤙

karasu-1.1BをLLaVA-v1.5-Instruct-620K-JAで学習させてみたけど期待していたほど性能は良くなさそう。Heron-Benchの平均は42.16でした。

Recently completed a project centered around a Multimodal AI system named 'VisualInsights.' This system is designed to generate AI images and also incorporates an image-to-text feature based on the Llava architecture.
Live Demo 👇🏻

LLaVA Llama-3 and Phi-3 now on MLX 🎉🚀

You can now run inference locally on your Mac.

pip install -U mlx-vlm

I’m getting ~50 tokens on a M3 Max.

Model cards 👇🏾

mlx-vlm v0.0.4 is here 🎉

New models 🤖:
- Idefics 2
- Llava (Phi and Llama 3)

Improvements 🚀:
- Q4 quantisation support for all models
- Less imports to use generate()

Up next 🚧:
- More models
- Support for multiple images

Please leave us a star and send a PR

We've been blessed again with new LLaVA-like models based on LLaMA 3 & Phi-3 🤩
Also passes the baklava benchmark 🤝✅

𝐋𝐋𝐚𝐕𝐀-𝐋𝐥𝐚𝐦𝐚-𝟑-𝟖𝐁 model 🚀 by InternLM - A LLaVA model fine-tuned from Meta-Llama-3-8B-Instruct & CLIP-ViT-Large-patch14-336

🥳LLaVA-Phi-3 Mini is available too. It outperforms LLaVA-v1.5-7B and matches the performance of LLaVA-Llama-3-8B 🤯 in multiple benchmarks.

先日公開した解像度が高い画像の入力を可能とした日本語VLMであるllava-jp-1.3b-v1.1についての記事を書きました！
#Qiita

qiita.com/toshi_456/item…

Parameter-free LLaVA for video captioning works like magic! 🤩
Let's take a look 🧶
(also find repositories below ⬇️ )

Wow llava with llama3 and phi-3 is really good!

Llava 1.5 was good for multilingual before without finetune but, new models are much better i think.

Po instalacji dostaniecie konsolę. Trzeba wpisać 'ollama run llava'

📺Train & Serve Custom Multi-modal Models 📺

- IDEFICS 2 + LLaVA Llama 3 -

🎩Chapeau Leo Tronchon Victor Sanh Hugo Laurençon of Hugging Face

ローカルでLLaVA-1.6動かしてみたけど思ったより全然軽いわ。これ何か面白いこと出来ないかなぁ。

Sol's_akkount

VIDEO RESOURCES:
- IDEFICS 2 Model: huggingface.co/HuggingFaceM4/…
- IDEFICS 2 Blog: huggingface.co/blog/idefics2
- LLaVA Llama 3 model: huggingface.co/xtuner/llava-l…
- Chess Dataset: huggingface.co/datasets/Treli…

PRE-WATCH VIDEOS:
- Moondream / Tiny multi-modal models (explains architecture):…

Meiの旅/
あたしはリンカーン・タウンシップ公共図書館の温かな石造りと古い鍵を使った入口に立っており、床が静けさを保ちつつも、読書愛好家の活気を感じられるような穏やかな...
(Powered Google map apis,phi3-3.8mi16,emi-2,LLaVA,etc.basePhoto:Lincoln Township Public Library)

Miの旅/
この施設は「高野食品」が主売される小屋と思わせます。食料品と飲み物を取り扱う店舗で、個性的なカラーグレインのストアメニューが提供されているようですね。
(Powered Google map apis,phi3-3.8mi16,animagine-xl-3.1,LLaVA,etc.basePhoto:Junko Yone)

🤺 #Esgrima World Cup de florete Hong Kong 🇭🇰

🥉MEDALLA DE BRONCE🥉

T4 ♂️
❌️ Carlos Llavador cae derrotado en tocado de oro por 14-15 🆚 Takahiro Shikine 🇯🇵 tras una gran remontada

👏🏻 Endulza un poco ese sabor amargo que se quedó tras el Preolímpico.

Grande Llava!!!

ローカルLLMの進化が本当に期待しかない

Visionまでローカルで高速で動く。
（動画は３倍なので、3分の１程度の性能だと思ってみてください。それでも十分実用性あります）

以下、評判の良かった２つのモデルをベースにしたVision ADAPTER付きモデル。

・LLaVA++ based on Phi-3
・LLaVA++ based on…