EmbeddedLLM (@embeddedllm) 's Twitter Profile
EmbeddedLLM

@embeddedllm

Your open-source AI ally. We specialize in integrating LLM into your business.

ID: 1716394660636295168

calendar_today23-10-2023 10:02:43

303 Tweet

621 Followers

1,1K Following

EmbeddedLLM (@embeddedllm) 's Twitter Profile Photo
EmbeddedLLM
@embeddedllm

2 years ago

🚀 Big news! AMD GPUs are now on par with NVIDIA for LLM inference. Achieving impressive speeds of 718.06 toks/s for Llama 2 7B using 1 GPU and 595.11 toks/s for 13B using 2 GPU. Thanks to ROCm 5.6 + vLLM v0.1.4 on MI210. 💪 #AMD #LLM #AI embeddedllm.com/blog/vllm_rocm/

🚀 Big news! <a href="/AMD/">AMD</a> GPUs are now on par with NVIDIA for LLM inference. Achieving impressive speeds of 718.06 toks/s for Llama 2 7B using 1 GPU and 595.11 toks/s for 13B using 2 GPU. Thanks to ROCm 5.6 + vLLM v0.1.4 on MI210. 💪 #AMD #LLM #AI embeddedllm.com/blog/vllm_rocm/