EmbeddedLLM (@embeddedllm) 's Twitter Profile
EmbeddedLLM

@embeddedllm

Your open-source AI ally. We specialize in integrating LLM into your business.

ID: 1716394660636295168

calendar_today23-10-2023 10:02:43

303 Tweet

621 Followers

1,1K Following

EmbeddedLLM (@embeddedllm) 's Twitter Profile Photo
EmbeddedLLM
@embeddedllm

4 months ago

Pro-tip for vLLM power-users: free ≈ 90 % of your GPU VRAM in seconds—no restarts required🚀 🚩 Why you’ll want this • Hot-swap new checkpoints on the same card • Rotate multiple LLMs on one GPU (batch jobs, micro-services, A/B tests) • Stage-based pipelines that call

Pro-tip for vLLM power-users: free ≈ 90 % of your GPU VRAM in seconds—no restarts required🚀 🚩 Why you’ll want this • Hot-swap new checkpoints on the same card • Rotate multiple LLMs on one GPU (batch jobs, micro-services, A/B tests) • Stage-based pipelines that call