BaseTen: Creating Machine Learning APIs with Tuhin Srivastava and Amir Haghighat Tuhin Srivastava softwareengineeringdaily.com/2021/05/19/bas…

Finally revealing the result of our efforts over the past year and change. It's been so amazing building this with everyone on the Baseten team

Today we're launching BaseTen. We've been working hard over the last 18 months to build the tool that we wished we had to operationalize our models. Please check it out, and if you're interested, we'd love to give you early access!

Great convo between Jeff Meyerson, Amir Haghighat et al. I like BaseTen, that's what I would imagine near future of deployed ML models will be leaning towards to, given the amount of overhead needed to put together: - hosting a model - integrating other types of logic - usable interface

Here’s another sneak peek at our Blueprint progress 👀 Meet our Web IDE Think of it as the single place for building, testing, and deploying API endpoints with generative AI models—in your browser.

We launched Blueprint today: an easy way for engineers to fine-tune and serve open source foundation models. 🧵

We keep getting asked by users if they can use the 70B parameter model in production. We're serving the chat variant of Llama-2 70B on 2xA100 and getting pretty great throughput — it's cooking!

Repurposing Tuhin Srivastava's Llama v2 truss, got FreeWilly 2 up in under a minute. `:s/meta-llama\/Llama-2-70b-chat-hf/stabilityai\/FreeWilly2`. 275GB of weights later we're running at 23 tok/s out of the box.

Ready to try open source LLMs? Switch from GPT to Mistral 7B in the smallest refactor you'll ever ship: just 3 tiny code changes. If you're making the jump, DM us for $1,000 in free credits. baseten.co/blog/gpt-vs-mi…

every day i get to work with a world class team supporting customers with world class products. today we get to dream a little bigger

when i tell people working in infra is like being a plumber people assume it’s because of lots of pipe connecting, when in fact it’s because i spend most of my day digging through shit

2 things. 1. i have loved working on this team. model performance is so much fun and so rewarding. 2. persistence is key. we started working on model performance end of 2023 and watching us slowly become better and better has been an incredible experience.

Congrats to Conviction and Embed companies Baseten Figure Harvey LangChain Mistral AI @sierraplatform Pika (and our many pioneering friends) for making the #ForbesAI50 list! Ground floor of the revolution that will lead to many massive companies.

The models are available at the following links: Llama 3 8B Instruct: baseten.co/library/llama-… Llama 3 70B Instruct: baseten.co/library/llama-…

hit new peak demand today, 3 million RPS. thanks for stress testing our infra anon internet friend

New Qwen-QWQ running at 90tokens/s generation speed on a single H100 Baseten using a new spec-dec stack. Around 2x more than the rest of the leaderboard (artificialanalysis.ai/leaderboards/p…).

you can just do things faster

💪🫡 still plenty of juice to squeeze out of this one

speculation, in this case a eagle-3, remains one of the biggest levers to go from good to great. amazing job to leapfrog the market and get the most out of our GPUs

so much potential in this model and abu coming out of the gates just ripping the landscape on perf