Cognitive Computations (@cognitivecompai) 's Twitter Profile
Cognitive Computations

@cognitivecompai

We make AI models Dolphin and Samantha
BTC 3ENBV6zdwyqieAXzZP2i3EjeZtVwEmAuo4

ID: 2854214132

calendar_today13-10-2014 10:22:51

5,5K Tweet

14,14K Followers

478 Following

Cognitive Computations (@cognitivecompai) 's Twitter Profile Photo
Cognitive Computations
@cognitivecompai

2 months ago

OLMoE is cool, but, comparing it with mistral-7b and llama3.1-8b, I'm not sure it is preferable. It's the same size, faster, but performs worse. It seems the same old trade-off - MoE is faster, but less capable, than equivalent sized dense models.

OLMoE is cool, but, comparing it with mistral-7b and llama3.1-8b, I'm not sure it is preferable. It's the same size, faster, but performs worse. It seems the same old trade-off - MoE is faster, but less capable, than equivalent sized dense models.