Eric Hartford (@cognitivecompai) 's Twitter Profile
Eric Hartford

@cognitivecompai

We make AI models Dolphin and Samantha
BTC 3ENBV6zdwyqieAXzZP2i3EjeZtVwEmAuo4
ko-fi.com/erichartford

ID: 2854214132

linkhttps://erichartford.com calendar_today13-10-2014 10:22:51

8,8K Tweet

16,16K Followers

518 Following

Eric Hartford (@cognitivecompai) 's Twitter Profile Photo
Eric Hartford
@cognitivecompai

10 months ago

OLMoE is cool, but, comparing it with mistral-7b and llama3.1-8b, I'm not sure it is preferable. It's the same size, faster, but performs worse. It seems the same old trade-off - MoE is faster, but less capable, than equivalent sized dense models.

OLMoE is cool, but, comparing it with mistral-7b and llama3.1-8b, I'm not sure it is preferable. It's the same size, faster, but performs worse. It seems the same old trade-off - MoE is faster, but less capable, than equivalent sized dense models.