Daniel Han (@danielhanchen) 's Twitter Profile
Daniel Han

@danielhanchen

Building @UnslothAI. Finetune train LLMs faster. LLMs bug hunter. OSS package github.com/unslothai/unsl…. YC S24. Prev ML at NVIDIA. Hyperlearn used by NASA.

ID: 717359704226172928

linkhttps://unsloth.ai/ calendar_today05-04-2016 14:34:16

2,2K Tweet

23,23K Followers

1,1K Following

Daniel Han (@danielhanchen) 's Twitter Profile Photo
Daniel Han
@danielhanchen

8 months ago

Phi-4 bug fixes: 1. EOS should be <|im_end|> not <|endoftext|> 2. Pad token EOS should be <|dummy_87|> 3. Chat template shouldn't default add "assistant" & Llama-fied Phi-4 & split QKV to increase accuracy for fine-tuning & made dynamic 4bit quants! Details: 1. The EOS should

Phi-4 bug fixes: 1. EOS should be &lt;|im_end|&gt; not &lt;|endoftext|&gt; 2. Pad token EOS should be &lt;|dummy_87|&gt; 3. Chat template shouldn't default add "assistant" &amp; Llama-fied Phi-4 &amp; split QKV to increase accuracy for fine-tuning &amp; made dynamic 4bit quants! Details: 1. The EOS should