Fine tuning works, like it’s not even hard to do and it works. You can use LoRA, QLoRA, full weights, whatever you have resources to do - it will work. You can even fine tune on top of a fine tune (like flan-t5 models) and it will also work

Some guides
- rentry.org/llm-training
- anyscale.com/blog/fine-tuni…
- asmirnov.xyz/doppelganger
- magazine.sebastianraschka.com/p/practical-ti…