has anybody been able to finetune a Mistral model with PPO? I'm using trlx and training works fine with Llama-2 but Mistral starts to give degenerate output very soon, even with super conservative hyperparameters (high KL penalty coefficient +small clip range + low learning rate)

🌟 𝐌𝐮𝐥𝐭𝐢𝐦𝐨𝐝𝐚𝐥 𝐃𝐏𝐎🌟 🔍 DPO over-prioritizes language-only preference 🚀 Introducing mDPO: optimizes image-conditioned preference 🏆 Best 3B MLLM with reduced hallucination, beats LLaVA 7/13B with DPO Collaboration with Microsoft Research huggingface.co/papers/2406.11…

Wow, reviewing an ACL SRW paper for the first time was brutal. So many things were confusing and poorly written & formatted that I didn't even know where to start! Is this usually the case with papers submitted to this track? I understand the goal of this track so I still tried

can we keep line numbers even for deanonymized versions of papers? It would be so much simpler to refer to specific parts of the paper this way instead of sharing screenshots

Unsolicited UX suggestion for LLM providers via a chat UI (e.g., OpenAI Anthropic Perplexity): It would be neat to have a conversation tree as a side bar in addition to the main conversation thread. I often have multiple follow-up questions for a given and so a tree

My team is looking for a fall intern to work on text watermarking. If you are interested or know someone who might be interested please feel free to reach out to me. We are the Responsible AI team part of the AGI org at Amazon. Please spread the world 🙏🏻