SaeedNajafi (@saeeednajafi) 's Twitter Profile
SaeedNajafi

@saeeednajafi

CS PhD at @ualberta | Interested in #NLP, #ML, and efficient computing.

ID: 1292508112361009155

calendar_today09-08-2020 17:09:03

105 Tweet

99 Followers

150 Following

SaeedNajafi (@saeeednajafi) 's Twitter Profile Photo
SaeedNajafi
@saeeednajafi

3 months ago

My solution to every GPU out-of-memory errors: use_peft: true lora_r: 64 lora_alpha: 64 lora_dropout: 0.01 lora_target_modules: - q_proj - k_proj - v_proj - o_proj bf16: true gradient_accumulation_steps: 16 optim: adamw_8bit