[training] use the lr when using 8bit adam. by sayakpaul · Pull Request #9796 · huggingface/diffusers (original) (raw)

@linoytsaban sorry I think I went too soon with the PR. I realized that we ALWAYS pack "lr" in params_to_optimize irrespective of the optimizer class we're using.

So, I think you were right not to pass the lr. However, when using "prodigy" we were still passing lr. So, to remove the inconsistency, I omitted those instances in this PR. Could you please give this another look?

Cc: @a-r-r-o-w for the changed Cog scripts.