[docs] add a snippet for compilation in the auraflow docs. by sayakpaul · Pull Request #11327 · huggingface/diffusers (original) (raw)
This flag changes whether we should use the same symbolic variable to represent input sizes that are the same.
So duck_shape = True means -- under dynamic shapes, upon compilation of a torch.compile-labelled component, if your inputs have dimensions that are the same, the compiler assumes they will be the same in the future as well.
For instance, this could happen if you generate 512x512 images first, which eventually becomes something like 64x64 hidden_states input to the transformer, and compilation adds this assumption s0 == s1 for future hidden_states input with dynamic shape s0 x s1. Then if you generate 1024x512 image next, the transformer block sees an input of, say, 128x64 hidden_states, and the aforementioned s0 == s1 assumption breaks, which triggers recompilation.
So duck_shape = False avoids adding that s0 == s1 assumption in the first place, so that no recompilation will be triggered from its breaking. Although this means compiler might have less opportunity to do optimizations (since it lost the s0 == s1 assumption), but its practical effects are hard to predict, and I only saw a small difference from my limited local experiment.