[Core] fix variant-identification. by sayakpaul · Pull Request #9253 · huggingface/diffusers (original) (raw)
I don't think so.
cls.download() will NOT download the (legacy) variant sharded checkpoint files because of what I mentioned in #9253 (comment). To confirm that I printed the files that are getting downloaded and here's the log:
model_filenames={'vae/diffusion_pytorch_model.fp16.safetensors', 'text_encoder/pytorch_model.fp16-00003-of-00004.bin', 'unet/diffusion_pytorch_model-00001-of-00002.safetensors', 'text_encoder/pytorch_model.fp16-00001-of-00004.bin', 'text_encoder/pytorch_model.fp16-00002-of-00004.bin', 'text_encoder/pytorch_model.fp16-00004-of-00004.bin', 'unet/diffusion_pytorch_model.safetensors.index.json', 'text_encoder/model.fp16-00004-of-00004.safetensors', 'vae/diffusion_pytorch_model.fp16.bin', 'text_encoder/model.fp16-00003-of-00004.safetensors', 'text_encoder/model.safetensors.index.fp16.json', 'unet/diffusion_pytorch_model-00002-of-00002.safetensors', 'text_encoder/pytorch_model.bin.index.fp16.json', 'text_encoder/model.fp16-00001-of-00004.safetensors', 'vae/diffusion_flax_model.msgpack', 'text_encoder/model.fp16-00002-of-00004.safetensors', 'safety_checker/model.fp16.safetensors', 'safety_checker/pytorch_model.fp16.bin'}
variant_filenames={'vae/diffusion_pytorch_model.fp16.safetensors', 'text_encoder/pytorch_model.fp16-00003-of-00004.bin', 'text_encoder/pytorch_model.fp16-00001-of-00004.bin', 'text_encoder/pytorch_model.fp16-00002-of-00004.bin', 'text_encoder/pytorch_model.fp16-00004-of-00004.bin', 'text_encoder/model.fp16-00004-of-00004.safetensors', 'vae/diffusion_pytorch_model.fp16.bin', 'text_encoder/model.fp16-00003-of-00004.safetensors', 'text_encoder/model.safetensors.index.fp16.json', 'text_encoder/pytorch_model.bin.index.fp16.json', 'text_encoder/model.fp16-00001-of-00004.safetensors', 'text_encoder/model.fp16-00002-of-00004.safetensors', 'safety_checker/model.fp16.safetensors', 'safety_checker/pytorch_model.fp16.bin'}
Notice that in model_filenames we're not picking the (legacy) variants associated to the UNet because
| def variant_compatible_siblings(filenames, variant=None) -> Union[List[os.PathLike], str]: |
|---|
is unable to match them with regex. We use model_filenames to craft our allow_patterns:
| allow_patterns = list(model_filenames) |
|---|
We can further confirm this by printing the contents of each subfolder from here:
| cached_folder = snapshot_download( |
|---|
unet = ['diffusion_pytorch_model-00001-of-00002.safetensors', 'config.json', 'diffusion_pytorch_model-00002-of-00002.safetensors', 'diffusion_pytorch_model.safetensors.index.json'] text_encoder = ['model.fp16-00003-of-00004.safetensors', 'model.safetensors.index.fp16.json', 'config.json', 'model.fp16-00002-of-00004.safetensors', 'model.fp16-00001-of-00004.safetensors', 'model.fp16-00004-of-00004.safetensors'] vae = ['config.json', 'diffusion_pytorch_model.fp16.safetensors']
Hopefully, my concern is clear now.