Loading a SDXL model using from_single_file forces the use of the StableDiffusionXLPipeline (original) (raw)

When loading a SDXL model using from_single_file(), the returned pipeline is always an instance of StableDiffusionXLPipeline. This means that if you try to load a model into a StableDiffusionXLImg2ImgPipeline the returned pipeline is always StableDiffusionXLPipeline instead.

The culprint is this line - https://github.com/huggingface/diffusers/blob/b024ebb96501b410ff21c8158cd075ceb3856404/src/diffusers/pipelines/stable_diffusion/convert_from_ckpt.py#L1527C32-L1527C32

Other instances in the same file respect the pipeline_class argument passed into the download_from_original_stable_diffusion_ckpt function which is in charge of inferring the pipeline type from the input model. The logic in the linked line will always return the aforementioned pipeline instead.

Can be fixed by instantiating the pipeline using pipeline_class instead but I imagine that some safety checks might need to be performed in order to verify that the new pipeline supports the text_encoder_2 and tokenizer_2 arguments.