Support ControlNet by ryu38 · Pull Request #153 · apple/ml-stable-diffusion (original) (raw)

The files you ended up with are a working model, when used along with a ControlNet model. But they won't work without a ControlNet model. That is, they won't work for regular inference, or for Image2Image.

To also get the Unet.mlmodelc so that the base model will work with and without a ControlNet in the pipeline, remove --unet-support-controlnet from the second command (the one after the &&). That pass will now add the Unet.mlmodelc to the files from the first pass.

The --unet-support-controlnet modifies the type of of Unet created by the --convert-unet argument. With just --convert-unetyou get a Unet.mlmodelc. With --convert-unet and unet-support-controlnet together, you get a ControlledUnet.mlmodelc. With just unet-support-controlnet you do not get any Unet.

Note: I believe that you will also need to use the --quantize-nbits 6 argument when converting the ControlNet model in order for it to work with a 6-bit base model.