GitHub - ArulselvanMadhavan/diffusers-ocaml: Diffusers API in OCaml (original) (raw)
Diffusers-OCaml: A Diffusers API in OCaml
This work is a port of the diffusers-rs library written in Rust.
Stable-Diffusion
How to run?
- Download weights and save it data directory. For instructions on how to do this, refer diffusers-rs-docs
- Once you have “.ot” files, you are ready to run stable diffusion
- I have found that placing unet and clip on GPU, and placing vae on cpu works the best for my GPU. Here is the command that does that.
dune exec stable_diffusion -- generate "lighthouse at dark" "vae" "data/pytorch_model.ot" "data/vae.ot" "data/unet.ot" - To run all models in CPU
dune exec stable_diffusion -- generate "lighthouse at dark" "all" "data/pytorch_model.ot" "data/vae.ot" "data/unet.ot" - To generate more than 1 sample, use the num_samples parameter
dune exec stable_diffusion -- generate "lighthouse at dark" "all" "data/pytorch_model.ot" "data/vae.ot" "data/unet.ot" --num_samples=2
Sample generated images
Performance on 8GB Nvidia GeForce GTX 1070 Mobile GPU
It takes about 27 seconds to generate an image. Measurements were done on a 12 CPU Intel(R) Core(TM) i7-8700K CPU @ 3.70GHz. Running all the steps in CPU takes little more than three minutes. I place vae on CPU; unet and clip on GPU
real 0m25.110s user 0m39.271s sys 0m11.518s
Image to Image generation
How to run?
dune exec img2img -- img2img media/in_img2img.jpg
Sample generated image
- Input image
- Output image
Performance on 8GB Nvidia GeForce GTX 1070 Mobile GPU
I placed vae on CPU; unet and clip on GPU
real 0m15.628s user 0m34.571s sys 0m5.833s
Inpaint
How to run?
dune exec inpaint -- generate media/sd_input.png media/sd_mask.png --cpu="vae" --prompt="Face of a panda, high resolution, sitting on a park bench"
Sample generated image
- Prompt: Face of a panda, high resolution, sitting on a park bench
Performance
I placed vae on CPU; unet and clip on GPU
real 0m28.055s user 0m51.600s sys 0m13.851s
Note
Only FP32 weights are supported.