cpu_offload vRAM memory consumption large than 4GB · Issue #1934 · huggingface/diffusers (original) (raw)

Skip to content

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Sign up

@Sanster

Description

@Sanster

Describe the bug

I am using the code from https://huggingface.co/docs/diffusers/optimization/fp16#offloading-to-cpu-with-accelerate-for-memory-savings to test cpu_offload, but the vRAM memory consumption is large than 4GB

GPU cpu_offload enabled vRAM cost
1080 Yes 4539MB
1080 No 5101MB
TITAN RTX Yes 5134MB
TITAN RTX No 5668MB

Reproduction

I am using the code from https://huggingface.co/docs/diffusers/optimization/fp16#offloading-to-cpu-with-accelerate-for-memory-savings

import torch from diffusers import StableDiffusionPipeline

pipe = StableDiffusionPipeline.from_pretrained( "runwayml/stable-diffusion-v1-5",

torch_dtype=torch.float16,

) pipe = pipe.to("cuda")

prompt = "a photo of an astronaut riding a horse on mars" pipe.enable_sequential_cpu_offload() image = pipe(prompt).images[0]

Logs

No response

System Info

test on 1080/TITAN RTX