[pipeline] CogVideoX-Fun Control by a-r-r-o-w · Pull Request #9671 · huggingface/diffusers (original) (raw)

import torch from diffusers import CogVideoXFunControlPipeline, DDIMScheduler from diffusers.utils import export_to_video, load_video

pipe = CogVideoXFunControlPipeline.from_pretrained( "alibaba-pai/CogVideoX-Fun-V1.1-5b-Pose", torch_dtype=torch.bfloat16 ) pipe.scheduler = DDIMScheduler.from_config(pipe.scheduler.config) pipe.to("cuda")

control_video = load_video( "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/diffusers/hiker.mp4" ) prompt = ( "An astronaut stands triumphantly at the peak of a towering mountain. Panorama of rugged peaks and " "valleys. Very futuristic vibe and animated aesthetic. Highlights of purple and golden colors in " "the scene. The sky is looks like an animated/cartoonish dream of galaxies, nebulae, stars, planets, " "moons, but the remainder of the scene is mostly realistic." )

video = pipe(prompt=prompt, control_video=control_video).frames[0] export_to_video(video, "output.mp4", fps=8)

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.