Dance Diffusion (original) (raw)

Dance Diffusion is by Zach Evans.

Dance Diffusion is the first in a suite of generative audio tools for producers and musicians released by Harmonai.

Make sure to check out the Schedulers guide to learn how to explore the tradeoff between scheduler speed and quality, and see the reuse components across pipelines section to learn how to efficiently load the same components into multiple pipelines.

DanceDiffusionPipeline

class diffusers.DanceDiffusionPipeline

< source >

( unet scheduler )

Parameters

• unet (UNet1DModel) — A UNet1DModel to denoise the encoded audio.
• scheduler (SchedulerMixin) — A scheduler to be used in combination with unet to denoise the encoded audio latents. Can be one ofIPNDMScheduler.

Pipeline for audio generation.

This model inherits from DiffusionPipeline. Check the superclass documentation for the generic methods implemented for all pipelines (downloading, saving, running on a particular device, etc.).

__call__

< source >

( batch_size: int = 1 num_inference_steps: int = 100 generator: Union = None audio_length_in_s: Optional = None return_dict: bool = True ) → AudioPipelineOutput or tuple

Parameters

• batch_size (int, optional, defaults to 1) — The number of audio samples to generate.
• num_inference_steps (int, optional, defaults to 50) — The number of denoising steps. More denoising steps usually lead to a higher-quality audio sample at the expense of slower inference.
• generator (torch.Generator, optional) — A torch.Generator to make generation deterministic.
• audio_length_in_s (float, optional, defaults to self.unet.config.sample_size/self.unet.config.sample_rate) — The length of the generated audio sample in seconds.
• return_dict (bool, optional, defaults to True) — Whether or not to return a AudioPipelineOutput instead of a plain tuple.

If return_dict is True, AudioPipelineOutput is returned, otherwise a tuple is returned where the first element is a list with the generated audio.

The call function to the pipeline for generation.

Example:

from diffusers import DiffusionPipeline from scipy.io.wavfile import write

model_id = "harmonai/maestro-150k" pipe = DiffusionPipeline.from_pretrained(model_id) pipe = pipe.to("cuda")

audios = pipe(audio_length_in_s=4.0).audios

To save locally

for i, audio in enumerate(audios): write(f"maestro_test_{i}.wav", pipe.unet.sample_rate, audio.transpose())

To dislay in google colab

import IPython.display as ipd

for audio in audios: display(ipd.Audio(audio, rate=pipe.unet.sample_rate))

AudioPipelineOutput

class diffusers.AudioPipelineOutput

< source >

( audios: ndarray )

Parameters

• audios (np.ndarray) — List of denoised audio samples of a NumPy array of shape (batch_size, num_channels, sample_rate).

Output class for audio pipelines.

< > Update on GitHub