Symmetry implementation custom script · AUTOMATIC1111/stable-diffusion-webui · Discussion #2441 (original) (raw)

After a bit of experimentation, putting the flips in the forward pass of CFGDenoiser seems to play best with all k-diffusion samplers:

class CFGDenoiser(torch.nn.Module): def init(self, model): super().init() self.inner_model = model self.mask = None self.nmask = None self.init_latent = None self.step = 0

def forward(self, x, sigma, uncond, cond, cond_scale, image_cond):
    if state.interrupted or state.skipped:
        raise InterruptedException

    conds_list, tensor = prompt_parser.reconstruct_multicond_batch(cond, self.step)
    uncond = prompt_parser.reconstruct_cond_batch(uncond, self.step)

    batch_size = len(conds_list)
    repeats = [len(conds_list[i]) for i in range(batch_size)]

    x_in = torch.cat([torch.stack([x[i] for _ in range(n)]) for i, n in enumerate(repeats)] + [x])
    image_cond_in = torch.cat([torch.stack([image_cond[i] for _ in range(n)]) for i, n in enumerate(repeats)] + [image_cond])
    sigma_in = torch.cat([torch.stack([sigma[i] for _ in range(n)]) for i, n in enumerate(repeats)] + [sigma])

    if state.sampling_step < state.sampling_steps*opts.sampler_mirroring_fraction:
        if opts.sampler_mirroring_method == 'Alternate Sample Flip':
            if opts.sampler_mirroring_mode == 'V-mirror':
                x_in[:, :, :, :] = torch.flip(x_in, [3])
            elif opts.sampler_mirroring_mode == 'H-mirror':
                x_in[:, :, :, :] = torch.flip(x_in, [2])
            elif opts.sampler_mirroring_mode == 'Rot-90':
                x_in[:, :, :, :] = torch.rot90(x_in, dims=[2, 3])
            elif opts.sampler_mirroring_mode == 'Rot-180':
                x_in[:, :, :, :] = torch.rot90(torch.rot90(x_in, dims=[2, 3]),dims=[2, 3])
        elif opts.sampler_mirroring_method == 'Avergage Flipped Copy':
            if opts.sampler_mirroring_mode == 'V-mirror':
                x_in[:, :, :, :] = (torch.flip(x_in, [3]) + x_in)/2
            elif opts.sampler_mirroring_mode == 'H-mirror':
                x_in[:, :, :, :] = (torch.flip(x_in, [2]) + x_in)/2
            elif opts.sampler_mirroring_mode == 'Rot-90':
                x_in[:, :, :, :] = (torch.rot90(x_in, (2, 3)) + x_in)/2
            elif opts.sampler_mirroring_mode == 'Rot-180':
                x_in[:, :, :, :] = (torch.rot90(torch.rot90(x_in, dims=[2, 3]),dims=[2, 3]) + x_in)/2
        ...

The 'Alternate Flip' method works much better than merging here, Producing some quite unexpectedly creative mirrored compositions at low sample cut-offs:

Horiz:
image

Vert:
image

180-rotation:
image

90-degree rotations tend to either make abstract dinner plates or loosely frame central subjects:
image

As well as the desired hard symmetry

image

image

Img2img also works, with the denoising levels one would expect for a pretty through transformation of the image:
image