Quality Improvement

[TDS4874]

Using DDIM, I found that aligning the alphas_cumprod tensor with the original Animatediff repository improves the quality. I haven't changed the way alphas_cumprod is calculated itself, so a proper implementation requires expertise. This is beyond my current capabilities. Could someone knowledgeable please correct it? Thank you.

Before and after the improvement. The generation conditions are identical for both.

[continue-revolution]

It would be extremely helpful if you could provide a skeleton code and I can look into how to hack A1111.

[TDS4874]

new_schedule.txt
new_schedule of alphas_cumprod when you run original Animatediff is outputed by changing scheduling_ddim.py in diffusers.

And I changed make_schedule method of DDIMSampler class in repositories\stable-diffusion-stability-ai\ldm\models\diffusion\ddim.py of webui repo like below.

def make_schedule(self, ddim_num_steps, ddim_discretize="uniform", ddim_eta=0., verbose=True):
    self.ddim_timesteps = make_ddim_timesteps(ddim_discr_method=ddim_discretize, num_ddim_timesteps=ddim_num_steps,
                                              num_ddpm_timesteps=self.ddpm_num_timesteps,verbose=verbose)
    alphas_cumprod = self.model.alphas_cumprod


    #insert to change schedule
    with open('new_schedule.json', 'r') as f:
        new_schedule_list = json.load(f)
    new_schedule_tensor = torch.tensor(new_schedule_list)
    if new_schedule_tensor.shape == alphas_cumprod.shape:
        new_schedule_converted = new_schedule_tensor.to(dtype=alphas_cumprod.dtype, device=alphas_cumprod.device)
        alphas_cumprod = new_schedule_converted
    #insert to change schedule


    assert alphas_cumprod.shape[0] == self.ddpm_num_timesteps, 'alphas have to be defined for each timestep'
    to_torch = lambda x: x.clone().detach().to(torch.float32).to(self.model.device)

[opparco]

import torch

beta_start = 0.00085
beta_end = 0.012
# beta_schedule = "linear"
num_train_timesteps = 1000  # default

betas = torch.linspace(beta_start, beta_end, num_train_timesteps, dtype=torch.float32)

alphas = 1.0 - betas
alphas_cumprod = torch.cumprod(alphas, dim=0)

[opparco]

https://gist.github.com/opparco/a36eaba42cb5712b04c0c143775524b4

[continue-revolution]

Thank you very much for your contribution. I have merged the gist into v1.3.0

[chuanSir123]

牛的牛的，只剩下75负面提示词了