Trajectory Consistency Distillation #7190
Description
Model/Pipeline/Scheduler description
Latent Consistency Model (LCM) extends the Consistency Model to the latent space and leverages the guided consistency distillation technique to achieve impressive performance in accelerating text-to-image synthesis. However, LCM struggles to generate images with both clarity and detailed intricacy. To address this limitation, the authors initially delve into and elucidate the underlying causes, identifying that the primary issue stems from errors in three distinct areas. Consequently, the authors introduce Trajectory Consistency Distillation (TCD), encompassing fine-tuning a teacher model on a trajectory consistency loss, along with a strategic stochastic sampling scheduler. The trajectory consistency loss function diminishes distillation errors by broadening the scope of the self-consistency boundary condition and endowing the TCD with the ability to accurately trace the entire trajectory of the Probability Flow ODE. Additionally, the strategic stochastic sampling scheduler is specifically designed to circumvent the accumulated errors inherent in multi-step consistency sampling, which is meticulously tailored to complement the TCD model. Experiments demonstrate that TCD not only significantly enhances image quality at low NFEs but also yields more detailed results compared to the teacher model at high NFEs.
Open source status
- The model implementation is available.
- The model weights are available (Only relevant if addition is not a scheduler).
Provide useful links for the implementation
Arxiv PDF: https://arxiv.org/pdf/2402.19159.pdf
Project Page: https://mhh0318.github.io/tcd/
Github (with Diffusers-based implementation): https://github.com/jabir-zheng/TCD
Huggingface (with TCD LoRA weights): https://huggingface.co/h1t/TCD-SDXL-LoRA