Separate denoised and noise estimation in Euler CFG++ #9008
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
The current CFG++
The current Euler CFG++ can work on the
CONST
sampling type (RF), which matches the algorithm from the appendix of the paper, but the guidance scale must be very low (~0.1), and it’s unstable.It tends to oversaturate when given more steps under the same guidance scales that seem to work with fewer steps.
CFG++ for flow models
Recently, the official proposed a new approach for flow models, which separates denoised and noise estimation during sampling.
It changes the algorithm back to
instead of
With this change, the behavior of the guidance scale is more similar to the other sampling types.

This should make it more usable with flow.
Euler ancestral CFG++
The current
euler_ancestral_RF
is more similar to the "Overshoot Sampling" approach (from the AMO sampler), which takes a larger step along the velocity and rescales to the next state with noise compensation.This method doesn't separate denoised and noise, so it's probably not suitable for CFG++.
Euler ancestral in k-diffusion seems to implement DDIM's

eta
parameter, which controls the interpolation between deterministic DDIM (eta = 0
) and stochastic DDPM (eta = 1
). A simple strategy is to preserve the original algorithm and apply RF's alpha and sigma, just like SANA's adaptation of dpm-solver to flow-dpm-solver. It can also match the "Stochastic Curved Euler Sampler", a DDPM variant for RF. (Algorithm 1 in 2506.15864).The ancestral CFG++ seems fine in the current implementation.

All the examples are generated by SD3.5 medium with simple scheduler.