Img2Img pipeline initialization fails on SD variants with no feature extractor/safety checker #1792
Description
Describe the bug
The SD2.1 base repo didn't have the safety checker components. It still loaded fine in a normal StableDiffusionPipeline, but failed when using the StableDiffusionImg2ImgPipeline since that requires a safety checker. Initializing it with the components of the base StableDiffusionPipeline worked but gave a warning since the safety checker is None for v2.1.
The issue with v2.1 specifically should now be fixed thanks to
- https://huggingface.co/stabilityai/stable-diffusion-2-1-base/commit/826d6be1b25b06a83b5250146b926ca9276bb944
- https://huggingface.co/stabilityai/stable-diffusion-2-1-base/commit/88bb1a46821197d1ac0cb54d1d09fb6e70b171bc
but more generally this is likely to crop up any time someone tries to load a variant with no safety checker into the Img2Img pipeline, so @patrickvonplaten suggested I open an issue anyway for others who might face the same bug.
Reproduction
pipe = StableDiffusionImg2ImgPipeline.from_pretrained( "stabilityai/stable-diffusion-2-1-base").to(device)
This fails with an error:
ValueError: Pipeline <class 'diffusers.pipelines.stable_diffusion.pipeline_stable_diffusion_img2img.StableDiffusionImg2ImgPipeline'> expected {'text_encoder', 'safety_checker', 'feature_extractor', 'vae', 'scheduler', 'unet', 'tokenizer'}, but only {'text_encoder', 'vae', 'scheduler', 'unet', 'tokenizer'} were passed.
Logs
No response
System Info
diffusers 0.11 on Google colab IIRC