integrate ort into textual-inversion and text-to-image examples #1576
Conversation
|
The documentation is not available anymore as the PR was closed or merged. |
prathikr
changed the title
|
@anton-l could I get a review on these scripts integrating ort with the remaining 2 stable diffusion tasks? |
patil-suraj
left a comment
patil-suraj
left a comment
There was a problem hiding this comment.
Looks good to me, thanks a lot for adding these examples! Would be nice to add a section in the readme, showing how to use this script along with installation instructions.
| accelerator.wait_for_everyone() | ||
| if accelerator.is_main_process: | ||
| unet = accelerator.unwrap_model(unet) | ||
| if args.use_ema: | ||
| ema_unet.copy_to(unet.parameters()) | ||
|
|
||
| pipeline = StableDiffusionPipeline.from_pretrained( | ||
| args.pretrained_model_name_or_path, | ||
| text_encoder=text_encoder, | ||
| vae=vae, | ||
| unet=unet, | ||
| revision=args.revision, | ||
| ) | ||
| pipeline.save_pretrained(args.output_dir) |
There was a problem hiding this comment.
Is it fine to save the unet directly given it's wrapped in ORTModule?
| subfolder="unet", | ||
| revision=args.revision, | ||
| ) | ||
| unet = ORTModule(unet) |
There was a problem hiding this comment.
Maybe a dump question: What model do we need to wrap during training, does need to be trainable or any module can be used ? Because in textual inversion the unet is not trained.
There was a problem hiding this comment.
@patil-suraj interesting, I did not catch this. Is it the text_encoder that gets trained? Is that a BERT based model or some other architecture?
There was a problem hiding this comment.
Yes, in textual inversion only the text embedding layer is trained not even the full-text encoder. And it's a CLIPTextModel.
4 participants
No description provided.