Loading pre-trained models (`torch.hub.load_state_dict_from_url()` or `torch.hub.load()`)

[kaiidams]

@NiklasGustafsson @ericstj @GeorgeS2019

Probably people want to have a place to download pre-trained model files so that they can fine-tune pre-made models by TorchVision and TorchAudio without converting files themselves. We need likes of torch.hub and a public storage.

• Add torchaudio.models.WaveRNN(). #703
• Started on torchvision.models. #595

[NiklasGustafsson]

Adding @luisquintanilla to the discussion -- this is something we need to address for ML.NET in general.

[yueyinqiu]

I'm wondering why it seems to be stopped here. Has this been solved, or maybe are there some commercial problems?

Would it be acceptable to upload the converted models to some platforms like hugging face in the name of an individual?

[NiklasGustafsson]

The reason is simple -- in the general case, you would want to have the ability to associate model weights with code, such as a NuGet package. Model weights are really like any other binary resource (to use the .NET name), just really big. That size is the problem:

1. NuGet Gallery won't accept packages larger than 250MB, so that excludes a lot of weights files. The weights can't just be embedded in the package, they have to reside outside.

2. We can provide a "standard" space for model weights, but someone will have to pay for it. That makes it hard.

3. We can make it the developer's problem -- find some location to store weights, paid or free. We already have this capability, and @shaltielshmid's TorchSharp extension library has expanded the model weight formats that can be supported.

[shaltielshmid]

One options that might be possible - if we are using PyBridge to load in the weights, maybe we can just link directly to the storage that PyTorch uses and then we don't have to worry about that.

@NiklasGustafsson What do you think? Is this possible?

I will note that I haven't researched how this works behind the scenes, so my answer is completely theoretical.

[GeorgeS2019]

@NiklasGustafsson
#1277 (comment)

Use huggingface to store

[NiklasGustafsson]

One options that might be possible - if we are using PyBridge to load in the weights, maybe we can just link directly to the storage that PyTorch uses and then we don't have to worry about that.

The challenge is what @yueyinqiu called 'a public storage' -- who pays? We should make it easy to download weights in a standardized manner, but (for now), we have to make it up to package (model) developers to decide where to store the weights publicly.

[shaltielshmid]

Right. I guess my response is regarding the linked issue with downloading the weights to a pretrained model (vgg) from the torch hub.
Maybe we can integrate using the already uploaded weights from the hub?

[NiklasGustafsson]

You mean like we do in torchvision.models but automatically finding the weights files?

[shaltielshmid]

Yes

I'm not sure if that's the place to implement it since it would require PyBridge to be installed, which would be another dependency

[NiklasGustafsson]

No, it would go elsewhere.

[GeorgeS2019]

@NiklasGustafsson @shaltielshmid

Could one of you comment of the feasibility of using HuggingFace?

https://huggingface.co/williamlzw/stable-diffusion-1-5-torchsharp

Ref: #1277 (comment)

[GeorgeS2019]

Hugging Face provides an interface that allows you to export Transformers models to TorchScript, making it possible to reuse these models in environments other than PyTorch-based Python programs.

Now TorchSharp support TorchScript: Is HuggingFace the right place for TorchSHarp to support TorchScript??

[GeorgeS2019]

@williamlzw

Could you share your experience of sharing TorchSharp model through HuggingFace.

• Is it free.
• Is it complicated?