Skip to content

Conversation

mostafajahanifar
Copy link
Collaborator

This PR is about moving model weights from TIA servers to HF 🤗

  • Uploading all the pretrained TIAToolbox model weights on HuggingFace
  • Using huggingface_hub for downloading/caching pretrained weights
  • Use None model pretrained argument during model initialization, to avoid redundant downloading of ImageNet weights

Copy link

codecov bot commented Jul 18, 2025

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 99.31%. Comparing base (e1eb5bd) to head (7cd2357).
⚠️ Report is 1 commits behind head on develop.

Additional details and impacted files
@@             Coverage Diff             @@
##           develop     #945      +/-   ##
===========================================
- Coverage    99.31%   99.31%   -0.01%     
===========================================
  Files           71       71              
  Lines         9163     9162       -1     
  Branches      1196     1195       -1     
===========================================
- Hits          9100     9099       -1     
  Misses          39       39              
  Partials        24       24              

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@shaneahmed shaneahmed linked an issue Jul 18, 2025 that may be closed by this pull request
6 tasks
@shaneahmed shaneahmed added the enhancement New feature or request label Jul 18, 2025
@shaneahmed shaneahmed added this to the Release v1.7.0 milestone Jul 18, 2025
@shaneahmed
Copy link
Member

shaneahmed commented Jul 24, 2025

Are the links in the yaml file still valid?

I would suggest changing these urls with repo_id. If we have a different repo in future we can use that in the yaml file.

@shaneahmed
Copy link
Member

shaneahmed commented Jul 24, 2025

Thanks @mostafajahanifar. This looks good to me. I have tested download of all files using test_get_pretrained_model locally and it works fine. Caching is also working fine. The only comment is to update the yaml file. If you can address this, I can merge this PR.

@Jiaqi-Lv Jiaqi-Lv requested a review from Copilot August 8, 2025 10:51
Copilot

This comment was marked as outdated.

@Jiaqi-Lv Jiaqi-Lv requested a review from Copilot August 29, 2025 10:04
Copy link
Contributor

@Copilot Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR migrates model weight hosting from TIA servers to HuggingFace for improved distribution and caching. The changes introduce HuggingFace Hub integration while modifying default behavior to avoid automatic ImageNet weight downloads.

Key changes:

  • Replace custom download logic with HuggingFace Hub for pretrained model weights
  • Change default weights parameter from "DEFAULT" to None to prevent unnecessary ImageNet downloads
  • Update model architecture handling for Inception and GoogleNet models

Reviewed Changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 4 comments.

File Description
tiatoolbox/models/architecture/vanilla.py Updates default weights parameter and reorganizes model architecture handling
tiatoolbox/models/architecture/init.py Replaces custom download with HuggingFace Hub integration
tests/test_utils.py Updates test to reflect new save_path behavior and adds debug output
requirements/requirements.txt Adds huggingface_hub dependency

Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.

@shaneahmed
Copy link
Member

@mostafajahanifar Any update on this? Please can you address the comments so we can merge this PR for the next release.

@shaneahmed shaneahmed linked an issue Oct 3, 2025 that may be closed by this pull request
@Jiaqi-Lv
Copy link
Collaborator

Jiaqi-Lv commented Oct 10, 2025

Are the links in the yaml file still valid?

I would suggest changing these urls with repo_id. If we have a different repo in future we can use that in the yaml file.

@shaneahmed
I have changed theurls to hf_repo_id=TIACentre/TIAToolbox_pretrained_weights. What I'm thinking is that we should be able to locate the model checkpoint on Hugging Face using hf_repo_id / {model_name}.pth. This works for most of our models, but doesn't work with a few models, for example:

  • The Hugging Face url for the model unet_tissue_mask_tsef is TIACentre/TIAToolbox_pretrained_weights/UNET-v0.0-TSEF.pth.
  • Another example is the model nuclick_light-pannuke, the Hugging Face URL is TIACentre/TIAToolbox_pretrained_weights/nuclick_unet-pannuke.pth.

In short, there are a few models whose name do not match the name of their checkpoints.
We could either:

  1. Manually rename these checkpoints on Hugging Face to match the model names as defined in the yaml file.
  2. Or we could keep the url field in the yaml file, making them point to the full url of the checkpoints on Hugging Face.

Personally I prefer option 1, I don't know why the checkpoint would have a different name to the model.

Update: I have gone with option 1, I have renamed these two checkpoints on Hugging Face.

@Jiaqi-Lv Jiaqi-Lv self-assigned this Oct 10, 2025
@Jiaqi-Lv Jiaqi-Lv requested a review from Copilot October 15, 2025 16:44
Copy link
Contributor

@Copilot Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

Copilot reviewed 5 out of 5 changed files in this pull request and generated 4 comments.

Comments suppressed due to low confidence (1)

tiatoolbox/models/architecture/init.py:42

  • Docstring return type states Path while the function annotation returns str. Update the docstring to match the actual return type or, preferably, return a Path object and keep the docstring as Path for consistency.
    Returns:
        Path:
            The local path to the cached pretrained weights after downloading.

Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.

@Jiaqi-Lv Jiaqi-Lv requested a review from Copilot October 16, 2025 14:37
Copy link
Contributor

@Copilot Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

Copilot reviewed 5 out of 5 changed files in this pull request and generated 2 comments.

Comments suppressed due to low confidence (1)

tiatoolbox/models/architecture/init.py:42

  • The return type is annotated as str, but the docstring still documents Path. Please align the docstring with the actual return type (or wrap the return in Path(...) to keep the previous Path return type).
    Returns:
        Path:
            The local path to the cached pretrained weights after downloading.

Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.

Copy link
Member

@shaneahmed shaneahmed left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@shaneahmed shaneahmed merged commit d173a8c into develop Oct 17, 2025
27 checks passed
@shaneahmed shaneahmed deleted the enhance-hf-weights branch October 17, 2025 11:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Cannot access the pretrained models on PCam 🐌 Slow tests

3 participants