Skip to content

[WIP] update licenses #675

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 10 commits into
base: 2023.06-software.eessi.io
Choose a base branch
from
Open

[WIP] update licenses #675

Show file tree
Hide file tree
Changes from 5 commits
Commits
Show all changes
10 commits
Select commit Hold shift + click to select a range
062b624
update licenses
MartinsNadia Aug 19, 2024
e517a4e
fix commits, branches, identation. Add content and ids
MartinsNadia Dec 9, 2024
1480ae1
adding licenses yml
MartinsNadia Feb 3, 2025
bbb8ee9
needs more refinement but to show the progress for now
Feb 27, 2025
3355eb2
updated a bit
Feb 28, 2025
2d7826e
progress in the license retrieval
Mar 3, 2025
3a52f9c
still some to go but to show progress
Mar 11, 2025
a267a00
got it!
Mar 11, 2025
e50b25c
cleaning and reworking
Apr 7, 2025
f224575
cleaning files
Apr 7, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
59 changes: 59 additions & 0 deletions .github/workflows/check_licenses.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,59 @@
name: Check and update licenses

on:
push:
branches: [ "*-software.eessi.io" ]
pull_request:
branches: [ "*-software.eessi.io" ]
permissions:
contents: write # set permissions for writing
Copy link
Member

@ocaisa ocaisa Feb 3, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This action is being given a lot of power at a global level, do we really need it to have write permissions? Isn't it enough that we get a notification if it fails?


jobs:
license_update:
runs-on: ubuntu-latest

steps:
- name: Checkout out software-layer repository
uses: actions/checkout@b4ffde65f46336ab88eb53be808477a3936bae11

- name: Set up Python
uses: actions/setup-python@0a5c61591373683505ea898e09a3ea4f39ef2b9c
with:
python-version: '3.9'

- name: Run license script and generate patch
id: check_licenses
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
run: |
python update_licenses.py --source=pypi TensorFlow
python update_licenses.py --source=github:easybuilders/easybuild EasyBuild
Copy link
Member

@ocaisa ocaisa Feb 3, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You are going to need to extract the software name from the easystack file, that is complicated. Perhaps instead you should add easystack (and easyconfig) support to update_licenses.py?

Copy link
Member

@ocaisa ocaisa Feb 3, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This could also help you decide where to look for licenses (since you have access to source_urls)

Copy link
Member

@ocaisa ocaisa Feb 3, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The other option is to gather software (and extension) names via Lmod and then check them against the yaml file.

if [ -f license_update.patch ] && [ -s license_update.patch ]; then
PATCH_CONTENT=$(cat license_update.patch)
echo "patch=$PATCH_CONTENT" >> $GITHUB_OUTPUT
fi

- name: Create a PR (if changes detected)
id: create_pull_request
uses: peter-evans/create-pull-request@5e914681df9dc83aa4e4905692ca88beb2f9e91f # v7.0.5
if: steps.check_licenses.outputs.patch != ''
with:
commit-message: "Auto PR: Update licenses"
title: "Auto PR: Update licenses"
body: ${{ steps.check_licenses.outputs.patch }}
branch: update-licenses-${{ github.run_number }}
base: [ "*-software.eessi.io" ]
Copy link
Member

@ocaisa ocaisa Feb 3, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Doesn't this mean that if a PR adds software, then it will trigger another PR to add the licence of the software to the yaml file? Instead, wouldn't you want to make a suggestion that they add the licence as part of the original PR?


- name: Apply patch (if no PR created)
Copy link
Member

@ocaisa ocaisa Feb 3, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why wouldn't the PR be created? Wouldn't that mean the previous step will have failed?

if: steps.create_pull_request.outputs.number == '' && steps.check_licenses.outputs.patch != ''
run: |
if [ -f license_update.patch ] && [ -s license_update.patch ]; then
git apply license_update.patch
else
echo "No changes to apply"
fi
git add licenses.json
git diff --cached --exit-code || git commit -m "Update licenses.json"
git push
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
Copy link
Member

@ocaisa ocaisa Feb 3, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Rather than auto-apply a patch, can a code suggestion be made (or a PR to the source repo using a token without write permission to EESSI)?

59 changes: 59 additions & 0 deletions licenses/check_licenses.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,59 @@
name: Check and update licenses
Copy link
Member

@ocaisa ocaisa Feb 3, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Duplicate file?


on:
push:
branches: [ "*-software.eessi.io" ]
pull_request:
branches: [ "*-software.eessi.io" ]
permissions:
contents: write # set permissions for writing

jobs:
license_update:
runs-on: ubuntu-latest

steps:
- name: Checkout out software-layer repository
uses: actions/checkout@b4ffde65f46336ab88eb53be808477a3936bae11

- name: Set up Python
uses: actions/setup-python@0a5c61591373683505ea898e09a3ea4f39ef2b9c
with:
python-version: '3.9'

- name: Run license script and generate patch
id: check_licenses
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
run: |
python update_licenses.py --source=pypi TensorFlow
python update_licenses.py --source=github:easybuilders/easybuild EasyBuild
if [ -f license_update.patch ] && [ -s license_update.patch ]; then
PATCH_CONTENT=$(cat license_update.patch)
echo "patch=$PATCH_CONTENT" >> $GITHUB_OUTPUT
fi

- name: Create a PR (if changes detected)
id: create_pull_request
uses: peter-evans/create-pull-request@5e914681df9dc83aa4e4905692ca88beb2f9e91f # v7.0.5
if: steps.check_licenses.outputs.patch != ''
with:
commit-message: "Auto PR: Update licenses"
title: "Auto PR: Update licenses"
body: ${{ steps.check_licenses.outputs.patch }}
branch: update-licenses-${{ github.run_number }}
base: [ "*-software.eessi.io" ]

- name: Apply patch (if no PR created)
if: steps.create_pull_request.outputs.number == '' && steps.check_licenses.outputs.patch != ''
run: |
if [ -f license_update.patch ] && [ -s license_update.patch ]; then
git apply license_update.patch
else
echo "No changes to apply"
fi
git add licenses.json
git diff --cached --exit-code || git commit -m "Update licenses.json"
git push
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
Loading