Skip to content

[CI] Remove dependency on bot user for uploading benchmark CI results #20333

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 12 commits into
base: sycl
Choose a base branch
from
Open

[CI] Remove dependency on bot user for uploading benchmark CI results #20333

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
12 commits
Select commit Hold shift + click to select a range
df8990c
test pushing to branch
ianayl Oct 3, 2025
7ce4420
give write permissions
ianayl Oct 8, 2025
866b64e
amend perms
ianayl Oct 8, 2025
b518e18
Revert "amend perms"
ianayl Oct 8, 2025
2109b73
Revert "give write permissions"
ianayl Oct 8, 2025
6e73cf7
test perms
ianayl Oct 8, 2025
e20975f
typo
ianayl Oct 8, 2025
3bea805
more perms
ianayl Oct 8, 2025
418bd1d
test fail
ianayl Oct 9, 2025
1539867
set to correct repo
ianayl Oct 9, 2025
8e1f1b8
switch branch
ianayl Oct 9, 2025
66c4b5c
try removing manually passing secrets
ianayl Oct 10, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/workflows/sycl-docs.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -64,7 +64,7 @@ jobs:
touch .nojekyll
# Update benchmarking dashboard configuration
cat << EOF > benchmarks/config.js
remoteDataUrl = 'https://raw.githubusercontent.com/intel/llvm-ci-perf-results/refs/heads/unify-ci/';
remoteDataUrl = 'https://raw.githubusercontent.com/intel/llvm/refs/heads/sycl-benchmark-ci-results/';
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

my note on this change - while it seems convenient, I can see on my PC, that repo intel/llvm-ci-perf-results is ~600MB - this will be extra MBs added to intel/llvm repo, which is already quite big - just to consider

defaultCompareNames = ["Baseline_PVC_L0"];
EOF
# Upload the generated docs as an artifact and deploy to GitHub Pages.
Expand Down
3 changes: 1 addition & 2 deletions .github/workflows/sycl-linux-run-tests.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -212,7 +212,7 @@ on:
- "run-only"

permissions:
contents: read
contents: write
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can we set the write permission only at the job step(s) that ened it?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let me attempt this and I'll get back to you

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks like permissions have a job-level granularity, I can't just give it to a single step.

Doesn't make much sense to nest an additional write permission for the job given that there is only a single job in the entire workflow, are we cool to keep it like this?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

security BKMs says to set top-level permissions the least required (at best read only), in case you ever add a second job

// ref. https://github.com/ossf/scorecard/blob/49c0eed3a423f00c872b5c3c9f1bbca9e8aae799/docs/checks.md#token-permissions

packages: read

jobs:
Expand Down Expand Up @@ -361,4 +361,3 @@ jobs:
build_ref: ${{ inputs.repo_ref }}
env:
RUNNER_TAG: ${{ inputs.runner }}
GITHUB_TOKEN: ${{ secrets.LLVM_SYCL_BENCHMARK_TOKEN }}
3 changes: 3 additions & 0 deletions .github/workflows/sycl-ur-perf-benchmarking.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -174,6 +174,9 @@ jobs:
run_benchmarks_build:
name: Run Benchmarks on Build
needs: [ build_sycl, sanitize_inputs ]
permissions:
contents: write
packages: read
strategy:
matrix:
include:
Expand Down
21 changes: 12 additions & 9 deletions devops/actions/run-tests/benchmark/action.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -75,6 +75,9 @@ runs:
python3 ./devops/scripts/benchmarks/presets.py query "$PRESET"
[ "$?" -ne 0 ] && exit 1 # Stop workflow if invalid preset
echo "PRESET=$PRESET" >> $GITHUB_ENV

# Set branch containing benchmark CI results:
echo "BENCHMARK_RESULTS_BRANCH=sycl-benchmark-ci-results" >> $GITHUB_ENV
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

perhaps just set it as an env (in the job scope, or top-level)...?

- name: Compute CPU core range to run benchmarks on
shell: bash
run: |
Expand Down Expand Up @@ -134,9 +137,10 @@ runs:

cd -
- name: Checkout results repo
shell: bash
run: |
git clone -b unify-ci https://github.com/intel/llvm-ci-perf-results
uses: actions/checkout@v5
with:
ref: ${{ env.BENCHMARK_RESULTS_BRANCH }}
path: llvm-ci-perf-results
- name: Run compute-benchmarks
env:
# Need to append "_<device>_<backend>" to save name in order to follow
Expand Down Expand Up @@ -237,9 +241,8 @@ runs:
shell: bash
run: |
cd "./llvm-ci-perf-results"
git config user.name "SYCL Benchmarking Bot"
git config user.email "[email protected]"
results_branch="unify-ci"
git config user.name "github-actions[bot]"
git config user.email "github-actions[bot]@users.noreply.github.com"

if git diff --quiet && git diff --cached --quiet; then
echo "No new results added, skipping push."
Expand All @@ -252,7 +255,7 @@ runs:
git commit -m "[GHA] Upload compute-benchmarks results from https://github.com/intel/llvm/actions/runs/${{ github.run_id }}"
results_file="$(git diff HEAD~1 --name-only -- results/ | head -n 1)"

if git push "https://[email protected]/intel/llvm-ci-perf-results.git" "$results_branch"; then
if git push; then
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

dont we need to specify the branch?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks to be already configured by actions/checkout@v5; I figured it was better to specify the branch once than to specify the branch multiple times.

echo "Push succeeded"
break
fi
Expand All @@ -262,8 +265,8 @@ runs:
cached_result="$(mktemp -d)/$(basename $results_file)"
mv "$results_file" "$cached_result"

git reset --hard "origin/$results_branch"
git pull origin "$results_branch"
git reset --hard "origin/$BENCHMARK_CI_RESULTS"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what's BENCHMARK_CI_RESULTS? am I missing something or wrong var name?

git pull

mv "$cached_result" "$results_file"
fi
Expand Down