Dry run not working on Pull request using GitHub Actions workflow

[supraja2805]

Problem Description

Dry run mode is not working on GitHub actions workflow when pull request is opened

What is actually happening

Its giving Error message as below
"Unexpected error during full sync: TypeError: Cannot read properties of undefined (reading 'check_suite')"

What is the expected behavior

It needs to be generated a report in pull request with the expected changes

Error output, if available

Unexpected error during full sync: TypeError: Cannot read properties of undefined (reading 'check_suite')

Context

Are you using the hosted instance of probot/settings or running your own?

Running own

If running your own instance, are you using it with github.com or GitHub Enterprise?

GitHub.com

Version of probot/settings

safe-settings@0.1.0-rc.26 full-sync

Version of GitHub Enterprise

[supraja2805]

workflow file:

name: Safe Settings Sync
on:
push:
branches:
- main
pull_request:
paths:
- safe-settings/**
- .github/workflows/safe-settings.yaml
schedule:
- cron: 0 */4 * * *

-->

workflow_dispatch:
inputs:
NOOP:
description: 'Run in no-op mode'
required: false
type: boolean
default: false

concurrency:
cancel-in-progress: true
group: >-
${{ github.workflow }}-${{ github.event.pull_request.number || github.ref }}

jobs:
safe-settings-sync:
runs-on: ubuntu-latest
env:
SAFE_SETTINGS_VERSION: 2.1.17
SAFE_SETTINGS_CODE_DIR: .safe-settings-code
FULL_SYNC_FROM_REF: ${{ github.ref }}
steps:
- name: Checkout source
uses: actions/checkout@11bd719

  - name: Checkout GitHub Safe-Settings repository
    uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683
    with:
      path: ${{ env.SAFE_SETTINGS_CODE_DIR }}
      ref: ${{ env.SAFE_SETTINGS_VERSION }}
      repository: github/safe-settings

  - name: Setup Node.js
    uses: actions/setup-node@3235b876344d2a9aa001b8d1453c930bba69e610
    with:
      cache-dependency-path:
        ${{ env.SAFE_SETTINGS_CODE_DIR }}/package-lock.json
      cache: npm
      node-version-file: ${{ env.SAFE_SETTINGS_CODE_DIR }}/.nvmrc

  - name: Install dependencies
    run: npm install
    working-directory: ${{ env.SAFE_SETTINGS_CODE_DIR }}

  # -->
  - name: Set no-op mode flag
    run: |
      echo "FULL_SYNC_NOOP=false" >> $GITHUB_ENV

      if [[ "${{ github.event_name }}" == "pull_request" ]]; then
        echo "FULL_SYNC_NOOP=true" >> $GITHUB_ENV
      fi

      if [[ "${{ github.event_name }}" == "workflow_dispatch" ]]; then
        if [[ "${{ inputs.NOOP }}" != "" ]]; then
          echo "FULL_SYNC_NOOP=${{ inputs.NOOP }}" >> $GITHUB_ENV
        fi
      fi

  - name: Run application
    run: npm run full-sync
    working-directory: ${{ env.SAFE_SETTINGS_CODE_DIR }}
    env:
      ADMIN_REPO: admin
      APP_ID: ${{ vars.SAFE_SETTINGS_APP_ID }}
      BLOCK_REPO_RENAME_BY_HUMAN: false
      CONFIG_PATH: safe-settings
      DEPLOYMENT_CONFIG_FILE_PATH:
        ${{ github.workspace }}/safe-settings/deployment.yaml
      GH_ORG: ${{ vars.SAFE_SETTINGS_GH_ORG }}
      GITHUB_CLIENT_ID: ${{ vars.SAFE_SETTINGS_GITHUB_CLIENT_ID }}
      GITHUB_CLIENT_SECRET:
        ${{ secrets.SAFE_SETTINGS_GITHUB_CLIENT_SECRET }}
      LOG_LEVEL: debug
      ENABLE_PR_COMMENT: true
      CREATE_PR_COMMENT: true
      PRIVATE_KEY: ${{ secrets.SAFE_SETTINGS_PRIVATE_KEY }}
      FULL_SYNC_NOP: ${{ env.FULL_SYNC_NOOP }}
      SETTINGS_FILE_PATH: organisation.yaml

[PendaGTP]

@supraja2805 You can check this example https://github.com/UCL-MIRSG/.github/blob/main/.github/workflows/safe-settings.yaml. Please also set the LOG_LEVEL to trace to see if we can get more debug info to understand where this issue comes from

[jamengual]

@PendaGTP I'm running the same workflow and I get the same error, I have checked the GH app permissions and suc,h but it always fails with :

ERROR (probot): Cannot read properties of undefined (reading 'check_suite')
DEBUG (probot): Stats [

It seems to compare the config changes fine.

After checking the code, it looks like we need to change settings.js to check for the status of the event to see if that info is available

const renderedCommentMessage = await eta.renderString(commetMessageTemplate, stats)

    if (env.CREATE_PR_COMMENT === 'true' && this.context.payload?.check_run?.check_suite?.pull_requests?.[0]) {
      const pullRequest = this.context.payload.check_run.check_suite.pull_requests[0]

      const summary = ````

[seanuleh]

I have the same issue; but additionally this isn't even running against the PR ref. It is only running against the default branch of the ADMIN_REPO. This is useless; this PR workflow would be great to see any diffs during a PR in the ADMIN_REPO but it seems we cant even run it against a non default ref