Fix custom jobs persisting in final repositories when removed from templates

Actions/CheckForUpdates/CheckForUpdates.ps1

@@ -254,8 +254,31 @@ foreach($checkfile in $checkfiles) {
 
                 if ($dstFileExists) {
                     if ($type -eq 'workflow') {
-                        Write-Host "Apply customizations from current repository: $dstFile"
-                        [Yaml]::ApplyCustomizations([ref] $srcContent, $dstFile)
+                        # Determine if current repository is a final repository (has templateUrl pointing to another repo)
+                        # Final repositories should not have custom jobs applied to prevent persistence of removed template jobs
+                        # unless allowCustomJobsInEndRepos is explicitly set to true
+                        $currentRepoReference = $env:GITHUB_REPOSITORY
+                        $isFinalRepository = $false
+                        $allowCustomJobsInEndRepos = $false
+
+                        if ($repoSettings.ContainsKey('allowCustomJobsInEndRepos')) {
+                            $allowCustomJobsInEndRepos = $repoSettings.allowCustomJobsInEndRepos
+                        }
+
+                        if ($repoSettings.templateUrl) {
+                            # Extract repository reference from templateUrl (e.g., "microsoft/AL-Go-PTE" from "https://github.com/microsoft/AL-Go-PTE@main")
+                            $templateRepoUrl = $repoSettings.templateUrl.Split('@')[0]
+                            $templateRepoReference = $templateRepoUrl.Split('/')[-2..-1] -join '/'
+                            $isFinalRepository = $templateRepoReference -ne $currentRepoReference
+                        }
+
+                        if ($isFinalRepository -and -not $allowCustomJobsInEndRepos) {
+                            Write-Host "Skipping custom jobs from current repository (final repository using template: $($repoSettings.templateUrl), allowCustomJobsInEndRepos: $allowCustomJobsInEndRepos): $dstFile"
+                        }
+                        else {
+                            Write-Host "Apply customizations from current repository: $dstFile"
+                            [Yaml]::ApplyCustomizations([ref] $srcContent, $dstFile)
+                        }
                     }
 
                     # file exists, compare and add to $updateFiles if different

Scenarios/UpdateAlGoSystemFiles.md

@@ -24,6 +24,10 @@
 
 1. By default, this workflow will apply any updates to the **workflow files (in .github\\workflows)** or **system scripts (in .AL-Go)** from the template repository used to spin up the repository. If you want to change branch or template Url, you can specify the `templateUrl@branch` when you run the workflow.
 
+1. **Custom jobs behavior**: When updating system files, custom jobs are handled differently based on repository type:
+   - **Template repositories** (without `templateUrl` setting): Custom jobs are preserved and applied
+   - **Final repositories** (with `templateUrl` pointing to another repository): Custom jobs from the current repository are NOT applied to prevent persistence of jobs removed from the template
+
 ______________________________________________________________________
 
 [back](../README.md)

Scenarios/settings.md

@@ -82,6 +82,7 @@ The repository settings are only read from the repository settings file (.github
 | <a id="buildModes"></a>buildModes | A list of build modes to use when building the AL-Go projects. Every AL-Go project will be built using each build mode. The following build modes have special meaning in AL-Go:<br /> **Default**: Apps are compiled as they are in the source code.<br />**Clean**: Should be used for Clean Mode. Use [Conditional Settings](https://aka.ms/algosettings#conditional-settings) with buildMode set the 'Clean' to specify preprocessorSymbols for clean mode.<br />**Translated**: `TranslationFile` compiler feature is enabled when compiling the apps.<br /><br />It is also possible to specify custom build modes by adding a build mode that is different than 'Default', 'Clean' or 'Translated' and use [conditional settings](https://aka.ms/algosettings#conditional-settings) to specify preprocessor symbols and other build settings for the build mode. |
 | <a id="useGitSubmodules"></a>useGitSubmodules | If your repository is using Git Submodules, you can set the `useGitSubmodules` setting to `"true"` or `"recursive"` in order to use these submodules during build workflows. If `useGitSubmodules` is not set, git submodules are not initialized. If the submodules reside in private repositories, you need to define a `gitSubmodulesToken` secret. Read [this](https://aka.ms/algosecrets#gitSubmodulesToken) for more information. |
 | <a id="commitOptions"></a>commitOptions | If you want more control over how AL-Go creates pull requests or commits changes to the repository you can define `commitOptions`. It is a structure defining how you want AL-Go to handle automated commits or pull requests coming from AL-Go (e.g. for Update AL-Go System Files). The structure contains the following properties:<br />**messageSuffix** = A string you want to append to the end of commits/pull requests created by AL-Go. This can be useful if you are using the Azure Boards integration (or similar integration) to link commits to work items. <br />`createPullRequest` : A boolean defining whether AL-Go should create a pull request or attempt to push directly in the branch.<br />**pullRequestAutoMerge** = A boolean defining whether you want AL-Go pull requests to be set to auto-complete. This will auto-complete the pull requests once all checks are green and all required reviewers have approved.<br />**pullRequestLabels** = A list of labels to add to the pull request. The labels need to be created in the repository before they can be applied.<br />If you want different behavior in different AL-Go workflows you can add the `commitOptions` setting to your [workflow-specific settings files](https://github.com/microsoft/AL-Go/blob/main/Scenarios/settings.md#where-are-the-settings-located). |
+| <a id="allowCustomJobsInEndRepos"></a>allowCustomJobsInEndRepos | Controls whether custom jobs are preserved in final repositories (repositories with a `templateUrl` pointing to another repository) when running Update AL-Go System Files. Default is **false**, which prevents custom jobs from persisting in final repositories to avoid jobs removed from templates continuing to exist. Set to **true** to allow custom jobs in final repositories. | false |
 | <a id="incrementalBuilds"></a>incrementalBuilds | A structure defining how you want AL-Go to handle incremental builds. When using incremental builds for a build, AL-Go will look for the latest successful CI/CD build, newer than the defined `retentionDays` and only rebuild projects or apps (based on `mode`) which needs to be rebuilt. The structure supports the following properties:<br />**onPush** = Determines whether incremental builds is enabled in CI/CD triggered by a merge/push event. Default is **false**.<br />**onPull_Request** = Determines whether incremental builds is enabled in Pull Requests. Default is **true**.<br />**onSchedule** = Determines whether incremental builds is enabled in CI/CD when running on a schedule. Default is **false**.<br />**retentionDays** = Number of days a successful build is good (and can be used for incremental builds). Default is **30**.<br />**mode** = Specifies the mode for incremental builds. Currently, two values are supported. Use **modifiedProjects** when you want to rebuild all apps in all modified projects and depending projects or **modifiedApps** if you want to rebuild modified apps and all apps with dependencies to this app.<br />**NOTE:** when running incremental builds, it is recommended to also set `workflowConcurrency` for the CI/CD workflow, as defined [here](https://aka.ms/algosettings#workflowConcurrency). |
 
 <a id="advanced"></a>
@@ -408,7 +409,10 @@ In the `needs` property, you specify which jobs should be complete before this j
     strategy: ${{ fromJson(needs.Initialization.outputs.environmentsMatrixJson) }}
 ```
 
-Custom jobs will be preserved when running Update AL-Go System Files.
+Custom jobs will be preserved when running Update AL-Go System Files, with the following behavior:
+
+- **Template repositories** (repositories without a `templateUrl` setting): Custom jobs are preserved and applied during updates
+- **Final repositories** (repositories with a `templateUrl` pointing to another repository): Custom jobs from the current repository are NOT applied during updates to prevent persistence of jobs removed from the template, unless the `allowCustomJobsInEndRepos` setting is set to `true`
 
 **Note** that installing [apps from the GitHub marketplace](https://github.com/marketplace?type=apps) might require you to add custom jobs or steps to some of the workflows to get the right integration. In custom jobs, you can use any [actions from the GitHub marketplace](https://github.com/marketplace?type=actions).