Add CI_JOB_TOKEN support

[retu2libc]

From the README:

Gitlab's $CI_BUILD_TOKEN infrastructure doesn't support accessing artifacts, so a private token must be used. This is very unfortunate and kludgey. This might be fixed in future Gitlab releases (if I bug them hard enough).

This seems to have been resolved on Gitlab's end since setting the request header JOB-TOKEN to $CI_JOB_TOKEN allows you to query the Gitlab artifacts api.

[JonathonReinhart]

Hi Chris,

Thanks for the heads-up.

It's not clear what, if anything needs to be done here to enable this.

1. It would be great to have a reference to GitLab docs / release notes describing what they've allowed with CI_JOB_TOKEN. We would need to make sure all of the API calls Art needs are permitted. And if there is any special configuration (e.g. GitLab project settings), this would need to be referenced.

2. Art uses the upstream python-gitlab client API. So if the HTTP headers need to be changed or anything, it will need to happen there, and Art will have to track the new version.

[haboustak]

A CI_JOB_TOKEN can be used via python-gitlab if we change the Gitlab client constructor to use job_token rather than private_token. This would require a backward-compatible option on art configure, and an extension of the configuration file.

For example, the first two commands would configure personal access tokens, the final command would configure a CI_JOB_TOKEN credential.

$ art configure $ART_USER_TOKEN
$ art configure --token-type private $ART_USER_TOKEN
$ art configure --token-type job $CI_JOB_TOKEN

However in 15.9 GitLab started taking steps to nerf CI_JOB_TOKEN, as this type of credential allows CI to act as the user that triggered the build for the duration of the pipeline and potentially has more permissions than it needs. While it's not entirely clear to me how one would leak a CI_JOB_TOKEN without also being able to act as the CI user for the duration of the job, GitLab views the use of these tokens as a security risk and has implemented additional controls.

Our use of art involves a personal access token from a dedicated "art" user. This user doesn't have permissions to any project and can only access the artifacts of projects with visibility level public or internal. I would imagine that finer-grain access controls would involve an explicit grant of permission to the "art" user on a per-project or per-group basis with the guest or reporter role, but I haven't personally evaluated what role is required to download artifacts using the API.

So ephemeral tokens are clearly better than long-lived tokens, but reduced-rights tokens are clearly better than those with full-rights. I think artifact security is irrelevant until you start to collect them from projects with the private visibility level.

[haboustak]

I think artifact security is irrelevant until you start to collect them from projects with the private visibility level.

I don't mean this to convey a cavalier attitude towards security, but rather "my impression is that this is when security starts to matter."

[kosma]

Just my two cents as the author: supporting artifacts from private repositories is essential and it's not a negotiable feature.

[haboustak]

Just my two cents as the author: supporting artifacts from private repositories is essential and it's not a negotiable feature.

This would be possible with either token type.

For CI_JOB_TOKEN, I find the GitLab docs post-15.9 regarding permissions a little difficult to understand, but if you enable the "token access" feature in CI/CD settings (enabled by default for new projects), then a project with visibility level private will need to maintain an "allow-list" of projects that can pull artifacts.

For the "art" user case, a private project needs to grant permissions to the "art" user to download artifacts (same as it is today).