Add support for CI_JOB_TOKEN

[retu2libc]

Closes #26 -- Adds support for:

$ art configure $GITLAB_URL --token-type [private|job] $TOKEN

and maintains legacy support for:

$ art configure $GITLAB_URL $TOKEN

[haboustak]

Hey @christheyankee, thanks for the PR.

Overall I think this looks good, however we should only store 1 token of any type in config.yml. It's unfortunate that the current key name is private_token, rather than just token. Our goal should be for the config schema to mirror the command-line.

backwards-compatible:

private_token: 'o_bla98df'

or modern:

token: 'o_bla98df'
token_type: 'private'

We would support loading the old format, but only save the modern format.

If there are multiple tokens in the configuration, we don't know which token to use, and trying to use both tokens would create even more unpredictable behavior than already provided by the inconsistent permissions available via CI_JOB_TOKEN. When the user runs art configure TOKEN they expect art to use that token, replacing any previously defined value.

[retu2libc]

When the user runs art configure TOKEN they expect art to use that token, replacing any previously defined value.

This behavior would actually be preserved since that would store a user token and python-gitlab uses the user token over the CI token when provided both.

[haboustak]

python-gitlab uses the user token over the CI token when provided both.

python-gitlab does do that currently, but it's just a hack they must implement because they pull tokens from myriad sources.

Their order of precedence ensures there aren't errors from the GitLab request, but it doesn't make any claim that the correct token is used.

art has more reasonable configuration. You tell it what token to use, and it uses that token, because only 1 token can be used. (unless/until we add logic to retry API requests that fail authentication)

[retu2libc]

ping @haboustak since you might not have seen f33b2b8

[haboustak]

Hey @christheyankee, sorry for the delay in getting back to you - I've been traveling.

When I get a chance over the next few days I'll pull down your version and use some of my existing projects to test and ensure the conversions work as expected. Should be good to go though.

[haboustak]

I finally managed to get around to trying this out over the weekend, sorry for taking so long.

The configuration and backward-compatibility changes all worked. Unfortunately, I couldn't get art download to download artifacts with a job token without further changes, and I don't think that art update can be made to work.

When testing art download using a job token, I received a 404 Not Found response when accessing the project GitLab API endpoint, for example: GET /api/v4/projects/open-source%2Fmusl-cross-make.

According to the CI_JOB_TOKEN docs a job token cannot be used to access the project or jobs API endpoints, only job artifacts. To work around this limitation, we can change the download command to get a lazy reference to the project (see this commit)

That should fix download, but the update command needs API access to both pipelines and jobs. It's probably OK for some use cases to support download differently than update, as long as we print a very clear error message when update is called and the token-type is job.

I performed my testing so far using GitLab CE, although I don't think EE would change the results.

[haboustak]

Thanks @christheyankee!