TEP-0090: Fan Out `TaskRuns` #4990

jerop · 2022-06-16T12:54:22Z

Changes

TEP-0090: Matrix proposed executing a PipelineTask in parallel TaskRuns and Runs with substitutions from combinations of Parameters in a Matrix.

This change implements the fan out of TaskRuns from a PipelineTask with a Matrix. The fanned-out TaskRuns are executed in parallel.

/kind feature

Submitter Checklist

As the author of this PR, please check off the items in this checklist:

Docs included if any changes are user facing
Tests included if any functionality added or changed
Follows the commit message standard
Meets the Tekton contributor standards (including
functionality, content, code)
Release notes block below has been filled in (if there are no user facing changes, use release note "NONE")

Release Notes

A `PipelineTask` with a `Matrix` is fanned out into parallel `TaskRuns` which are executed in parallel.

jerop · 2022-06-16T12:55:06Z

cc @abayer @dibyom @lbernick @pritidesai please take a look

tekton-robot · 2022-06-16T12:56:16Z

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage to re-run this coverage report

File	Old Coverage	New Coverage	Delta
pkg/reconciler/pipelinerun/pipelinerun.go	86.1%	85.8%	-0.3
pkg/reconciler/pipelinerun/resources/pipelinerunresolution.go	94.5%	94.2%	-0.2

tekton-robot · 2022-06-16T13:08:26Z

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage to re-run this coverage report

File	Old Coverage	New Coverage	Delta
pkg/reconciler/pipelinerun/pipelinerun.go	86.1%	85.8%	-0.3
pkg/reconciler/pipelinerun/resources/pipelinerunresolution.go	94.5%	94.6%	0.1

tekton-robot · 2022-06-16T13:28:56Z

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage to re-run this coverage report

File	Old Coverage	New Coverage	Delta
pkg/reconciler/pipelinerun/pipelinerun.go	86.1%	85.8%	-0.3
pkg/reconciler/pipelinerun/resources/pipelinerunresolution.go	94.5%	94.6%	0.1

tekton-robot · 2022-06-16T13:29:46Z

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage to re-run this coverage report

File	Old Coverage	New Coverage	Delta
pkg/reconciler/pipelinerun/pipelinerun.go	86.1%	85.8%	-0.3
pkg/reconciler/pipelinerun/resources/pipelinerunresolution.go	94.5%	94.6%	0.1

jerop · 2022-06-16T13:34:14Z

/test pull-tekton-pipeline-go-coverage

tekton-robot · 2022-06-16T13:36:19Z

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage to re-run this coverage report

File	Old Coverage	New Coverage	Delta
pkg/reconciler/pipelinerun/pipelinerun.go	86.1%	85.8%	-0.3
pkg/reconciler/pipelinerun/resources/pipelinerunresolution.go	94.5%	94.6%	0.1

tekton-robot · 2022-06-16T14:01:05Z

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage to re-run this coverage report

File	Old Coverage	New Coverage	Delta
pkg/reconciler/pipelinerun/pipelinerun.go	86.1%	85.8%	-0.3
pkg/reconciler/pipelinerun/resources/pipelinerunresolution.go	94.5%	94.6%	0.1

tekton-robot · 2022-06-16T14:09:10Z

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage to re-run this coverage report

File	Old Coverage	New Coverage	Delta
pkg/reconciler/pipelinerun/pipelinerun.go	86.1%	85.8%	-0.3
pkg/reconciler/pipelinerun/resources/pipelinerunresolution.go	94.5%	94.6%	0.1

jerop · 2022-06-16T14:58:24Z

/retest

jerop · 2022-06-16T15:25:45Z

TLS handshake timeout 🤔

/retest

jerop · 2022-06-16T16:17:47Z

TLS handshake timeout 🤔

/retest

tekton-robot · 2022-06-16T16:31:02Z

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage to re-run this coverage report

File	Old Coverage	New Coverage	Delta
pkg/reconciler/pipelinerun/pipelinerun.go	86.1%	86.0%	-0.1
pkg/reconciler/pipelinerun/resources/pipelinerunresolution.go	94.5%	94.6%	0.1

jerop · 2022-06-16T17:04:11Z

/retest

tekton-robot · 2022-06-16T17:06:29Z

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage to re-run this coverage report

File	Old Coverage	New Coverage	Delta
pkg/reconciler/pipelinerun/pipelinerun.go	86.1%	86.0%	-0.1
pkg/reconciler/pipelinerun/resources/pipelinerunresolution.go	94.5%	94.6%	0.1

jerop · 2022-06-16T17:20:48Z

/test pull-tekton-pipeline-alpha-integration-tests
/test pull-tekton-pipeline-integration-tests

jerop · 2022-06-16T18:38:34Z

/test pull-tekton-pipeline-alpha-integration-tests
/test pull-tekton-pipeline-integration-tests

pkg/reconciler/pipelinerun/pipelinerun.go

pkg/reconciler/pipelinerun/resources/pipelinerunresolution.go

tekton-robot · 2022-06-16T21:36:33Z

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage to re-run this coverage report

File	Old Coverage	New Coverage	Delta
pkg/reconciler/pipelinerun/pipelinerun.go	86.4%	86.3%	-0.1
pkg/reconciler/pipelinerun/resources/pipelinerunresolution.go	94.3%	94.4%	0.1

jerop · 2022-06-16T23:58:03Z

/test pull-tekton-pipeline-alpha-integration-tests

jerop · 2022-06-17T11:16:29Z

/test pull-tekton-pipeline-alpha-integration-tests

tekton-robot · 2022-06-17T12:14:09Z

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage to re-run this coverage report

File	Old Coverage	New Coverage	Delta
pkg/reconciler/pipelinerun/pipelinerun.go	86.4%	86.3%	-0.1
pkg/reconciler/pipelinerun/resources/pipelinerunresolution.go	94.3%	94.4%	0.1

[TEP-0090: Matrix][tep-0090] proposed executing a `PipelineTask` in parallel `TaskRuns` and `Runs` with substitutions from combinations of `Parameters` in a `Matrix`. This change implements the fan out of `TaskRuns` from a `PipelineTask` with a `Matrix`. The fanned-out `TaskRuns` are executed in parallel. [tep-0090]: https://github.com/tektoncd/community/blob/main/teps/0090-matrix.md

tekton-robot · 2022-06-17T12:42:28Z

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage to re-run this coverage report

File	Old Coverage	New Coverage	Delta
pkg/reconciler/pipelinerun/pipelinerun.go	86.4%	86.3%	-0.1
pkg/reconciler/pipelinerun/resources/pipelinerunresolution.go	94.3%	94.4%	0.1

jerop · 2022-06-17T13:14:26Z

/test pull-tekton-pipeline-integration-tests

tekton-robot · 2022-06-17T13:58:10Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: lbernick

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~OWNERS~~ [lbernick]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

abayer · 2022-06-17T14:28:41Z

/lgtm

jerop · 2022-06-17T15:45:33Z

/test pull-tekton-pipeline-alpha-integration-tests

Error from server (InternalError): error when creating "STDIN": Internal error occurred: failed calling webhook "validation.webhook.pipeline.tekton.dev": failed to call webhook: Post "https://tekton-pipelines-webhook.tekton-pipelines.svc:443/resource-validation?timeout=10s": context deadline exceeded in sidecar-ready-script

jerop · 2022-06-17T16:04:10Z

/test pull-pipeline-kind-k8s-v1-21-e2e

abayer · 2022-06-17T16:37:38Z

/test pull-tekton-pipeline-build-tests

jerop · 2022-06-17T17:06:35Z

/test pull-tekton-pipeline-build-tests

pritidesai · 2022-06-17T23:23:28Z

thanks a bunch @jerop for this, appreciate all the efforts you have put in 🎉 ❤️ Can we please add an example and an e2e test for this feature, if they do not already exist?

jerop · 2022-06-21T12:19:10Z

thanks a bunch @jerop for this, appreciate all the efforts you have put in 🎉 ❤️ Can we please add an example and an e2e test for this feature, if they do not already exist?

@pritidesai fanning out of taskruns is not yet complete - at this point we create the taskruns but they are not yet passed to the pipelinerun as described in https://github.com/tektoncd/community/blob/main/teps/0090-matrix.md#status

if i were to add examples now, the pipelinerun would stay running and timeout

opening the PRs implementing the status, which would include the examples and end-to-end tests

[TEP-0090: Matrix][tep-0090] proposed executing a `PipelineTask` in parallel `TaskRuns` and `Runs` with substitutions from combinations of `Parameters` in a `Matrix`. In #4990, we implemented fanning out of `TaskRuns`, but they were not embedded in `PipelineRun` status. Now that the minimal status are embedded through `ChildReferences`, this change validates that the reconciled `PipelineRun` has the expected `ChildReferences`. [tep-0090]: https://github.com/tektoncd/community/blob/main/teps/0090-matrix.md

tekton-robot added release-note Denotes a PR that will be considered when it comes time to generate release notes. kind/feature Categorizes issue or PR as related to a new feature. labels Jun 16, 2022

tekton-robot requested review from afrittoli and pritidesai June 16, 2022 12:54

tekton-robot added the size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. label Jun 16, 2022

jerop force-pushed the tep-0090-createtaskruns branch from 4985acd to 66534ed Compare June 16, 2022 13:05

jerop force-pushed the tep-0090-createtaskruns branch 2 times, most recently from d7d8318 to f466791 Compare June 16, 2022 13:26

jerop force-pushed the tep-0090-createtaskruns branch from f466791 to 1c85caa Compare June 16, 2022 13:49

jerop force-pushed the tep-0090-createtaskruns branch from 1c85caa to dd11cdb Compare June 16, 2022 14:06

jerop force-pushed the tep-0090-createtaskruns branch 2 times, most recently from 40e99a5 to 9640ebf Compare June 16, 2022 14:36

lbernick reviewed Jun 16, 2022

View reviewed changes

pkg/reconciler/pipelinerun/pipelinerun.go Show resolved Hide resolved

pkg/reconciler/pipelinerun/resources/pipelinerunresolution.go Outdated Show resolved Hide resolved

jerop force-pushed the tep-0090-createtaskruns branch 3 times, most recently from 910ac46 to 8826b99 Compare June 16, 2022 21:33

jerop force-pushed the tep-0090-createtaskruns branch from 8826b99 to f0a9c38 Compare June 17, 2022 12:12

jerop force-pushed the tep-0090-createtaskruns branch from f0a9c38 to cb1c733 Compare June 17, 2022 12:40

lbernick approved these changes Jun 17, 2022

View reviewed changes

tekton-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Jun 17, 2022

tekton-robot assigned abayer Jun 17, 2022

tekton-robot added the lgtm Indicates that a PR is ready to be merged. label Jun 17, 2022

tekton-robot merged commit 5e53c61 into tektoncd:main Jun 17, 2022

jerop deleted the tep-0090-createtaskruns branch June 17, 2022 17:21

jerop mentioned this pull request Jun 23, 2022

TEP-0090: Test reconciled PipelineRun from Pipeline with Matrix #5031

Merged

3 tasks

jerop mentioned this pull request Jun 28, 2022

TEP-0090: Fan Out Runs #5049

Merged

5 tasks

TEP-0090: Fan Out TaskRuns #4990

TEP-0090: Fan Out TaskRuns #4990

Uh oh!

Conversation

jerop commented Jun 16, 2022

Changes

Submitter Checklist

Release Notes

Uh oh!

jerop commented Jun 16, 2022

Uh oh!

tekton-robot commented Jun 16, 2022

Uh oh!

tekton-robot commented Jun 16, 2022

Uh oh!

tekton-robot commented Jun 16, 2022

Uh oh!

tekton-robot commented Jun 16, 2022

Uh oh!

jerop commented Jun 16, 2022

Uh oh!

tekton-robot commented Jun 16, 2022

Uh oh!

tekton-robot commented Jun 16, 2022

Uh oh!

tekton-robot commented Jun 16, 2022

Uh oh!

jerop commented Jun 16, 2022

Uh oh!

jerop commented Jun 16, 2022

Uh oh!

jerop commented Jun 16, 2022

Uh oh!

tekton-robot commented Jun 16, 2022

Uh oh!

jerop commented Jun 16, 2022

Uh oh!

tekton-robot commented Jun 16, 2022

Uh oh!

jerop commented Jun 16, 2022

Uh oh!

jerop commented Jun 16, 2022

Uh oh!

Uh oh!

Uh oh!

tekton-robot commented Jun 16, 2022

Uh oh!

jerop commented Jun 16, 2022

Uh oh!

jerop commented Jun 17, 2022

Uh oh!

tekton-robot commented Jun 17, 2022

Uh oh!

tekton-robot commented Jun 17, 2022

Uh oh!

jerop commented Jun 17, 2022

Uh oh!

tekton-robot commented Jun 17, 2022

Uh oh!

abayer commented Jun 17, 2022

Uh oh!

jerop commented Jun 17, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

jerop commented Jun 17, 2022

Uh oh!

abayer commented Jun 17, 2022

Uh oh!

jerop commented Jun 17, 2022

Uh oh!

pritidesai commented Jun 17, 2022

Uh oh!

jerop commented Jun 21, 2022

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

TEP-0090: Fan Out `TaskRuns` #4990

TEP-0090: Fan Out `TaskRuns` #4990

jerop commented Jun 17, 2022 •

edited

Loading