This repository was archived by the owner on Mar 21, 2025. It is now read-only.
Run tekton tasks tests in parallel #432
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,24 @@ | ||
| package main | ||
|
|
||
| import ( | ||
| "flag" | ||
| "log" | ||
| "os" | ||
|
|
||
| "github.com/opendevstack/pipeline/internal/testoutput" | ||
| ) | ||
|
|
||
| type options struct { | ||
| testResultDir string | ||
| } | ||
|
|
||
| func main() { | ||
| opts := options{} | ||
| flag.StringVar(&opts.testResultDir, "test-result-dir", os.Getenv("TEST_RESULT_DIR"), "test-result-dir") | ||
| flag.Parse() | ||
|
|
||
| err := testoutput.DisentangleTestOutputs(opts.testResultDir, os.Stdin, os.Stdout) | ||
| if err != nil { | ||
| log.Fatalf("could not disentangle test output: %v", err) | ||
| } | ||
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,77 @@ | ||
| package testoutput | ||
|
|
||
| import ( | ||
| "encoding/json" | ||
| "fmt" | ||
| "io" | ||
| "os" | ||
| "path/filepath" | ||
| "time" | ||
| ) | ||
|
|
||
| // TestEvent taken from `go doc test2json`, unfortunately, this struct is not | ||
| // exposed from any core go packages so we have to define it ourselves | ||
| type TestEvent struct { | ||
| Time time.Time `json:",omitempty"` | ||
| Action string | ||
| Package string `json:",omitempty"` | ||
| Test string `json:",omitempty"` | ||
| Elapsed float64 `json:",omitempty"` | ||
| Output string `json:",omitempty"` | ||
| } | ||
|
|
||
| // DisentangleTestOutputs separates test outputs from a stream of interleaved | ||
| // test events into separate files per individual testcase | ||
| func DisentangleTestOutputs(testResultsPath string, in *os.File, out *os.File) error { | ||
| // holds a mapping from test to the file handles the output is redirected to | ||
| var fileHandles = make(map[string]*os.File) | ||
| decoder := json.NewDecoder(in) | ||
|
|
||
| for { | ||
| var event TestEvent | ||
| if err := decoder.Decode(&event); err == io.EOF { | ||
| break | ||
| } else if err != nil { | ||
| return err | ||
| } | ||
|
|
||
| // switch over fixed set of actions (c.f. `go doc test2json`) | ||
| switch event.Action { | ||
| case "run": | ||
| filePath := filepath.Join(testResultsPath, event.Test+".log") | ||
| dir := filepath.Dir(filePath) | ||
| err := os.MkdirAll(dir, 0777) | ||
| if err != nil { | ||
| return fmt.Errorf("failed to create directory: %w", err) | ||
| } | ||
|
|
||
| f, err := os.OpenFile(filePath, os.O_CREATE|os.O_WRONLY|os.O_TRUNC, 0644) | ||
| if err != nil { | ||
| return fmt.Errorf("failed to open output file: %w", err) | ||
| } | ||
| fileHandles[event.Test] = f | ||
| case "pause", "cont", "skip": // do nothing | ||
|
There was a problem hiding this comment. Similarly to #432 (comment) the output for a skipped test would be captured in the prior output so it would only lead to us printing information twice if we react on |
||
| case "pass", "fail": | ||
| f, ok := fileHandles[event.Test] | ||
|
There was a problem hiding this comment. Should there be some error reporting if it is not ok? |
||
| if ok { | ||
| err := f.Close() | ||
|
There was a problem hiding this comment. I assume test pass and fail is typically captured in prior output. Perhaps its worthwhile to nonetheless write this explicitly in the log. There was a problem hiding this comment. @henrjk you're spot-on, There was a problem hiding this comment. The object for events of type So the only additional log that we could provide reacting on |
||
| if err != nil { | ||
| return fmt.Errorf("could not close file handle: %w", err) | ||
| } | ||
| } | ||
| case "bench", "output": | ||
| // 1. redirect to file | ||
| f, ok := fileHandles[event.Test] | ||
| if ok { | ||
| if _, err := fmt.Fprint(f, event.Output); err != nil { | ||
| return fmt.Errorf("failed to write to output file: %w", err) | ||
| } | ||
| } | ||
| // 2. echo the output with the json stripped off to `out` | ||
| if _, err := fmt.Fprint(out, event.Output); err != nil { | ||
| return fmt.Errorf("failed to write to output stream: %w", err) | ||
| } | ||
| } | ||
| } | ||
| return nil | ||
| } | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I am curious did you need this file flags and permissions? Have you different flags/permissions first - perhaps os.Create() and it did not work?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Good catch, we'll check
os.Create()instead, looks like this is a convenience wrapper aroundos.OpenFile()