[TEP-0076] Add array support for emitting results

This commit provides support for emitting array results via changing TaskRunResult value from string to ArrayorString and add ResultValue for PipelineResourceResult. Previous to this commit we can only emit string type result.

Author: Yongxuanzhang

docs/tasks.md

@@ -330,7 +330,7 @@ steps:
       echo "I am supposed to sleep for 60 seconds!"
       sleep 60
     timeout: 5s
-``` 
+```
 
 #### Specifying `onError` for a `step`
 
@@ -450,7 +450,7 @@ Parameter names:
 
 For example, `foo.Is-Bar_` is a valid parameter name, but `barIsBa$` or `0banana` are not.
 
-> NOTE: 
+> NOTE:
 > 1. Parameter names are **case insensitive**. For example, `APPLE` and `apple` will be treated as equal. If they appear in the same TaskSpec's params, it will be rejected as invalid.
 > 2. If a parameter name contains dots (.), it must be referenced by using the [bracket notation](#substituting-parameters-and-resources) with either single or double quotes i.e. `$(params['foo.bar'])`, `$(params["foo.bar"])`. See the following example for more information.
 
@@ -684,6 +684,30 @@ or [at the `Pipeline` level](./pipelines.md#configuring-execution-results-at-the
 **Note:** The maximum size of a `Task's` results is limited by the container termination message feature of Kubernetes,
 as results are passed back to the controller via this mechanism. At present, the limit is
 ["4096 bytes"](https://github.com/kubernetes/kubernetes/blob/96e13de777a9eb57f87889072b68ac40467209ac/pkg/kubelet/container/runtime.go#L632).
+
+**Note:**  The result type currently support `string` and `array` (`array` is alpha gated feature), you can write `array` results via JSON escaped format. In the example below, the task specifies one files in the `results` field and write `array` to the file. And `array` is currently supported in Task level not in Pipeline level.
+
+```
+kind: Task
+apiVersion: tekton.dev/v1beta1
+metadata:
+  name: write-array
+  annotations:
+    description: |
+      A simple task that writes array
+spec:
+  results:
+    - name: array-results
+      type: array
+      description: The array results
+  steps:
+    - name: write-array
+      image: bash:latest
+      script: |
+        #!/usr/bin/env bash
+        echo -n "[\"hello\",\"world\"]" | tee $(results.array-results.path)
+```
+
 Results are written to the termination message encoded as JSON objects and Tekton uses those objects
 to pass additional information to the controller. As such, `Task` results are best suited for holding
 small amounts of data, such as commit SHAs, branch names, ephemeral namespaces, and so on.
@@ -1181,10 +1205,10 @@ log into the `Pod` and add a `Step` that pauses the `Task` at the desired stage.
 
 ### Running Step Containers as a Non Root User
 
-All steps that do not require to be run as a root user should make use of TaskRun features to 
-designate the container for a step runs as a user without root permissions. As a best practice, 
-running containers as non root should be built into the container image to avoid any possibility 
-of the container being run as root. However, as a further measure of enforcing this practice, 
+All steps that do not require to be run as a root user should make use of TaskRun features to
+designate the container for a step runs as a user without root permissions. As a best practice,
+running containers as non root should be built into the container image to avoid any possibility
+of the container being run as root. However, as a further measure of enforcing this practice,
 steps can make use of a `securityContext` to specify how the container should run.
 
 An example of running Task steps as a non root user is shown below:
@@ -1228,17 +1252,17 @@ spec:
       runAsUser: 1001
 ```
 
-In the example above, the step `show-user-2000` specifies via a `securityContext` that the container 
-for the step should run as user 2000. A `securityContext` must still be specified via a TaskRun `podTemplate` 
-for this TaskRun to run in a Kubernetes environment that enforces running containers as non root as a requirement. 
+In the example above, the step `show-user-2000` specifies via a `securityContext` that the container
+for the step should run as user 2000. A `securityContext` must still be specified via a TaskRun `podTemplate`
+for this TaskRun to run in a Kubernetes environment that enforces running containers as non root as a requirement.
 
-The `runAsNonRoot` property specified via the `podTemplate` above validates that steps part of this TaskRun are 
-running as non root users and will fail to start any step container that attempts to run as root. Only specifying 
-`runAsNonRoot: true` will not actually run containers as non root as the property simply validates that steps are not 
+The `runAsNonRoot` property specified via the `podTemplate` above validates that steps part of this TaskRun are
+running as non root users and will fail to start any step container that attempts to run as root. Only specifying
+`runAsNonRoot: true` will not actually run containers as non root as the property simply validates that steps are not
 running as root. It is the `runAsUser` property that is actually used to set the non root user ID for the container.
 
-If a step defines its own `securityContext`, it will be applied for the step container over the `securityContext` 
-specified at the pod level via the TaskRun `podTemplate`. 
+If a step defines its own `securityContext`, it will be applied for the step container over the `securityContext`
+specified at the pod level via the TaskRun `podTemplate`.
 
 More information about Pod and Container Security Contexts can be found via the [Kubernetes website](https://kubernetes.io/docs/tasks/configure-pod-container/security-context/#set-the-security-context-for-a-pod).
 

examples/v1beta1/taskruns/alpha/emit-array-results.yaml

@@ -0,0 +1,39 @@
+kind: Task
+apiVersion: tekton.dev/v1beta1
+metadata:
+  name: write-array
+  annotations:
+    description: |
+      A simple task that writes array
+spec:
+  results:
+    - name: array-results
+      type: array
+      description: The array results
+  steps:
+    - name: write-array
+      image: bash:latest
+      script: |
+        #!/usr/bin/env bash
+        echo -n "[\"hello\",\"world\"]" | tee $(results.array-results.path)
+    - name: check-results-array
+      image: ubuntu
+      script: |
+        #!/bin/bash
+        VALUE=$(cat $(results.array-results.path))
+        EXPECTED=[\"hello\",\"world\"]
+        diff=$(diff <(printf "%s\n" "${VALUE[@]}") <(printf "%s\n" "${EXPECTED[@]}"))
+        if [[ -z "$diff" ]]; then
+            echo "TRUE"
+        else
+            echo "FALSE"
+        fi
+---
+kind: TaskRun
+apiVersion: tekton.dev/v1beta1
+metadata:
+  name: write-array-tr
+spec:
+  taskRef:
+    name: write-array
+    kind: task

pkg/apis/pipeline/v1beta1/openapi_generated.go

@@ -141,17 +141,43 @@ type ArrayOrString struct {
 
 // UnmarshalJSON implements the json.Unmarshaller interface.
 func (arrayOrString *ArrayOrString) UnmarshalJSON(value []byte) error {
-	switch value[0] {
-	case '[':
-		arrayOrString.Type = ParamTypeArray
-		return json.Unmarshal(value, &arrayOrString.ArrayVal)
-	case '{':
-		arrayOrString.Type = ParamTypeObject
-		return json.Unmarshal(value, &arrayOrString.ObjectVal)
-	default:
+	// ArrayOrString is used for Results Value as well, the results can be any kind of
+	// data so we need to check if it is empty.
+	if len(value) == 0 {
 		arrayOrString.Type = ParamTypeString
-		return json.Unmarshal(value, &arrayOrString.StringVal)
+		return nil
+	}
+	if value[0] == '[' {
+		// We're trying to Unmarshal to []string, but for cases like []int or other types
+		// of nested array which we don't support yet, we should continue and Unmarshal
+		// it to String. If the Type being set doesn't match what it actually should be,
+		// it will be captured by validation in reconciler.
+		// if failed to unmarshal to array, we will convert the value to string and marshal it to string
+		var a []string
+		if err := json.Unmarshal(value, &a); err == nil {
+			arrayOrString.Type = ParamTypeArray
+			arrayOrString.ArrayVal = a
+			return nil
+		}
+	}
+	if value[0] == '{' {
+		// if failed to unmarshal to map, we will convert the value to string and marshal it to string
+		var m map[string]string
+		if err := json.Unmarshal(value, &m); err == nil {
+			arrayOrString.Type = ParamTypeObject
+			arrayOrString.ObjectVal = m
+			return nil
+		}
+	}
+
+	// By default we unmarshal to string
+	arrayOrString.Type = ParamTypeString
+	if err := json.Unmarshal(value, &arrayOrString.StringVal); err == nil {
+		return nil
 	}
+	arrayOrString.StringVal = string(value)
+
+	return nil
 }
 
 // MarshalJSON implements the json.Marshaller interface.

pkg/apis/pipeline/v1beta1/param_types.go

@@ -227,6 +227,10 @@ func TestArrayOrString_UnmarshalJSON(t *testing.T) {
 		input  map[string]interface{}
 		result v1beta1.ArrayOrString
 	}{
+		{
+			input:  map[string]interface{}{"val": 123},
+			result: *v1beta1.NewArrayOrString("123"),
+		},
 		{
 			input:  map[string]interface{}{"val": "123"},
 			result: *v1beta1.NewArrayOrString("123"),
@@ -282,6 +286,49 @@ func TestArrayOrString_UnmarshalJSON(t *testing.T) {
 	}
 }
 
+func TestArrayOrString_UnmarshalJSON_Directly(t *testing.T) {
+	cases := []struct {
+		desc     string
+		input    string
+		expected v1beta1.ArrayOrString
+	}{
+		{desc: "empty value", input: ``, expected: *v1beta1.NewArrayOrString("")},
+		{desc: "int value", input: `1`, expected: *v1beta1.NewArrayOrString("1")},
+		{desc: "int array", input: `[1,2,3]`, expected: *v1beta1.NewArrayOrString("[1,2,3]")},
+		{desc: "nested array", input: `[1,\"2\",3]`, expected: *v1beta1.NewArrayOrString(`[1,\"2\",3]`)},
+		{desc: "string value", input: `hello`, expected: *v1beta1.NewArrayOrString("hello")},
+		{desc: "array value", input: `["hello","world"]`, expected: *v1beta1.NewArrayOrString("hello", "world")},
+		{desc: "object value", input: `{"hello":"world"}`, expected: *v1beta1.NewObject(map[string]string{"hello": "world"})},
+	}
+
+	for _, c := range cases {
+		aos := v1beta1.ArrayOrString{}
+		if err := aos.UnmarshalJSON([]byte(c.input)); err != nil {
+			t.Errorf("Failed to unmarshal input '%v': %v", c.input, err)
+		}
+		if !reflect.DeepEqual(aos, c.expected) {
+			t.Errorf("Failed to unmarshal input '%v': expected %+v, got %+v", c.input, c.expected, aos)
+		}
+	}
+}
+
+func TestArrayOrString_UnmarshalJSON_Error(t *testing.T) {
+	cases := []struct {
+		desc  string
+		input string
+	}{
+		{desc: "empty value", input: "{\"val\": }"},
+		{desc: "wrong beginning value", input: "{\"val\": @}"},
+	}
+
+	for _, c := range cases {
+		var result ArrayOrStringHolder
+		if err := json.Unmarshal([]byte(c.input), &result); err == nil {
+			t.Errorf("Should return err but got nil '%v'", c.input)
+		}
+	}
+}
+
 func TestArrayOrString_MarshalJSON(t *testing.T) {
 	cases := []struct {
 		input  v1beta1.ArrayOrString

pkg/apis/pipeline/v1beta1/param_types_test.go

@@ -39,7 +39,7 @@ type TaskRunResult struct {
 	Type ResultsType `json:"type,omitempty"`
 
 	// Value the given value of the result
-	Value string `json:"value"`
+	Value ArrayOrString `json:"value"`
 }
 
 // ResultsType indicates the type of a result;
@@ -54,7 +54,9 @@ type ResultsType string
 // Valid ResultsType:
 const (
 	ResultsTypeString ResultsType = "string"
+	ResultsTypeArray  ResultsType = "array"
+	ResultsTypeObject ResultsType = "object"
 )
 
 // AllResultsTypes can be used for ResultsTypes validation.
-var AllResultsTypes = []ResultsType{ResultsTypeString}
+var AllResultsTypes = []ResultsType{ResultsTypeString, ResultsTypeArray, ResultsTypeObject}

pkg/apis/pipeline/v1beta1/result_types.go

@@ -17,22 +17,21 @@ import (
 	"context"
 	"fmt"
 
+	"github.com/tektoncd/pipeline/pkg/apis/config"
 	"knative.dev/pkg/apis"
 )
 
 // Validate implements apis.Validatable
-func (tr TaskResult) Validate(_ context.Context) *apis.FieldError {
+func (tr TaskResult) Validate(ctx context.Context) (errs *apis.FieldError) {
 	if !resultNameFormatRegex.MatchString(tr.Name) {
 		return apis.ErrInvalidKeyName(tr.Name, "name", fmt.Sprintf("Name must consist of alphanumeric characters, '-', '_', and must start and end with an alphanumeric character (e.g. 'MyName',  or 'my-name',  or 'my_name', regex used for validation is '%s')", ResultNameFormat))
 	}
-	// Validate the result type
-	validType := false
-	for _, allowedType := range AllResultsTypes {
-		if tr.Type == allowedType {
-			validType = true
-		}
+	// Array and Object is alpha feature
+	if tr.Type == ResultsTypeArray || tr.Type == ResultsTypeObject {
+		return errs.Also(ValidateEnabledAPIFields(ctx, "results type", config.AlphaAPIFields))
 	}
-	if !validType {
+
+	if tr.Type != ResultsTypeString {
 		return apis.ErrInvalidValue(tr.Type, "type", fmt.Sprintf("type must be string"))
 	}
 

pkg/apis/pipeline/v1beta1/result_validation.go

@@ -2605,8 +2605,8 @@
         },
         "value": {
           "description": "Value the given value of the result",
-          "type": "string",
-          "default": ""
+          "default": {},
+          "$ref": "#/definitions/v1beta1.ArrayOrString"
         }
       }
     },

pkg/apis/pipeline/v1beta1/swagger.json

@@ -305,7 +305,7 @@ func TestTaskSpecValidate(t *testing.T) {
 			}},
 		},
 	}, {
-		name: "valid result type",
+		name: "valid result type string",
 		fields: fields{
 			Steps: []v1beta1.Step{{
 				Image: "my-image",
@@ -317,6 +317,32 @@ func TestTaskSpecValidate(t *testing.T) {
 				Description: "my great result",
 			}},
 		},
+	}, {
+		name: "valid result type array",
+		fields: fields{
+			Steps: []v1beta1.Step{{
+				Image: "my-image",
+				Args:  []string{"arg"},
+			}},
+			Results: []v1beta1.TaskResult{{
+				Name:        "MY-RESULT",
+				Type:        v1beta1.ResultsTypeArray,
+				Description: "my great result",
+			}},
+		},
+	}, {
+		name: "valid result type object",
+		fields: fields{
+			Steps: []v1beta1.Step{{
+				Image: "my-image",
+				Args:  []string{"arg"},
+			}},
+			Results: []v1beta1.TaskResult{{
+				Name:        "MY-RESULT",
+				Type:        v1beta1.ResultsTypeObject,
+				Description: "my great result",
+			}},
+		},
 	}, {
 		name: "valid task name context",
 		fields: fields{