Change in code-generator v0.20.0 breaks operation overrides

[jaypipes]

The top part of kinesis-controller's generator.yaml file looks like this:

operations:
  # NOTE)jaypipes): According to
  # https://docs.aws.amazon.com/kinesis/latest/APIReference/API_DescribeStream.html,
  # the DescribeStream API should not be used. Instead, the
  # DescribeStreamSummary API and ListShards API should be used.
  DescribeStreamSummary:
    resource_name: Stream
    operation_type: READ_ONE
    output_wrapper_field_path: StreamDescriptionSummary

If I execute make build-controller SERVICE=kinesis with v0.20.0 of code-generator, I get a failure:

[jaypipes@thelio code-generator]$ make build-controller SERVICE=kinesis
building ack-generate ... ok.
installing controller-gen v0.7.0 ... ok.
==== building kinesis-controller ====
Copying common custom resource definitions into kinesis
Building Kubernetes API objects for kinesis
Generating deepcopy code for kinesis
Generating custom resource definitions for kinesis
Building service controller for kinesis
Error: template: /home/jaypipes/go/src/github.com/aws-controllers-k8s/code-generator/templates/pkg/resource/sdk.go.tpl:73:3: executing "sdk_find_read_one" at <GoCodeRequiredFieldsMissingFromReadOneInput .CRD "r.ko" 1>: error calling GoCodeRequiredFieldsMissingFromReadOneInput: GENERATION FAILURE! there's a required field StreamName in Shape DescribeStreamInput that isn't in either the CR's Spec or Status structs!
make: *** [Makefile:41: build-controller] Error 1

However, if I check out v0.19.3 of the code-generator, I get no errors:

[jaypipes@thelio code-generator]$ git checkout v0.19.3
Previous HEAD position was 585f06b Upgrade ACK runtime to v0.20.0 (#360)
HEAD is now at 87477ae Merge pull request #356 from vijtrip2/ack-rt-v0.19.3
[jaypipes@thelio code-generator]$ make build-controller SERVICE=kinesis
building ack-generate ... ok.
==== building kinesis-controller ====
Copying common custom resource definitions into kinesis
Building Kubernetes API objects for kinesis
Generating deepcopy code for kinesis
Generating custom resource definitions for kinesis
Building service controller for kinesis
Generating RBAC manifests for kinesis
Running gofmt against generated code for kinesis
Updating additional GitHub repository maintenance files
==== building kinesis-controller release artifacts ====
Building release artifacts for kinesis-v0.0.0-non-release-version
Generating common custom resource definitions
Generating custom resource definitions for kinesis
Generating RBAC manifests for kinesis

So, it seems that something was added to code-gen that breaks the custom operations configuration somehow.

[jaypipes]

Hmm, seems I was wrong... it looks like it's a non-deterministic failure. Just re-ran make build-controller SERVICE=kinesis again with v0.19.3 after seeing there were no real changes between v0.19.3 and v0.20.0 that could have impacted code generation and get a failure:

[jaypipes@thelio code-generator]$ make build-controller SERVICE=kinesis
building ack-generate ... ok.
==== building kinesis-controller ====
Copying common custom resource definitions into kinesis
Building Kubernetes API objects for kinesis
Generating deepcopy code for kinesis
Generating custom resource definitions for kinesis
Building service controller for kinesis
Error: template: /home/jaypipes/go/src/github.com/aws-controllers-k8s/code-generator/templates/pkg/resource/sdk.go.tpl:73:3: executing "sdk_find_read_one" at <GoCodeRequiredFieldsMissingFromReadOneInput .CRD "r.ko" 1>: error calling GoCodeRequiredFieldsMissingFromReadOneInput: GENERATION FAILURE! there's a required field StreamName in Shape DescribeStreamInput that isn't in either the CR's Spec or Status structs!
make: *** [Makefile:41: build-controller] Error 1

I suspect something isn't sorting deterministically somewhere..

[jaypipes]

There are two bugs in the code generator's construction of the
OperationMap during API inference.

The first bug was a non-deterministic behaviour that showed itself when
an API had multiple Operations of a single operation type (say,
READ_ONE). The code that was responsible for determining which Operation
mapped to which operation type for a particular resource was looping
over the aws-sdk-go private/model/api:API's Operations field,
which is a map. Depending on the parsing of the underlying api-2.json
files, same-typed Operations with different IDs -- e.g.
DescribeStreamSummary and DescribeStream -- were being mapped to a
single operation type -- OpTypeGet. Depending on which Operation ID came
last, that is the operation that was being mapped, regardless of which
one appeared in the OperationConfig configuration option that handles
overrides.

The second bug was that the OpTypeFromString method was too strict in
its comparison logic. The strings "get", "READ_ONE", "read_one" and
"Get" should all lead to OpTypeGet, but only "Get" was being matched.