HTTPRequestHeaderFilter should specify the order of operations

[jpeach]

What happened:

HTTPRequestHeaderFilter has add/set/remove operations that manipulate the set of request headers. Depending on the order that a proxy applies these operations, the final result can be different.

What you expected to happen:

Header manipulations should be consistent across proxies.

[fejta-bot]

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-contributor-experience at kubernetes/community.
/lifecycle stale

[hbagdi]

/remove-lifecycle stale
/lifecycle frozen

[shaneutt]

It seems plausible that there may be situations where you would have multiple verbs acting on the same header. For instance, axiomatically a user might use Set followed by Add to erase any values previously associated with a header, and then add another value on top of that. We are marking this help wanted as we're looking for contributors with strong use cases to help champion and drive this forward.

[tokers]

@shaneutt I can help do this if possible :)

[tokers]

/assign

[tokers]

I think another case is clarifying the behavior for Set, Let's say we have two X-Foo headers:

X-Foo: v1
X-Foo: v2

What will happen after we apply the rule:

set:
  X-Foo: v3

Will we have two X-Foo headers and their values are v3 or we just have one X-Foo header (Because we can treat the original X-Foo header's value as v1, v2.

[rainest]

I think it's reasonable to approach this by simply rejecting any configuration that includes multiple Sets or Adds (or both a Set and Add) for the same header, which should be simpler to implement since you can handle it entirely in Gateway API resource processing and don't need to worry about whether an implementation can easily conform to a given order. Is there a good reason we'd need to allow more than one Set or Add for the same header?

Per https://www.rfc-editor.org/rfc/rfc7230#section-3.2.2 multiple instances of the same header essentially always end up as a single CSV It is a bit ambiguous since it says senders must not generate multiple instances of the same header, but if they do recipients may combine them, without indicating what should happen if recipients opt to not combine them--maybe discard the additional instances after the first? In any case, it does seem clear that a CSV is the correct version of multiple values.

I don't know of an obvious reason why we shouldn't be able to require that users adding HTTPRequestHeaderFilters add multiple values as a single header instance with a CSV in advance: the values are going to end up in that form anyway, HTTPRequestHeaderFilters aren't standalone resources and can't be natively reused across HTTPRouteRules, and requiring that removes user-facing ambiguity about the final order (you don't need to know the spec to know what the order of the values will be because you've specified the order in its final form already), and I can't think of any configuration that you could express with multiple Sets/Adds that you couldn't with a single CSV Set/Add.

We would still need to worry Adds applied to a request that already contains that header, since we would still need to choose an order there. We'd also need to determine how to reject it--presumably we'd use some Condition on the HTTPRoute to indicate the issue, but we'd then need to decide whether we do not manipulate that header at all or choose one of the Adds/Sets to use.

In either case, whether we only handle Adds atop existing request headers or allow users to specify multiple Sets and Adds for the same header, given the

[recipients may handle multiple of the same header] by appending each subsequent field value to the combined field value in order

in the RFC language, I do think we'd want the order to be existing,first add,second add. Multiple Sets or mixed Sets and Adds are more complicated since, per the above, it's unclear whether they'd be absolute (you only get the value of one of the Sets) or additive (you get a CSV of values from all Sets and Adds), which I think is more argument in favor of just rejecting them.

FWIW, in our existing header mutation implementation we can easily control the order of headers within an operation, but don't have the ability to control operation order: our Sets always come before our Adds. We would not be able to easily implement a spec that required that Sets override Adds (if you have Set X-Foo: a and Add X-Foo: b you get X-Foo: a instead of X-Foo: a, b) or that Add values come first (if you have Set X-Foo: a and Add X-Foo: b you get X-Foo: b, a instead of X-Foo: a, b).