Support custom CRDs for ExtensionServer in Standalone Mode #7331
Conversation
…nt scheme Teach the offline controller to recognise custom extension-server policy GVKs supplied by an extension so they can be decoded and later processed by the translator. - newOfflineGatewayAPIClient now accepts extServerPoliciesGVKs - each GVK is registered as Unstructured/UnstructuredList in the fake client's scheme - call-site updated to pass the collected policy GVKs Signed-off-by: daum3ns <[email protected]>
…backend resource registration Signed-off-by: daum3ns <[email protected]>
when reading from yaml Signed-off-by: daum3ns <[email protected]>
Add policy resources from ExtensionManager to extGKs so custom extension policies are recognised during translation. Signed-off-by: daum3ns <[email protected]>
Signed-off-by: daum3ns <[email protected]>
Signed-off-by: daum3ns <[email protected]>
server and policy resource Signed-off-by: daum3ns <[email protected]>
…f an EnvoyGateway configuration containing a standalone extension server. Signed-off-by: daum3ns <[email protected]>
Signed-off-by: daum3ns <[email protected]>
yaml and it gets the default namespace Signed-off-by: daum3ns <[email protected]>
Signed-off-by: daum3ns <[email protected]>
Signed-off-by: daum3ns <[email protected]>
can check custom resource definitions Signed-off-by: daum3ns <[email protected]>
Signed-off-by: daum3ns <[email protected]>
daum3ns
force-pushed
the
support-custom-CRDs-for-ExtensionServer-in-Standalone-Mode
branch
from
1f7a84b to
d66478e
Compare
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## main #7331 +/- ##
==========================================
+ Coverage 70.92% 72.40% +1.48%
==========================================
Files 229 231 +2
Lines 41071 34084 -6987
==========================================
- Hits 29131 24680 -4451
+ Misses 10221 7629 -2592
- Partials 1719 1775 +56 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
Signed-off-by: daum3ns <[email protected]>
ExtensionManager.resources and ExtensionManager.backendResources, use assertions Signed-off-by: daum3ns <[email protected]>
|
@shawnh2 i have added a test scenario which shows we have the same problem for the other 2 resources fields in the extension manager in standalone mode.... i think i will address this in this PR as well.. (or would you prefer a separate Issue/PR ?) |
Signed-off-by: daum3ns <[email protected]>
… ExtensionRefFilters Signed-off-by: daum3ns <[email protected]>
Signed-off-by: daum3ns <[email protected]>
Signed-off-by: daum3ns <[email protected]>
Signed-off-by: daum3ns <[email protected]>
Signed-off-by: daum3ns <[email protected]>
extensionmanager.backendResources Signed-off-by: daum3ns <[email protected]>
| for _, policy := range srv.EnvoyGateway.ExtensionManager.PolicyResources { | ||
| if policy.Kind == un.GetKind() && policy.Version == un.GroupVersionKind().Version && policy.Group == un.GroupVersionKind().Group { | ||
| un.SetNamespace(namespace) | ||
| resources.ExtensionServerPolicies = append(resources.ExtensionServerPolicies, *un) |
There was a problem hiding this comment.
why items of PolicyResources will be appended into ExtensionServerPolicies? instead of ExtensionRefFilters?
There was a problem hiding this comment.
I looked up how the kubernetes controller does it:
it puts PolicyResources in ExtensionServerPolicies
https://github.com/envoyproxy/gateway/blob/main/internal/provider/kubernetes/controller.go#L2685
and backendResources to ExtensionRefFilters:
https://github.com/envoyproxy/gateway/blob/main/internal/provider/kubernetes/controller.go#L793
internal/gatewayapi/resource/load.go
Outdated
|
|
||
| // loadKubernetesYAMLToResources converts a Kubernetes YAML string into GatewayAPI Resources. | ||
| func loadKubernetesYAMLToResources(input []byte, addMissingResources bool) (*Resources, error) { | ||
| func loadKubernetesYAMLToResources(srv *config.Server, input []byte, addMissingResources bool) (*Resources, error) { |
There was a problem hiding this comment.
can we only pass srv.EnvoyGateway.ExtensionManager here? since it's the only field in srv we need.
And also can we make it the last parameter of this function?
| assert.True(t, reconciler.client.Scheme().Recognizes(schema.GroupVersionKind{Group: extBackendGVK.Group, Version: extBackendGVK.Version, Kind: extBackendGVK.Kind})) | ||
|
|
||
| // Verify the custom resource can be loaded from YAML | ||
| inFile := "./testdata/custom-resource.yaml" |
There was a problem hiding this comment.
| inFile := "./testdata/custom-resource.yaml" | |
| inFile := "testdata/custom-resource.yaml" |
instead of srv.EnvoyGateway (replace function parameter, adapt callers and tests) Signed-off-by: daum3ns <[email protected]>
Signed-off-by: daum3ns <[email protected]>
3 participants
This change enables the file-based (standalone) provider to understand and load custom resources that are defined by an extension server.
LoadResourcesFromYAMLBytesnow accepts a server config pointer. When a resource kind is unknown, the loader checks the configured ExtensionManager and, if the GVK matches a declared policy resource, stores the object as an ExtensionServerPolicynewOfflineGatewayAPIClientfunction now accepts extensionServerPoliciesGVKs, which are then registered in the scheme. this allows the offline reconciler to manipulate custom resourcesFixes #7141
Release Notes: Yes/No
Notes:
I found out that we most probably have the same problem with ExtensionManagers
backendResourcesandresourcesfield (at least the config_loader test can easily be extended to show that the registration in the scheme is missing).I think it could be fixed in a similar way (although not sure whether the translator needs to be adapted as well). See the commented code snippets...
However, I'm very new to envoy-gateway so I'm still raising this PR to get a first, much appreciated feedback whether it makes sense to continue in this direction..