From 78667d66443ed66910d1070e5db9ba0915effc0b Mon Sep 17 00:00:00 2001 From: Eric Bishop Date: Wed, 29 Jan 2025 11:38:43 -0500 Subject: [PATCH 1/9] GEP-3388 HTTP Retry Budget API Design --- geps/gep-3388/index.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/geps/gep-3388/index.md b/geps/gep-3388/index.md index ecf10dcd67..aa993170a0 100644 --- a/geps/gep-3388/index.md +++ b/geps/gep-3388/index.md @@ -49,7 +49,7 @@ By default, Envoy uses a static threshold for retries. But when configured, Envo The Linkerd implementation of retry budgets is configured alongside service route configuration, within the [ServiceProfile CRD](https://linkerd.io/2.12/reference/service-profiles/), limiting the number of total retries for a service as a percentage of the number of recent requests. In practice, this functions similarly to Envoy's retry budget implementation, as it is configured in a single location and measures the ratio of retry requests to original requests across all traffic destined for the service. -Linkerd uses [budgeted retries](https://linkerd.io/2.15/features/retries-and-timeouts/) as the default configuration to specify retries to a service, but - as of [edge-24.7.5](https://github.com/linkerd/linkerd2/releases/tag/edge-24.7.5) - supports counted retries. In all cases, retries are implemented by the `linkerd2-proxy` making the request on behalf on an application workload. +Usage of the ServiceProfile CRD is significantly rarer today, given Linkerd's support for per-route configuration through Gateway API Route resources. Though historically, [retry budgets](https://linkerd.io/2.15/features/retries-and-timeouts/) were the default configuration to specify retries to a service, with support for counted retries later being add in [edge-24.7.5](https://github.com/linkerd/linkerd2/releases/tag/edge-24.7.5). In all cases, retries are implemented by the `linkerd2-proxy` making the request on behalf on an application workload. Linkerd's budgeted retries allow retrying an indefinite number of times, as long as the fraction of retries remains within the budget. Budgeted retries are supported only using Linkerd's native ServiceProfile CRD, which allows enabling retries, setting the retry budget (by default, 20% plus 10 "extra" retries per second), and configuring the window over which the fraction of retries to non-retries is calculated. The `retryBudget` field of the ServiceProfile spec can be configured with the following optional parameters: From c737a824d5207cd90c0ed1962e43c71901e3446e Mon Sep 17 00:00:00 2001 From: Mike Morris <1149913+mikemorris@users.noreply.github.com> Date: Thu, 30 Jan 2025 11:21:25 -0500 Subject: [PATCH 2/9] add first draft of YAML for retry budgets --- geps/gep-3388/index.md | 111 +++++++++++++++++++++++++++++++++++- geps/gep-3388/metadata.yaml | 2 +- 2 files changed, 109 insertions(+), 4 deletions(-) diff --git a/geps/gep-3388/index.md b/geps/gep-3388/index.md index aa993170a0..41ed8477b6 100644 --- a/geps/gep-3388/index.md +++ b/geps/gep-3388/index.md @@ -1,7 +1,7 @@ # GEP-3388: Retry Budgets * Issue: [#3388](https://github.com/kubernetes-sigs/gateway-api/issues/3388) -* Status: Provisional +* Status: Implementable (See status definitions [here](/geps/overview/#gep-states).) @@ -29,7 +29,7 @@ Multiple data plane proxies offer optional configuration for budgeted retries, i Configuring a limit for client retries is an important factor in building a resilient system, allowing requests to be successfully retried during periods of intermittent failure. But too many client-side retries can also exacerbate consistent failures and slow down recovery, quickly overwhelming a failing system and leading to cascading failures such as retry storms. Configuring a sane limit for max client-side retries is often challenging in complex systems. Allowing an application developer (Ana) to configure a dynamic "retry budget" reduces the risk of a high number of retries across clients. It allows a service to perform as expected in both times of high & low request load, as well as both during periods of intermittent & consistent failures. -While retry budget configuration has been a frequently discussed feature within the community, differences in the semantics between data plane implementations creates a challenge for a consensus on the correct location for the configuration. This proposal aims to determine where retry budget's should be defined within the Gateway API, and whether data plane proxies may need to be altered to accommodate the specification. +While retry budget configuration has been a frequently discussed feature within the community, differences in the semantics between data plane implementations creates a challenge for a consensus on the correct location for the configuration. This proposal aims to determine where retry budget's should be defined within the Gateway API, and whether data plane proxies may need to be altered to accommodate the specification. ### Background on implementations @@ -85,7 +85,112 @@ TODO ### YAML -TODO +```yaml +apiVersion: gateway.networking.x-k8s.io/v1alpha1 +kind: RetryPolicy +metadata: + name: retry-policy-example +spec: + targetRefs: + - group: "" + kind: Service + name: foo + from: + - kind: Mesh + namespace: istio-system + name: istio + - kind: Gateway + name: foo-ingress + codes: + - 500 + - 502 + - 503 + - 504 + attempts: 2 + backoff: 100ms + budgetPercent: 20 + budgetInterval: 10s + minRetryRate: + count: 3 + interval: 1s +status: + ancestors: + - ancestorRef: + kind: Mesh + namespace: istio-system + name: istio + controllerName: "istio.io/mesh-controller" + conditions: + - type: "Accepted" + status: "True" + reason: "Accepted" + - ancestorRef: + kind: Gateway + namespace: foo-ns + name: foo-ingress + controllerName: "istio.io/mesh-controller" + conditions: + - type: "Accepted" + status: "False" + reason: "Invalid" + message: "RetryPolicy fields budgetPercentage, budgetInterval and minRetryRate are not supported for Istio ingress gateways." +``` + +```yaml +apiVersion: gateway.networking.x-k8s.io/v1alpha1 +kind: BackendTrafficPolicy +metadata: + name: traffic-policy-example +spec: + targetRefs: + - group: "" + kind: Service + name: foo + from: + - kind: Mesh + namespace: istio-system + name: istio + - kind: Gateway + name: foo-ingress + retry: + codes: + - 500 + - 502 + - 503 + - 504 + attempts: 2 + backoff: 100ms + budgetPercent: 20 + budgetInterval: 10s + minRetryRate: + count: 3 + interval: 1s + sessionPersistence: + ... + status: + ancestors: + - ancestorRef: + kind: Mesh + namespace: istio-system + name: istio + controllerName: "istio.io/mesh-controller" + conditions: + - type: "Accepted" + status: "False" + reason: "Invalid" + message: "BackendTrafficPolicy field sessionPersistence is not supported for Istio mesh traffic." + - ancestorRef: + kind: Gateway + namespace: foo-ns + name: foo-ingress + controllerName: "istio.io/mesh-controller" + conditions: + - type: "Accepted" + status: "False" + reason: "Invalid" + message: "BackendTrafficPolicy fields retry.budgetPercentage, retry.budgetInterval and retry.minRetryRate are not supported for Istio ingress gateways." + ... +``` ## Conformance Details diff --git a/geps/gep-3388/metadata.yaml b/geps/gep-3388/metadata.yaml index 02bbf069b8..8315a18d85 100644 --- a/geps/gep-3388/metadata.yaml +++ b/geps/gep-3388/metadata.yaml @@ -2,7 +2,7 @@ apiVersion: internal.gateway.networking.k8s.io/v1alpha1 kind: GEPDetails number: 3388 name: Retry Budgets -status: Provisional +status: Implementable # Any authors who contribute to the GEP in any way should be listed here using # their Github handle. authors: From db8bded331f609568e459fc651b7205acae9c714 Mon Sep 17 00:00:00 2001 From: Mike Morris <1149913+mikemorris@users.noreply.github.com> Date: Thu, 30 Jan 2025 12:03:27 -0500 Subject: [PATCH 3/9] add Go structs for retry budget API design proposal --- geps/gep-3388/index.md | 177 +++++++++++++++++++++++++++++++++++++---- 1 file changed, 162 insertions(+), 15 deletions(-) diff --git a/geps/gep-3388/index.md b/geps/gep-3388/index.md index 41ed8477b6..05179a22e0 100644 --- a/geps/gep-3388/index.md +++ b/geps/gep-3388/index.md @@ -79,9 +79,154 @@ The implementation of a version of Linkerd's `ttl` parameter within Envoy might ## API +Two possible API designs are provided below, likely only one should be selected for implementation. + ### Go -TODO +```golang +type RetryPolicy struct { + // RetryPolicy defines the configuration for when to retry a request to a target backend. + // Implementations SHOULD retry on connection errors (disconnect, reset, timeout, + // TCP failure) if a retry stanza is configured. + // + // Support: Extended + // + // +optional + // + // + // Note: there is no Override or Default policy configuration. + + metav1.TypeMeta `json:",inline"` + metav1.ObjectMeta `json:"metadata,omitempty"` + + // Spec defines the desired state of BackendLBPolicy. + Spec RetryPolicySpec `json:"spec"` + + // Status defines the current state of BackendLBPolicy. + Status PolicyStatus `json:"status,omitempty"` +} + +type RetryPolicySpec struct { + // TargetRef identifies an API object to apply policy to. + // Currently, Backends (i.e. Service, ServiceImport, or any + // implementation-specific backendRef) are the only valid API + // target references. + // +listType=map + // +listMapKey=group + // +listMapKey=kind + // +listMapKey=name + // +kubebuilder:validation:MinItems=1 + // +kubebuilder:validation:MaxItems=16 + TargetRefs []LocalPolicyTargetReference `json:"targetRefs"` + + // TODO: This captures the basic idea, but should likely be a new type. + From []ReferenceGrantFrom `json:"from,omitempty"` + + CommonRetryPolicy `json:",inline"` +} + +type BackendTrafficPolicy struct { + // BackendTrafficPolicy defines the configuration for how traffic to a target backend should be handled. + // + // Support: Extended + // + // +optional + // + // + // Note: there is no Override or Default policy configuration. + + metav1.TypeMeta `json:",inline"` + metav1.ObjectMeta `json:"metadata,omitempty"` + + // Spec defines the desired state of BackendTrafficPolicy. + Spec BackendTrafficPolicySpec `json:"spec"` + + // Status defines the current state of BackendTrafficPolicy. + Status PolicyStatus `json:"status,omitempty"` +} + +type BackendTrafficPolicySpec struct { + // TargetRef identifies an API object to apply policy to. + // Currently, Backends (i.e. Service, ServiceImport, or any + // implementation-specific backendRef) are the only valid API + // target references. + // +listType=map + // +listMapKey=group + // +listMapKey=kind + // +listMapKey=name + // +kubebuilder:validation:MinItems=1 + // +kubebuilder:validation:MaxItems=16 + TargetRefs []LocalPolicyTargetReference `json:"targetRefs"` + + // TODO: This captures the basic idea, but should likely be a new type. + From []ReferenceGrantFrom `json:"from,omitempty"` + + // Retry defines the configuration for when to retry a request to a target backend. + // + // Implementations SHOULD retry on connection errors (disconnect, reset, timeout, + // TCP failure) if a retry stanza is configured. + // + // Support: Extended + // + // +optional + // + Retry *CommonRetryPolicy `json:"retry,omitempty"` + + // SessionPersistence defines and configures session persistence + // for the backend. + // + // Support: Extended + // + // +optional + SessionPersistence *SessionPersistence `json:"sessionPersistence,omitempty"` +} + +// CommonRetryPolicy defines the configuration for when to retry a request. +// +type CommonRetryPolicy struct { + // TODO: Does it make sense to include this configuration in the policy or not? + // + // Support: Extended + // + // +optional + HTTP *HTTPRouteRetry `json:"http,omitempty"` + + // Support: Extended + // + // +optional + BudgetPercent *Int `json:"budgetPercent,omitempty"` + + // Support: Extended + // + // +optional + BudgetInterval *Duration `json:"budgetInterval,omitempty"` + + // Support: Extended + // + // +optional + minRetryRate *RequestRate `json:"retryRate,omitempty"` +} + +// RequestRate expresses a rate of requests over a given period of time. +// +type RequestRate struct { + // Support: Extended + // + // +optional + Count *Int `json:"count,omitempty"` + + // Support: Extended + // + // +optional + Interval *Duration `json:"interval,omitempty"` +} + +// Duration is a string value representing a duration in time. The foramat is +// as specified in GEP-2257, a strict subset of the syntax parsed by Golang +// time.ParseDuration. +// +// +kubebuilder:validation:Pattern=`^([0-9]{1,5}(h|m|s|ms)){1,4}$` +type Duration string ### YAML @@ -101,13 +246,14 @@ spec: name: istio - kind: Gateway name: foo-ingress - codes: - - 500 - - 502 - - 503 - - 504 - attempts: 2 - backoff: 100ms + http: + codes: + - 500 + - 502 + - 503 + - 504 + attempts: 2 + backoff: 100ms budgetPercent: 20 budgetInterval: 10s minRetryRate: @@ -153,13 +299,14 @@ spec: - kind: Gateway name: foo-ingress retry: - codes: - - 500 - - 502 - - 503 - - 504 - attempts: 2 - backoff: 100ms + http: + codes: + - 500 + - 502 + - 503 + - 504 + attempts: 2 + backoff: 100ms budgetPercent: 20 budgetInterval: 10s minRetryRate: From 797c731ebec7a393026fd54d93e1eee5bc7b9a49 Mon Sep 17 00:00:00 2001 From: Eric Bishop <60610299+ericdbishop@users.noreply.github.com> Date: Thu, 30 Jan 2025 19:43:12 -0500 Subject: [PATCH 4/9] Apply suggestions from code review Co-authored-by: Mike Morris Co-authored-by: Lior Lieberman --- geps/gep-3388/index.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/geps/gep-3388/index.md b/geps/gep-3388/index.md index 05179a22e0..bc90e943c9 100644 --- a/geps/gep-3388/index.md +++ b/geps/gep-3388/index.md @@ -204,7 +204,7 @@ type CommonRetryPolicy struct { // Support: Extended // // +optional - minRetryRate *RequestRate `json:"retryRate,omitempty"` + MinRetryRate *RequestRate `json:"minRetryRate,omitempty"` } // RequestRate expresses a rate of requests over a given period of time. @@ -221,7 +221,7 @@ type RequestRate struct { Interval *Duration `json:"interval,omitempty"` } -// Duration is a string value representing a duration in time. The foramat is +// Duration is a string value representing a duration in time. The format is // as specified in GEP-2257, a strict subset of the syntax parsed by Golang // time.ParseDuration. // From 5a76d185dfd0bb538aaf208020e2774c81477808 Mon Sep 17 00:00:00 2001 From: Mike Morris <1149913+mikemorris@users.noreply.github.com> Date: Thu, 6 Feb 2025 22:38:28 -0500 Subject: [PATCH 5/9] remove HTTPRouteRetry from CommonRetryPolicy --- geps/gep-3388/index.md | 23 ----------------------- 1 file changed, 23 deletions(-) diff --git a/geps/gep-3388/index.md b/geps/gep-3388/index.md index bc90e943c9..634c164925 100644 --- a/geps/gep-3388/index.md +++ b/geps/gep-3388/index.md @@ -184,13 +184,6 @@ type BackendTrafficPolicySpec struct { // CommonRetryPolicy defines the configuration for when to retry a request. // type CommonRetryPolicy struct { - // TODO: Does it make sense to include this configuration in the policy or not? - // - // Support: Extended - // - // +optional - HTTP *HTTPRouteRetry `json:"http,omitempty"` - // Support: Extended // // +optional @@ -246,14 +239,6 @@ spec: name: istio - kind: Gateway name: foo-ingress - http: - codes: - - 500 - - 502 - - 503 - - 504 - attempts: 2 - backoff: 100ms budgetPercent: 20 budgetInterval: 10s minRetryRate: @@ -299,14 +284,6 @@ spec: - kind: Gateway name: foo-ingress retry: - http: - codes: - - 500 - - 502 - - 503 - - 504 - attempts: 2 - backoff: 100ms budgetPercent: 20 budgetInterval: 10s minRetryRate: From c32298cb00919ff0a388038b1ffe4e603d9e855f Mon Sep 17 00:00:00 2001 From: Mike Morris <1149913+mikemorris@users.noreply.github.com> Date: Thu, 6 Feb 2025 22:40:19 -0500 Subject: [PATCH 6/9] remove From scoping, deferred to future memorandum GEP as a potential common pattern for policy attachment --- geps/gep-3388/index.md | 18 ------------------ 1 file changed, 18 deletions(-) diff --git a/geps/gep-3388/index.md b/geps/gep-3388/index.md index 634c164925..5a1a587506 100644 --- a/geps/gep-3388/index.md +++ b/geps/gep-3388/index.md @@ -119,9 +119,6 @@ type RetryPolicySpec struct { // +kubebuilder:validation:MaxItems=16 TargetRefs []LocalPolicyTargetReference `json:"targetRefs"` - // TODO: This captures the basic idea, but should likely be a new type. - From []ReferenceGrantFrom `json:"from,omitempty"` - CommonRetryPolicy `json:",inline"` } @@ -158,9 +155,6 @@ type BackendTrafficPolicySpec struct { // +kubebuilder:validation:MaxItems=16 TargetRefs []LocalPolicyTargetReference `json:"targetRefs"` - // TODO: This captures the basic idea, but should likely be a new type. - From []ReferenceGrantFrom `json:"from,omitempty"` - // Retry defines the configuration for when to retry a request to a target backend. // // Implementations SHOULD retry on connection errors (disconnect, reset, timeout, @@ -233,12 +227,6 @@ spec: - group: "" kind: Service name: foo - from: - - kind: Mesh - namespace: istio-system - name: istio - - kind: Gateway - name: foo-ingress budgetPercent: 20 budgetInterval: 10s minRetryRate: @@ -277,12 +265,6 @@ spec: - group: "" kind: Service name: foo - from: - - kind: Mesh - namespace: istio-system - name: istio - - kind: Gateway - name: foo-ingress retry: budgetPercent: 20 budgetInterval: 10s From 04c4b7bfc8f4135e7668f2b95260e0b9893d880e Mon Sep 17 00:00:00 2001 From: Mike Morris <1149913+mikemorris@users.noreply.github.com> Date: Thu, 6 Feb 2025 22:44:58 -0500 Subject: [PATCH 7/9] remove standalone RetryPolicy examples in favor of BackendTrafficPolicy --- geps/gep-3388/index.md | 78 ------------------------------------------ 1 file changed, 78 deletions(-) diff --git a/geps/gep-3388/index.md b/geps/gep-3388/index.md index 5a1a587506..fa9e73104d 100644 --- a/geps/gep-3388/index.md +++ b/geps/gep-3388/index.md @@ -79,49 +79,9 @@ The implementation of a version of Linkerd's `ttl` parameter within Envoy might ## API -Two possible API designs are provided below, likely only one should be selected for implementation. - ### Go ```golang -type RetryPolicy struct { - // RetryPolicy defines the configuration for when to retry a request to a target backend. - // Implementations SHOULD retry on connection errors (disconnect, reset, timeout, - // TCP failure) if a retry stanza is configured. - // - // Support: Extended - // - // +optional - // - // - // Note: there is no Override or Default policy configuration. - - metav1.TypeMeta `json:",inline"` - metav1.ObjectMeta `json:"metadata,omitempty"` - - // Spec defines the desired state of BackendLBPolicy. - Spec RetryPolicySpec `json:"spec"` - - // Status defines the current state of BackendLBPolicy. - Status PolicyStatus `json:"status,omitempty"` -} - -type RetryPolicySpec struct { - // TargetRef identifies an API object to apply policy to. - // Currently, Backends (i.e. Service, ServiceImport, or any - // implementation-specific backendRef) are the only valid API - // target references. - // +listType=map - // +listMapKey=group - // +listMapKey=kind - // +listMapKey=name - // +kubebuilder:validation:MinItems=1 - // +kubebuilder:validation:MaxItems=16 - TargetRefs []LocalPolicyTargetReference `json:"targetRefs"` - - CommonRetryPolicy `json:",inline"` -} - type BackendTrafficPolicy struct { // BackendTrafficPolicy defines the configuration for how traffic to a target backend should be handled. // @@ -217,44 +177,6 @@ type Duration string ### YAML -```yaml -apiVersion: gateway.networking.x-k8s.io/v1alpha1 -kind: RetryPolicy -metadata: - name: retry-policy-example -spec: - targetRefs: - - group: "" - kind: Service - name: foo - budgetPercent: 20 - budgetInterval: 10s - minRetryRate: - count: 3 - interval: 1s -status: - ancestors: - - ancestorRef: - kind: Mesh - namespace: istio-system - name: istio - controllerName: "istio.io/mesh-controller" - conditions: - - type: "Accepted" - status: "True" - reason: "Accepted" - - ancestorRef: - kind: Gateway - namespace: foo-ns - name: foo-ingress - controllerName: "istio.io/mesh-controller" - conditions: - - type: "Accepted" - status: "False" - reason: "Invalid" - message: "RetryPolicy fields budgetPercentage, budgetInterval and minRetryRate are not supported for Istio ingress gateways." -``` - ```yaml apiVersion: gateway.networking.x-k8s.io/v1alpha1 kind: BackendTrafficPolicy From 7d9ef6e9e2d50a26d1d82447b4aa8551f7475cbd Mon Sep 17 00:00:00 2001 From: Eric Bishop <60610299+ericdbishop@users.noreply.github.com> Date: Fri, 7 Feb 2025 09:19:21 -0500 Subject: [PATCH 8/9] Update geps/gep-3388/index.md Co-authored-by: Flynn --- geps/gep-3388/index.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/geps/gep-3388/index.md b/geps/gep-3388/index.md index fa9e73104d..66a25c7880 100644 --- a/geps/gep-3388/index.md +++ b/geps/gep-3388/index.md @@ -29,7 +29,7 @@ Multiple data plane proxies offer optional configuration for budgeted retries, i Configuring a limit for client retries is an important factor in building a resilient system, allowing requests to be successfully retried during periods of intermittent failure. But too many client-side retries can also exacerbate consistent failures and slow down recovery, quickly overwhelming a failing system and leading to cascading failures such as retry storms. Configuring a sane limit for max client-side retries is often challenging in complex systems. Allowing an application developer (Ana) to configure a dynamic "retry budget" reduces the risk of a high number of retries across clients. It allows a service to perform as expected in both times of high & low request load, as well as both during periods of intermittent & consistent failures. -While retry budget configuration has been a frequently discussed feature within the community, differences in the semantics between data plane implementations creates a challenge for a consensus on the correct location for the configuration. This proposal aims to determine where retry budget's should be defined within the Gateway API, and whether data plane proxies may need to be altered to accommodate the specification. +While retry budget configuration has been a frequently discussed feature within the community, differences in the semantics between data plane implementations creates a challenge for a consensus on the correct location for the configuration. This proposal aims to determine where retry budgets should be defined within the Gateway API, and whether data plane proxies may need to be altered to accommodate the specification. ### Background on implementations From 1e2c618bd39ad2a83abda2315d831c72815811ad Mon Sep 17 00:00:00 2001 From: Eric Bishop <60610299+ericdbishop@users.noreply.github.com> Date: Fri, 7 Feb 2025 09:48:02 -0500 Subject: [PATCH 9/9] Update geps/gep-3388/index.md Co-authored-by: Flynn --- geps/gep-3388/index.md | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/geps/gep-3388/index.md b/geps/gep-3388/index.md index 66a25c7880..8f63436739 100644 --- a/geps/gep-3388/index.md +++ b/geps/gep-3388/index.md @@ -49,7 +49,9 @@ By default, Envoy uses a static threshold for retries. But when configured, Envo The Linkerd implementation of retry budgets is configured alongside service route configuration, within the [ServiceProfile CRD](https://linkerd.io/2.12/reference/service-profiles/), limiting the number of total retries for a service as a percentage of the number of recent requests. In practice, this functions similarly to Envoy's retry budget implementation, as it is configured in a single location and measures the ratio of retry requests to original requests across all traffic destined for the service. -Usage of the ServiceProfile CRD is significantly rarer today, given Linkerd's support for per-route configuration through Gateway API Route resources. Though historically, [retry budgets](https://linkerd.io/2.15/features/retries-and-timeouts/) were the default configuration to specify retries to a service, with support for counted retries later being add in [edge-24.7.5](https://github.com/linkerd/linkerd2/releases/tag/edge-24.7.5). In all cases, retries are implemented by the `linkerd2-proxy` making the request on behalf on an application workload. +(Note that budgeted retries have become less commonly used since Linkerd added support for counted retries in [edge-24.7.5](https://github.com/linkerd/linkerd2/releases/tag/edge-24.7.5): ServiceProfile operates at the level of a backend workload, meaning that it cannot configure anything at the level of a route, but counted retries can be configured using annotations on Service, HTTPRoute, and GRPCRoute.) + +For both counted retries and budgeted retries, the actual retry logic is implemented by the `linkerd2-proxy` making the request on behalf on an application workload. The receiving proxy is not aware of the retry configuration at all. Linkerd's budgeted retries allow retrying an indefinite number of times, as long as the fraction of retries remains within the budget. Budgeted retries are supported only using Linkerd's native ServiceProfile CRD, which allows enabling retries, setting the retry budget (by default, 20% plus 10 "extra" retries per second), and configuring the window over which the fraction of retries to non-retries is calculated. The `retryBudget` field of the ServiceProfile spec can be configured with the following optional parameters: