Adds weight parameter type definition for RRF retrievers.

[mridula-s109]

Changes

• RRFRetriever.retrievers: RetrieverContainer[] → RRFRetrieverContainer[]
• New RRFRetrieverContainer extends RetrieverContainer with optional weight?: float
• Defaults to 1.0, must be non-negative

Related: elastic/elasticsearch#130658, elastic/elasticsearch#136477

[Copilot]

Pull Request Overview

This PR adds support for weighted retrievers in the RRF (Reciprocal Rank Fusion) retriever by introducing a new weight parameter. The change allows individual retrievers to have different influence levels when combining their results.

• Introduced RRFRetrieverContainer class extending RetrieverContainer with optional weight parameter
• Updated RRFRetriever.retrievers type from RetrieverContainer[] to RRFRetrieverContainer[]
• Added comprehensive documentation for the weight parameter behavior and constraints

---

_{Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.}

[github-actions]

Following you can find the validation changes against the target branch for the APIs.

API	Status	Request	Response
index	🟢	1445/1445 → 1443/1443	1447/1447 → 1445/1445
indices.create	🔴	1378/1402 → 1385/1409	1402/1402 → 1409/1409
indices.refresh	🟢	329/329 → 327/327	329/329 → 327/327
ml.get_job_stats	🟢	30/30 → 29/29	30/30 → 29/29
ml.put_job	🟢	65/65 → 64/64	65/65 → 64/64

You can validate these APIs yourself by using the make validate target.

[leemthompo]

You'll need to backport this to 9.2 right?

[mridula-s109]

You'll need to backport this to 9.2 right?

Thanks for the reminder! I've added the backport-9.2 label but the automation isn't triggering. Does the backport job only run after the PR is merged, or is there a specific way to trigger it manually?

[leemthompo]

The backport job only runs after the PR is merged (if there are conflicts might need to use the manual backport tool to clean those up)

[pmpailis]

I think that we also have to include the RetrieverContainer definition here as we have:

retrievers: [
   { 
         "retriever": {},
         "weight": 1
]

Maybe we could also combine with the nested structure of LinearRetriever which uses an InnerRetriever component.

[flobernd]

Thanks for the PR @mridula-s109 !

From looking at the JSON example provided in elastic/elasticsearch#130658, @pmpailis seems to be right that the structure is not correct this way.

"retrievers": [ // <- RRFRetriever.retrievers
  {
    "retriever": { // <- RRFRetrieverComponent.retriever
      "standard": { // <- RetrieverContainer variant
        "query": { "match": { "description": "pizza" } } 
      }
    },
    "weight": 0.7 // <- RRFRetrieverComponent.weigth
  },

I think it should rather look like this:

export class RRFRetrieverComponent { // name taken from the server code
  retriever: RetrieverContainer
  weigth?: float
}

and

// In RRFRetriever:
retrievers: RRFRetrieverComponent[]

Btw.: I'm a little bit surprised that we ship this breaking change in a minor release. Is this expected?

Side note: For container variants, it's important to mark all fields that are not actual variants with the @container_property tag (see here).

[pmpailis]

Btw.: I'm a little bit surprised that we ship this breaking change in a minor release. Is this expected?

I think that this is an extension, i.e. an alternative format that we support, and not a breaking change. The previous format has not gone away (which also implies some adjustments in this PR)

[flobernd]

@pmpailis

I think that this is an extension, i.e. an alternative format that we support, and not a breaking change. The previous format has not gone away

Ah right, I just saw the corresponding test cases 👍

which also implies some adjustments in this PR

Unfortunately. This kind of extension will result in an untagged/undiscriminated union in the specification - which is the worst possible construct to have for all statically typed clients 😐

[Anaethelion]

I have removed the 9.2 backport label.
This sadly comes too late for us to include in the release of clients, especially with an untagged union which causes a breaking change in the API.
Please for now merge to main, thank you!