Skip to content

Conversation

kumarUjjawal
Copy link
Contributor

@kumarUjjawal kumarUjjawal commented Sep 30, 2025

Pull Request

Related issue

Fixes #698

What does this PR do?

PR checklist

Please check if your PR fulfills the following requirements:

  • Does this PR fix an existing issue, or have you listed the changes applied in the PR description (and why they are needed)?
  • Have you read the contributing guidelines?
  • Have you made sure that the title is accurate and descriptive of the changes?

Thank you so much for contributing to Meilisearch!

Summary by CodeRabbit

  • New Features

    • Search queries can include media payloads for multimodal retrieval.
    • Added an experimental toggle to enable multimodal capabilities.
    • Embedders support fragment configurations for indexing and search to enable multimodal embeddings.
  • Documentation

    • New code sample demonstrating a search request with media alongside hybrid retrieval.
    • Updated example text in the embedders template for clarity.

Copy link
Contributor

coderabbitai bot commented Sep 30, 2025

Walkthrough

Adds multimodal support: a new experimental multimodal flag, search queries accepting a media payload, and embedder fragment fields for indexing/search. Includes constructors, builders, serialization, tests, and a documentation sample demonstrating media in a search request.

Changes

Cohort / File(s) Summary of changes
Experimental features: multimodal flag
src/features.rs
Added multimodal: bool to ExperimentalFeaturesResult (serde default); multimodal: Option<bool> to ExperimentalFeatures; initialized in new; added set_multimodal(bool) builder; tests updated.
Search query: media support
src/search.rs
Added media: Option<Value> to SearchQuery, initialized in new, and with_media(Value) builder to attach media payload; tests added for serialization.
Embedder fragments (indexing/search)
src/settings.rs
Added pub struct EmbedderFragment { pub value: serde_json::Value }; added indexing_fragments and search_fragments (optional HashMap<String, EmbedderFragment>) to Embedder; serialization/tests updated.
Docs / Samples update
.code-samples.meilisearch.yaml
Added search_parameter_reference_media_1 sample demonstrating with_media(json!(...)); minor text change in update_embedders_1 template.

Sequence Diagram(s)

sequenceDiagram
  autonumber
  participant App as Client App
  participant SDK as SDK
  participant MS as Meilisearch Server

  rect rgba(230,240,255,0.6)
    note over SDK: Update embedder settings with fragments
    App->>SDK: call update_embedder(settings with indexingFragments/searchFragments)
    SDK->>MS: PUT /indexes/:uid/settings/embedders (JSON with fragments)
    MS-->>SDK: 200 / Task info
    SDK-->>App: Task response
  end

  rect rgba(230,255,230,0.6)
    note over SDK: Toggle experimental multimodal feature
    App->>SDK: ExperimentalFeatures.set_multimodal(true)
    SDK->>MS: PATCH /experimental-features { multimodal: true }
    MS-->>SDK: { multimodal: true, ... }
    SDK-->>App: ExperimentalFeaturesResult
  end

  rect rgba(255,245,230,0.6)
    note over SDK: Search including media payload
    App->>SDK: SearchQuery.with_media(Value) + other params
    SDK->>MS: POST /indexes/:uid/search { media: {...}, ... }
    MS-->>SDK: Search results
    SDK-->>App: Results
  end
Loading

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~25 minutes

Possibly related PRs

Suggested labels

enhancement

Suggested reviewers

  • curquiza
  • irevoire

Poem

I thump the keys with whiskered pride,
Media in pockets, fragments allied.
Multimodal hops through query and test,
JSON carrots tucked in their nest.
Ship the patch — the rabbit’s blessed. 🥕

Pre-merge checks and finishing touches

❌ Failed checks (1 warning)
Check name Status Explanation Resolution
Out of Scope Changes Check ⚠️ Warning A small textual change was made to the document template in the update_embedders_1 sample, which is not part of the multi-modal search objectives defined in issue #698 and appears unrelated to the requested feature implementation. Remove or relocate the unrelated documentation change to a separate pull request so this PR remains focused on multi-modal search support.
✅ Passed checks (4 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title Check ✅ Passed The title “Add support for multi-modal search” concisely and accurately captures the primary change made by this pull request, which introduces media payload handling, experimental multimodal flags, and embedder fragment settings to support multi-modal search.
Linked Issues Check ✅ Passed The implementation fulfills all tasks from issue #698 by extending Embedder settings with indexingFragments and searchFragments, adding corresponding tests, and including a code sample under the search_parameter_reference_media_1 key in the .code-samples.meilisearch.yaml file.
Docstring Coverage ✅ Passed Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%.
✨ Finishing touches
  • 📝 Generate docstrings
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Post copyable unit tests in a comment

📜 Recent review details

Configuration used: CodeRabbit UI

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between f72885d and 1891ff3.

📒 Files selected for processing (1)
  • src/search.rs (4 hunks)
🚧 Files skipped from review as they are similar to previous changes (1)
  • src/search.rs
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (1)
  • GitHub Check: integration-tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 0

🧹 Nitpick comments (6)
src/search.rs (3)

411-414: Media field shape looks good; keep as optional.

Adds media: Option<Value> with proper serde skipping. Consider documenting expected JSON shape (object of fragments) since this is experimental.


703-707: Builder ergonomics: consider a fallible generic helper.

with_media(Value) is fine. Optionally add try_with_media<T: Serialize>(&self, media: &T) -> Result<&mut Self, serde_json::Error> or with_media_json<T: Serialize>(&mut self, media: T) to avoid requiring callers to pre-build Value.

+impl<'a, Http: HttpClient> SearchQuery<'a, Http> {
+    /// Attach media fragments from any serializable structure.
+    pub fn with_media_json<'b, T: Serialize>(
+        &'b mut self,
+        media: T,
+    ) -> &'b mut SearchQuery<'a, Http> {
+        self.media = serde_json::to_value(media).ok();
+        self
+    }
+}

1110-1136: Serialization test is solid; add a negative case.

Great direct check that media serializes as expected. Consider an extra test asserting media is omitted when unset.

.code-samples.meilisearch.yaml (1)

1944-1957: New sample aligns with API additions.

Example shows .with_media(json!(...)) alongside hybrid search; this mirrors the new builder. Optionally preface with a note in docs that the multimodal experimental feature must be enabled.

src/settings.rs (2)

150-157: Embedder fragments exposed—API surface looks correct.

indexing_fragments and search_fragments as Option<HashMap<String, EmbedderFragment>> with camelCase serialization aligns with expected wire format. Consider adding brief docs stating these are experimental.


3157-3241: Good serialization coverage for fragments.

Test validates both indexingFragments and searchFragments, plus request/response. Consider also asserting a round‑trip from_value::<Embedder>(serialized) to catch deserialization regressions.

📜 Review details

Configuration used: CodeRabbit UI

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 910f9e9 and f72885d.

📒 Files selected for processing (4)
  • .code-samples.meilisearch.yaml (1 hunks)
  • src/features.rs (6 hunks)
  • src/search.rs (4 hunks)
  • src/settings.rs (3 hunks)
🧰 Additional context used
🧠 Learnings (1)
📚 Learning: 2025-06-12T13:28:23.700Z
Learnt from: LukasKalbertodt
PR: meilisearch/meilisearch-rust#625
File: src/search.rs:368-370
Timestamp: 2025-06-12T13:28:23.700Z
Learning: In the Meilisearch Rust client, `SearchQuery` serializes its per-query federation settings under the key `federationOptions`; only the top-level multi-search parameter is named `federation`.

Applied to files:

  • src/search.rs
🧬 Code graph analysis (1)
src/search.rs (2)
src/indexes.rs (26)
  • client (186-188)
  • client (232-234)
  • client (320-322)
  • client (376-378)
  • client (428-430)
  • client (473-475)
  • client (522-525)
  • client (556-558)
  • client (634-636)
  • client (700-702)
  • client (970-972)
  • client (1038-1040)
  • client (1089-1091)
  • client (1133-1135)
  • client (1186-1188)
  • client (1245-1247)
  • new (81-89)
  • new (1783-1790)
  • new (1994-2000)
  • index (2171-2171)
  • index (2188-2189)
  • index (2226-2226)
  • index (2298-2298)
  • index (2326-2326)
  • index (2352-2352)
  • index (2379-2379)
src/features.rs (1)
  • new (56-66)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (1)
  • GitHub Check: integration-tests
🔇 Additional comments (7)
src/settings.rs (2)

2815-2815: Import addition OK.

use serde_json::json; required for tests.


159-163: Struct matches Meilisearch 1.16 spec

The EmbedderFragment { value: serde_json::Value } definition correctly wraps each fragment’s payload under a "value" key as required by the server schema. No changes needed.

src/features.rs (5)

17-19: Backward-compatible result field.

multimodal: bool with #[serde(default)] avoids breaking on older servers.


50-52: Request knob for multimodal.

Optional multimodal field mirrors the response; consistent with existing flags.


64-65: Constructor default None—consistent.

Keeps updates explicit.


149-152: Setter reads cleanly.

set_multimodal matches existing builder style.


168-178: Test covers end-to-end toggle.

Verifies update + get for multimodal. Looks good.

Copy link

codecov bot commented Sep 30, 2025

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 86.21%. Comparing base (5e84bdf) to head (1891ff3).

Additional details and impacted files
@@            Coverage Diff             @@
##             main     #712      +/-   ##
==========================================
+ Coverage   85.90%   86.21%   +0.30%     
==========================================
  Files          19       19              
  Lines        6079     6186     +107     
==========================================
+ Hits         5222     5333     +111     
+ Misses        857      853       -4     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[v1.16.0] Add support for multi-modal search

1 participant