Skip to content

Conversation

@meiravgri
Copy link
Collaborator

@meiravgri meiravgri commented Oct 12, 2025

MOD-11784 Align first batch size updateTriggerThreshold for non-compressed svs tiered index

In tiered SVS indexes, vector transfer to the backend SVS index is triggered as follows:

  • First batch determined by trainingTriggerThreshold, defaults to ~10k vectors (10 * block_size = 1024)
  • Subsequent batches determined by updateTriggerThreshold, defaults to 1024 vectors (block_size)

For non-compressed SVS indexes, the large initial batch provides no performance or accuracy benefit, and creates an inconsistent user experience compared to subsequent batches.

MOD-11785 Expose SVS_DEFAULT_UPDATE_THRESHOLD via C API

  • Renamed SVS_DEFAULT_UPDATE_THRESHOLDSVS_VAMANA_DEFAULT_UPDATE_THRESHOLD
  • Exposed SVS_VAMANA_DEFAULT_UPDATE_THRESHOLD to vec_sim_common.h to allow usage in RedisSearch.

…_THRESHOLD`

* Exposed `SVS_VAMANA_DEFAULT_UPDATE_THRESHOLD` to `vec_sim_common.h` to allow usage in RedisSearch.
@meiravgri meiravgri changed the title [MOD-11784] use updateTriggerThreshold for training TH for non- compression index in svs tired index [MOD-11784] [MOD-11785] use updateTriggerThreshold for training TH for non- compression index in svs tired index Oct 12, 2025
@meiravgri meiravgri changed the title [MOD-11784] [MOD-11785] use updateTriggerThreshold for training TH for non- compression index in svs tired index [MOD-11784] [MOD-11785] use updateTriggerThreshold as training TH for non- compressed svs tired index Oct 12, 2025
@codecov
Copy link

codecov bot commented Oct 12, 2025

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 96.68%. Comparing base (9fb223a) to head (b54304f).
⚠️ Report is 2 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main     #795      +/-   ##
==========================================
+ Coverage   96.66%   96.68%   +0.01%     
==========================================
  Files         126      126              
  Lines        7707     7714       +7     
==========================================
+ Hits         7450     7458       +8     
+ Misses        257      256       -1     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@meiravgri meiravgri enabled auto-merge October 12, 2025 07:28
Copy link
Collaborator

@alonre24 alonre24 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice, one cosmetic suggestion

@meiravgri meiravgri requested a review from alonre24 October 13, 2025 10:49
@meiravgri meiravgri added this pull request to the merge queue Oct 13, 2025
Merged via the queue into main with commit 4f58e35 Oct 13, 2025
17 checks passed
@meiravgri meiravgri deleted the meiravg_non_compressed_svs_first_batch branch October 13, 2025 14:18
github-actions bot pushed a commit that referenced this pull request Oct 13, 2025
… non- compressed svs tired index (#795)

* use updateTriggerThreshold for training TH for non- compression index in svs tired index

* * Renamed `SVS_DEFAULT_UPDATE_THRESHOLD` → `SVS_VAMANA_DEFAULT_UPDATE_THRESHOLD`
* Exposed `SVS_VAMANA_DEFAULT_UPDATE_THRESHOLD` to `vec_sim_common.h` to allow usage in RedisSearch.

* fix test

* apply review  suggetion

(cherry picked from commit 4f58e35)
@github-actions
Copy link

github-merge-queue bot pushed a commit that referenced this pull request Oct 15, 2025
…TH for non- compressed svs tired index (#798)

* [MOD-11784] [MOD-11785] use updateTriggerThreshold as training TH for non- compressed svs tired index (#795)

* use updateTriggerThreshold for training TH for non- compression index in svs tired index

* * Renamed `SVS_DEFAULT_UPDATE_THRESHOLD` → `SVS_VAMANA_DEFAULT_UPDATE_THRESHOLD`
* Exposed `SVS_VAMANA_DEFAULT_UPDATE_THRESHOLD` to `vec_sim_common.h` to allow usage in RedisSearch.

* fix test

* apply review  suggetion

(cherry picked from commit 4f58e35)

* remov is multi

---------

Co-authored-by: meiravgri <[email protected]>
Co-authored-by: meiravgri <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants