-
Couldn't load subscription status.
- Fork 22
[MOD-11784] [MOD-11785] use updateTriggerThreshold as training TH for non- compressed svs tired index #795
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
… in svs tired index
…_THRESHOLD` * Exposed `SVS_VAMANA_DEFAULT_UPDATE_THRESHOLD` to `vec_sim_common.h` to allow usage in RedisSearch.
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #795 +/- ##
==========================================
+ Coverage 96.66% 96.68% +0.01%
==========================================
Files 126 126
Lines 7707 7714 +7
==========================================
+ Hits 7450 7458 +8
+ Misses 257 256 -1 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Nice, one cosmetic suggestion
… non- compressed svs tired index (#795) * use updateTriggerThreshold for training TH for non- compression index in svs tired index * * Renamed `SVS_DEFAULT_UPDATE_THRESHOLD` → `SVS_VAMANA_DEFAULT_UPDATE_THRESHOLD` * Exposed `SVS_VAMANA_DEFAULT_UPDATE_THRESHOLD` to `vec_sim_common.h` to allow usage in RedisSearch. * fix test * apply review suggetion (cherry picked from commit 4f58e35)
|
Successfully created backport PR for |
…TH for non- compressed svs tired index (#798) * [MOD-11784] [MOD-11785] use updateTriggerThreshold as training TH for non- compressed svs tired index (#795) * use updateTriggerThreshold for training TH for non- compression index in svs tired index * * Renamed `SVS_DEFAULT_UPDATE_THRESHOLD` → `SVS_VAMANA_DEFAULT_UPDATE_THRESHOLD` * Exposed `SVS_VAMANA_DEFAULT_UPDATE_THRESHOLD` to `vec_sim_common.h` to allow usage in RedisSearch. * fix test * apply review suggetion (cherry picked from commit 4f58e35) * remov is multi --------- Co-authored-by: meiravgri <[email protected]> Co-authored-by: meiravgri <[email protected]>
MOD-11784 Align first batch size updateTriggerThreshold for non-compressed svs tiered index
In tiered SVS indexes, vector transfer to the backend SVS index is triggered as follows:
trainingTriggerThreshold, defaults to ~10k vectors (10 * block_size = 1024)updateTriggerThreshold, defaults to 1024 vectors (block_size)For non-compressed SVS indexes, the large initial batch provides no performance or accuracy benefit, and creates an inconsistent user experience compared to subsequent batches.
MOD-11785 Expose
SVS_DEFAULT_UPDATE_THRESHOLDvia C APISVS_DEFAULT_UPDATE_THRESHOLD→SVS_VAMANA_DEFAULT_UPDATE_THRESHOLDSVS_VAMANA_DEFAULT_UPDATE_THRESHOLDtovec_sim_common.hto allow usage in RedisSearch.