Skip to content

Conversation

@prudhvigodithi
Copy link
Member

@prudhvigodithi prudhvigodithi commented Aug 29, 2025

Description

The existing behavior for Indexing uses direct multiplication (doubleValue * scalingFactor) in parseCreateField method but Querying uses BigDecimal multiplication in scale(). I was able to reproduce the issue as shown in #12433 and with this fix I was able to see the correct results. This PR uses same direct multiplication for both Indexing and Querying.

Initially I have attempted to use BigDecimal for both Indexing and Querying c632a23 which also fixed the issue, but just updating the scale() seemed to be simple as it does not require to reindex the data.

Related Issues

Related to #12433

Check List

  • Functionality includes testing.
  • API changes companion pull request created, if applicable.
  • Public documentation issue/PR created, if applicable.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

Signed-off-by: Prudhvi Godithi <[email protected]>
Signed-off-by: Prudhvi Godithi <[email protected]>
Signed-off-by: Prudhvi Godithi <[email protected]>
@prudhvigodithi
Copy link
Member Author

Adding @kkewwei @msfroh for some inputs.
Thanks

@prudhvigodithi prudhvigodithi self-assigned this Aug 29, 2025
@github-actions
Copy link
Contributor

❌ Gradle check result for e07d545: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

@prudhvigodithi
Copy link
Member Author

curl -X PUT "http://localhost:9200/test-bigdecimal-index-000001" \
  -H "Content-Type: application/json" \
  -d '{
    "mappings": {
      "dynamic": "strict",
      "properties": {
        "field0_BigDecimal": {
          "type": "scaled_float",
          "doc_values": false,
          "scaling_factor": 100.0
        }
      }
    }
  }'
curl -X POST "http://localhost:9200/test-bigdecimal-index-000001/_doc/1" \
  -H "Content-Type: application/json" \
  -d '{
    "field0_BigDecimal": 92233720368547750
  }'

curl -X POST "http://localhost:9200/test-bigdecimal-index-000001/_search" \
  -H "Content-Type: application/json" \
  -d '{
    "query": {
      "match": {
        "field0_BigDecimal": {
          "query": "92233720368547750"
        }
      }
    }
  }'
{"took":18,"timed_out":false,"_shards":{"total":1,"successful":1,"skipped":0,"failed":0},"hits":{"total":{"value":1,"relation":"eq"},"max_score":1.0,"hits":[{"_index":"test-bigdecimal-index-000001","_id":"1","_score":1.0,"_source":{
    "field0_BigDecimal": 92233720368547750
  }}]}}

With this change I can see the indexing and query value is same as 9223372036854774784.

@github-actions
Copy link
Contributor

github-actions bot commented Sep 2, 2025

❌ Gradle check result for cf3602e: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

Signed-off-by: Prudhvi Godithi <[email protected]>
@github-actions
Copy link
Contributor

github-actions bot commented Sep 3, 2025

✅ Gradle check result for f402fb8: SUCCESS

@codecov
Copy link

codecov bot commented Sep 3, 2025

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 72.88%. Comparing base (3e747cb) to head (d1b5170).
⚠️ Report is 3 commits behind head on main.

Additional details and impacted files
@@             Coverage Diff              @@
##               main   #19188      +/-   ##
============================================
+ Coverage     72.79%   72.88%   +0.09%     
- Complexity    69605    69677      +72     
============================================
  Files          5658     5658              
  Lines        320079   320085       +6     
  Branches      46345    46345              
============================================
+ Hits         232996   233290     +294     
+ Misses        68230    67899     -331     
- Partials      18853    18896      +43     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@github-actions
Copy link
Contributor

github-actions bot commented Sep 3, 2025

❌ Gradle check result for 6655829: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

Signed-off-by: Prudhvi Godithi <[email protected]>
@github-actions
Copy link
Contributor

github-actions bot commented Sep 3, 2025

❌ Gradle check result for 959c5a5: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

@github-actions
Copy link
Contributor

github-actions bot commented Sep 3, 2025

✅ Gradle check result for 959c5a5: SUCCESS

@kkewwei
Copy link
Contributor

kkewwei commented Sep 4, 2025

The PR can solve the term query with the same precision loss. However, precision loss becomes problematic in range query, see #12433 (comment)

Alternatively, we can declare this bug in "range": range query currently suffer from inaccurate matching, and it will be
solved by the following big_decimal type.

@prudhvigodithi
Copy link
Member Author

Alternatively, we can declare this bug in "range": range query currently suffer from inaccurate matching, and it will be
solved by the following big_decimal type.

Yes I was able to reproduce this #12433 (comment), so @kkewwei your suggestion is to continue with existing BigDecimal (the existing behavior) and do not use the direct multiplication ?

@prudhvigodithi
Copy link
Member Author

I think I got it now, you are planning to introduce a new type big_decimal which can solve the precision even for the range query as well.

@prudhvigodithi prudhvigodithi marked this pull request as ready for review September 4, 2025 15:20
@prudhvigodithi prudhvigodithi requested a review from a team as a code owner September 4, 2025 15:20
@kkewwei
Copy link
Contributor

kkewwei commented Sep 4, 2025

I think I got it now, you are planning to introduce a new type big_decimal which can solve the precision even for the range query as well.

Yes.

Signed-off-by: Prudhvi Godithi <[email protected]>
@github-actions
Copy link
Contributor

github-actions bot commented Sep 5, 2025

❌ Gradle check result for d1b5170: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

@github-actions
Copy link
Contributor

github-actions bot commented Sep 5, 2025

❌ Gradle check result for d1b5170: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

@github-actions
Copy link
Contributor

github-actions bot commented Sep 6, 2025

❌ Gradle check result for d1b5170: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

@github-actions
Copy link
Contributor

github-actions bot commented Sep 6, 2025

✅ Gradle check result for d1b5170: SUCCESS

Copy link
Contributor

@kkewwei kkewwei left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@github-project-automation github-project-automation bot moved this from Todo to In Progress in Performance Roadmap Sep 6, 2025
@kkewwei kkewwei merged commit 4aff917 into opensearch-project:main Sep 6, 2025
34 of 40 checks passed
@github-project-automation github-project-automation bot moved this from In Progress to Done in Performance Roadmap Sep 6, 2025
@prudhvigodithi prudhvigodithi added v3.3.0 Search Search query, autocomplete ...etc labels Sep 6, 2025
kh3ra pushed a commit to kh3ra/OpenSearch that referenced this pull request Sep 9, 2025
…9188)

* Initial commit

Signed-off-by: Prudhvi Godithi <[email protected]>

* initial commit

Signed-off-by: Prudhvi Godithi <[email protected]>

* Fix the precision loss

Signed-off-by: Prudhvi Godithi <[email protected]>

* Update range query

Signed-off-by: Prudhvi Godithi <[email protected]>

* Add tests

Signed-off-by: Prudhvi Godithi <[email protected]>

* UPstream fetch

Signed-off-by: Prudhvi Godithi <[email protected]>

* Fix tests

Signed-off-by: Prudhvi Godithi <[email protected]>

* Address comments

Signed-off-by: Prudhvi Godithi <[email protected]>

* Upstream fetch

Signed-off-by: Prudhvi Godithi <[email protected]>

---------

Signed-off-by: Prudhvi Godithi <[email protected]>
jainankitk pushed a commit to jainankitk/OpenSearch that referenced this pull request Sep 22, 2025
…9188)

* Initial commit

Signed-off-by: Prudhvi Godithi <[email protected]>

* initial commit

Signed-off-by: Prudhvi Godithi <[email protected]>

* Fix the precision loss

Signed-off-by: Prudhvi Godithi <[email protected]>

* Update range query

Signed-off-by: Prudhvi Godithi <[email protected]>

* Add tests

Signed-off-by: Prudhvi Godithi <[email protected]>

* UPstream fetch

Signed-off-by: Prudhvi Godithi <[email protected]>

* Fix tests

Signed-off-by: Prudhvi Godithi <[email protected]>

* Address comments

Signed-off-by: Prudhvi Godithi <[email protected]>

* Upstream fetch

Signed-off-by: Prudhvi Godithi <[email protected]>

---------

Signed-off-by: Prudhvi Godithi <[email protected]>
jainankitk pushed a commit to jainankitk/OpenSearch that referenced this pull request Sep 22, 2025
…9188)

* Initial commit

Signed-off-by: Prudhvi Godithi <[email protected]>

* initial commit

Signed-off-by: Prudhvi Godithi <[email protected]>

* Fix the precision loss

Signed-off-by: Prudhvi Godithi <[email protected]>

* Update range query

Signed-off-by: Prudhvi Godithi <[email protected]>

* Add tests

Signed-off-by: Prudhvi Godithi <[email protected]>

* UPstream fetch

Signed-off-by: Prudhvi Godithi <[email protected]>

* Fix tests

Signed-off-by: Prudhvi Godithi <[email protected]>

* Address comments

Signed-off-by: Prudhvi Godithi <[email protected]>

* Upstream fetch

Signed-off-by: Prudhvi Godithi <[email protected]>

---------

Signed-off-by: Prudhvi Godithi <[email protected]>
Signed-off-by: Ankit Jain <[email protected]>
jainankitk pushed a commit to jainankitk/OpenSearch that referenced this pull request Sep 22, 2025
…9188)

* Initial commit

Signed-off-by: Prudhvi Godithi <[email protected]>

* initial commit

Signed-off-by: Prudhvi Godithi <[email protected]>

* Fix the precision loss

Signed-off-by: Prudhvi Godithi <[email protected]>

* Update range query

Signed-off-by: Prudhvi Godithi <[email protected]>

* Add tests

Signed-off-by: Prudhvi Godithi <[email protected]>

* UPstream fetch

Signed-off-by: Prudhvi Godithi <[email protected]>

* Fix tests

Signed-off-by: Prudhvi Godithi <[email protected]>

* Address comments

Signed-off-by: Prudhvi Godithi <[email protected]>

* Upstream fetch

Signed-off-by: Prudhvi Godithi <[email protected]>

---------

Signed-off-by: Prudhvi Godithi <[email protected]>
Signed-off-by: Ankit Jain <[email protected]>
asimmahmood1 pushed a commit to jainankitk/OpenSearch that referenced this pull request Sep 23, 2025
…9188)

* Initial commit

Signed-off-by: Prudhvi Godithi <[email protected]>

* initial commit

Signed-off-by: Prudhvi Godithi <[email protected]>

* Fix the precision loss

Signed-off-by: Prudhvi Godithi <[email protected]>

* Update range query

Signed-off-by: Prudhvi Godithi <[email protected]>

* Add tests

Signed-off-by: Prudhvi Godithi <[email protected]>

* UPstream fetch

Signed-off-by: Prudhvi Godithi <[email protected]>

* Fix tests

Signed-off-by: Prudhvi Godithi <[email protected]>

* Address comments

Signed-off-by: Prudhvi Godithi <[email protected]>

* Upstream fetch

Signed-off-by: Prudhvi Godithi <[email protected]>

---------

Signed-off-by: Prudhvi Godithi <[email protected]>
pranikum pushed a commit to pranikum/OpenSearch that referenced this pull request Sep 23, 2025
…9188)

* Initial commit

Signed-off-by: Prudhvi Godithi <[email protected]>

* initial commit

Signed-off-by: Prudhvi Godithi <[email protected]>

* Fix the precision loss

Signed-off-by: Prudhvi Godithi <[email protected]>

* Update range query

Signed-off-by: Prudhvi Godithi <[email protected]>

* Add tests

Signed-off-by: Prudhvi Godithi <[email protected]>

* UPstream fetch

Signed-off-by: Prudhvi Godithi <[email protected]>

* Fix tests

Signed-off-by: Prudhvi Godithi <[email protected]>

* Address comments

Signed-off-by: Prudhvi Godithi <[email protected]>

* Upstream fetch

Signed-off-by: Prudhvi Godithi <[email protected]>

---------

Signed-off-by: Prudhvi Godithi <[email protected]>
vinaykpud pushed a commit to vinaykpud/OpenSearch that referenced this pull request Sep 26, 2025
…9188)

* Initial commit

Signed-off-by: Prudhvi Godithi <[email protected]>

* initial commit

Signed-off-by: Prudhvi Godithi <[email protected]>

* Fix the precision loss

Signed-off-by: Prudhvi Godithi <[email protected]>

* Update range query

Signed-off-by: Prudhvi Godithi <[email protected]>

* Add tests

Signed-off-by: Prudhvi Godithi <[email protected]>

* UPstream fetch

Signed-off-by: Prudhvi Godithi <[email protected]>

* Fix tests

Signed-off-by: Prudhvi Godithi <[email protected]>

* Address comments

Signed-off-by: Prudhvi Godithi <[email protected]>

* Upstream fetch

Signed-off-by: Prudhvi Godithi <[email protected]>

---------

Signed-off-by: Prudhvi Godithi <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Search Search query, autocomplete ...etc v3.3.0

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

2 participants