Add `per_minute`, `per_hour`, `per_day` function support #4531

dai-chen · 2025-10-13T21:38:30Z

Description

This PR extends timechart command's per_* function support (introduced in #4464) by adding three additional per_* functions that normalize metrics to different time units:

per_minute(field): Calculates the per-minute rate (sum-per-second multiplies by 60)
per_hour(field): Calculates the per-hour rate (sum-per-second multiplies by 3600)
per_day(field): Calculates the per-day rate (sum-per-second multiplies by 86400)

TODO: After #4550 is resolved, migrate from the current second-based approach to millisecond-based calculations, e.g., timestampdiff(MILLISECOND, span_start, span_end) to support millisecond span.

Related Issues

Resolves #4350

Check List

New functionality includes testing.
New functionality has been documented.
New functionality has javadoc added.
New functionality has a user manual doc added.
New PPL command checklist all confirmed.
API changes companion pull request created.
Commits are signed per the DCO using --signoff or -s.
Public documentation issue/PR created.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

Signed-off-by: Chen Dai <[email protected]>

RyanL1997

Hi @dai-chen , thanks for the change, and it is LGTM. Just left a very minor question on the doc format.

docs/user/ppl/cmd/timechart.rst

Signed-off-by: Chen Dai <[email protected]>

ykmr1224

LGTM other than minor comment.

core/src/test/java/org/opensearch/sql/ast/tree/TimechartTest.java

Signed-off-by: Chen Dai <[email protected]>

dai-chen · 2025-10-24T15:51:43Z

@RyanL1997 @ykmr1224 Addressed comments and merged from main.

Waiting for CI complete. Please re-approve. Thanks!

* Support per_minute/hour/day function Signed-off-by: Chen Dai <[email protected]> * Remove unused UT and IT Signed-off-by: Chen Dai <[email protected]> * Add more test for edge case Signed-off-by: Chen Dai <[email protected]> * Address PR comments in doc Signed-off-by: Chen Dai <[email protected]> * Address PR comments in UT Signed-off-by: Chen Dai <[email protected]> --------- Signed-off-by: Chen Dai <[email protected]> (cherry picked from commit 435e4b2) Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>

* Support per_minute/hour/day function * Remove unused UT and IT * Add more test for edge case * Address PR comments in doc * Address PR comments in UT --------- (cherry picked from commit 435e4b2) Signed-off-by: Chen Dai <[email protected]> Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>

* default-main: (34 commits) Enhance dynamic source clause to support only metadata filters (opensearch-project#4554) Make nested alias type support referring to outer context (opensearch-project#4673) Update big5 ppl queries and check plans (opensearch-project#4668) Support push down sort after limit (opensearch-project#4657) Use table scan rowType in filter pushdown could fix rename issue (opensearch-project#4670) Fix: Support Alias Fields in MIN, MAX, FIRST, LAST, and TAKE Aggregations (opensearch-project#4621) Fix bin nested fields issue (opensearch-project#4606) Add `per_minute`, `per_hour`, `per_day` function support (opensearch-project#4531) Pushdown sort aggregate metrics (opensearch-project#4603) Followup: Change ComparableLinkedHashMap to compare Key than Value (opensearch-project#4648) Mitigate the CI failure caused by 500 Internal Server Error (opensearch-project#4646) Allow renaming group-by fields to existing field names (opensearch-project#4586) Publish internal modules separately for downstream reuse (opensearch-project#4484) Revert "Update grammar files and developer guide (opensearch-project#4301)" (opensearch-project#4643) Support Automatic Type Conversion for REX/SPATH/PARSE Command Extractions (opensearch-project#4599) Replace all dots in fields of table scan's PhysType (opensearch-project#4633) Return comparable LinkedHashMap in `valueForCalcite()` of ExprTupleValue (opensearch-project#4629) Refactor JsonExtractAllFunctionIT and MapConcatFunctionIT (opensearch-project#4623) Pushdown case function in aggregations as range queries (opensearch-project#4400) Update GEOIP function to support IP types as input (opensearch-project#4613) ... # Conflicts: # docs/user/ppl/functions/conversion.rst

* default-main: (34 commits) Enhance dynamic source clause to support only metadata filters (opensearch-project#4554) Make nested alias type support referring to outer context (opensearch-project#4673) Update big5 ppl queries and check plans (opensearch-project#4668) Support push down sort after limit (opensearch-project#4657) Use table scan rowType in filter pushdown could fix rename issue (opensearch-project#4670) Fix: Support Alias Fields in MIN, MAX, FIRST, LAST, and TAKE Aggregations (opensearch-project#4621) Fix bin nested fields issue (opensearch-project#4606) Add `per_minute`, `per_hour`, `per_day` function support (opensearch-project#4531) Pushdown sort aggregate metrics (opensearch-project#4603) Followup: Change ComparableLinkedHashMap to compare Key than Value (opensearch-project#4648) Mitigate the CI failure caused by 500 Internal Server Error (opensearch-project#4646) Allow renaming group-by fields to existing field names (opensearch-project#4586) Publish internal modules separately for downstream reuse (opensearch-project#4484) Revert "Update grammar files and developer guide (opensearch-project#4301)" (opensearch-project#4643) Support Automatic Type Conversion for REX/SPATH/PARSE Command Extractions (opensearch-project#4599) Replace all dots in fields of table scan's PhysType (opensearch-project#4633) Return comparable LinkedHashMap in `valueForCalcite()` of ExprTupleValue (opensearch-project#4629) Refactor JsonExtractAllFunctionIT and MapConcatFunctionIT (opensearch-project#4623) Pushdown case function in aggregations as range queries (opensearch-project#4400) Update GEOIP function to support IP types as input (opensearch-project#4613) ... Signed-off-by: Asif Bashar <[email protected]>

…project#4531) * Support per_minute/hour/day function Signed-off-by: Chen Dai <[email protected]> * Remove unused UT and IT Signed-off-by: Chen Dai <[email protected]> * Add more test for edge case Signed-off-by: Chen Dai <[email protected]> * Address PR comments in doc Signed-off-by: Chen Dai <[email protected]> * Address PR comments in UT Signed-off-by: Chen Dai <[email protected]> --------- Signed-off-by: Chen Dai <[email protected]>

Support per_minute/hour/day function

b3da82b

Signed-off-by: Chen Dai <[email protected]>

dai-chen self-assigned this Oct 13, 2025

dai-chen added enhancement New feature or request PPL Piped processing language backport 2.19-dev labels Oct 13, 2025

Remove unused UT and IT

bed7437

Signed-off-by: Chen Dai <[email protected]>

dai-chen marked this pull request as ready for review October 14, 2025 18:01

Add more test for edge case

b4c9362

Signed-off-by: Chen Dai <[email protected]>

RyanL1997 previously approved these changes Oct 16, 2025

View reviewed changes

docs/user/ppl/cmd/timechart.rst Show resolved Hide resolved

Address PR comments in doc

28bb1a9

Signed-off-by: Chen Dai <[email protected]>

dai-chen dismissed RyanL1997’s stale review via 28bb1a9 October 22, 2025 15:58

Merge branch 'main' into add-per-function-support-minute-hour-day

5916f81

RyanL1997 previously approved these changes Oct 23, 2025

View reviewed changes

ykmr1224 previously approved these changes Oct 23, 2025

View reviewed changes

core/src/test/java/org/opensearch/sql/ast/tree/TimechartTest.java Outdated Show resolved Hide resolved

dai-chen added 2 commits October 24, 2025 08:48

Address PR comments in UT

b7acc98

Signed-off-by: Chen Dai <[email protected]>

Merge branch 'main' into add-per-function-support-minute-hour-day

d12f90d

dai-chen dismissed stale reviews from ykmr1224 and RyanL1997 via d12f90d October 24, 2025 15:48

RyanL1997 approved these changes Oct 24, 2025

View reviewed changes

ykmr1224 approved these changes Oct 24, 2025

View reviewed changes

dai-chen merged commit 435e4b2 into opensearch-project:main Oct 24, 2025
33 checks passed

opensearch-trigger-bot bot mentioned this pull request Oct 24, 2025

[Backport 2.19-dev] Add per_minute, per_hour, per_day function support #4662

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Add `per_minute`, `per_hour`, `per_day` function support #4531

Add `per_minute`, `per_hour`, `per_day` function support #4531

Uh oh!

dai-chen commented Oct 13, 2025 •

edited

Loading

Uh oh!

RyanL1997 left a comment

Uh oh!

Uh oh!

ykmr1224 left a comment

Uh oh!

Uh oh!

dai-chen commented Oct 24, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Add per_minute, per_hour, per_day function support #4531

Add per_minute, per_hour, per_day function support #4531

Uh oh!

Conversation

dai-chen commented Oct 13, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Related Issues

Check List

Uh oh!

RyanL1997 left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

ykmr1224 left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

dai-chen commented Oct 24, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Add `per_minute`, `per_hour`, `per_day` function support #4531

Add `per_minute`, `per_hour`, `per_day` function support #4531

dai-chen commented Oct 13, 2025 •

edited

Loading