Skip to content

Increase timeout in Gradle Launcher smoke tests #8142

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 1 commit into from
Jan 2, 2025
Merged

Increase timeout in Gradle Launcher smoke tests #8142

merged 1 commit into from
Jan 2, 2025

Conversation

@nikita-tkachenko-datadog
Copy link
Contributor

@nikita-tkachenko-datadog nikita-tkachenko-datadog commented Jan 2, 2025 •
edited by atlassian bot
Loading

What Does This Do

Increase Gradle build timeout in Gradle Launcher smoke tests from 30s to 60s.

Motivation

Reduce tests flakiness.
When running the test on IBM Semeru 17 Gradle builds may take longer than 30 seconds.

Contributor Checklist

Jira ticket: SDTEST-1409

@nikita-tkachenko-datadog nikita-tkachenko-datadog added comp: testing Testing tag: no release notes Changes to exclude from release notes run-tests: all Run all tests labels Jan 2, 2025
@pr-commenter
Copy link

pr-commenter bot commented Jan 2, 2025

Benchmarks

Startup

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master nikita-tkachenko/gradle-launcher-smoke-timeout
git_commit_date 1735815904 1735822331
git_commit_sha c24db36 5b9454c
release_version 1.45.0-SNAPSHOT~c24db361be 1.45.0-SNAPSHOT~5b9454c79f
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1735824871 1735824871
ci_job_id 752383398 752383398
ci_pipeline_id 52000883 52000883
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
module Agent Agent
parent None None
variant iast iast

Summary

Found 6 performance improvements and 12 performance regressions! Performance is the same for 38 metrics, 7 unstable metrics.

scenario Δ mean execution_time candidate mean execution_time baseline mean execution_time
scenario:startup:insecure-bank:iast:GlobalTracer worse
[+56.232ms; +65.468ms] or [+22.748%; +26.484%]		 308.050ms 247.200ms
scenario:startup:insecure-bank:iast:Remote Config better
[-72.796µs; -21.403µs] or [-10.938%; -3.216%]		 618.440µs 665.539µs
scenario:startup:insecure-bank:iast:Telemetry better
[-1.381ms; -1.091ms] or [-15.885%; -12.553%]		 7.459ms 8.695ms
scenario:startup:insecure-bank:iast_HARDCODED_SECRET_DISABLED:Agent.start worse
[+33.911ms; +81.142ms] or [+2.871%; +6.870%]		 1.239s 1.181s
scenario:startup:insecure-bank:iast_HARDCODED_SECRET_DISABLED:GlobalTracer worse
[+57.383ms; +67.901ms] or [+23.206%; +27.459%]		 309.925ms 247.283ms
scenario:startup:insecure-bank:iast_HARDCODED_SECRET_DISABLED:Telemetry better
[-1.315ms; -0.760ms] or [-15.164%; -8.764%]		 7.634ms 8.672ms
scenario:startup:insecure-bank:iast_TELEMETRY_OFF:GlobalTracer worse
[+55.524ms; +61.588ms] or [+22.449%; +24.901%]		 305.894ms 247.338ms
scenario:startup:insecure-bank:iast_TELEMETRY_OFF:Telemetry better
[-1.360ms; -1.022ms] or [-16.056%; -12.067%]		 7.279ms 8.470ms
scenario:startup:insecure-bank:tracing:GlobalTracer worse
[+53.352ms; +64.907ms] or [+20.549%; +24.999%]		 318.767ms 259.637ms
scenario:startup:petclinic:appsec:Agent.start worse
[+31.656ms; +57.896ms] or [+2.664%; +4.873%]		 1.233s 1.188s
scenario:startup:petclinic:appsec:GlobalTracer worse
[+59.412ms; +66.104ms] or [+23.456%; +26.098%]		 316.044ms 253.286ms
scenario:startup:petclinic:iast:Agent.start worse
[+32.799ms; +68.037ms] or [+2.782%; +5.772%]		 1.229s 1.179s
scenario:startup:petclinic:iast:GlobalTracer worse
[+58.285ms; +64.904ms] or [+23.659%; +26.346%]		 307.950ms 246.355ms
scenario:startup:petclinic:iast:Telemetry better
[-1.275ms; -0.913ms] or [-14.927%; -10.688%]		 7.448ms 8.542ms
scenario:startup:petclinic:profiling:Agent.start worse
[+38.375ms; +62.260ms] or [+3.013%; +4.888%]		 1.324s 1.274s
scenario:startup:petclinic:profiling:GlobalTracer worse
[+60.062ms; +65.212ms] or [+16.137%; +17.521%]		 434.838ms 372.201ms
scenario:startup:petclinic:tracing:GlobalTracer worse
[+55.474ms; +62.510ms] or [+21.490%; +24.216%]		 317.132ms 258.140ms
scenario:startup:petclinic:tracing:Remote Config better
[-60.689µs; -28.228µs] or [-8.477%; -3.943%]		 671.487µs 715.945µs
Startup time reports for petclinic 
gantt
    title petclinic - global startup overhead: candidate=1.45.0-SNAPSHOT~5b9454c79f, baseline=1.45.0-SNAPSHOT~c24db361be

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.058 s) : 0, 1058264
Total [baseline] (10.427 s) : 0, 10426947
Agent [candidate] (1.092 s) : 0, 1092248
Total [candidate] (10.419 s) : 0, 10418923
section appsec
Agent [baseline] (1.188 s) : 0, 1188077
Total [baseline] (10.68 s) : 0, 10679912
Agent [candidate] (1.233 s) : 0, 1232853
Total [candidate] (10.72 s) : 0, 10719717
section iast
Agent [baseline] (1.179 s) : 0, 1178801
Total [baseline] (10.935 s) : 0, 10935044
Agent [candidate] (1.229 s) : 0, 1229219
Total [candidate] (11.027 s) : 0, 11026791
section profiling
Agent [baseline] (1.274 s) : 0, 1273844
Total [baseline] (10.838 s) : 0, 10838242
Agent [candidate] (1.324 s) : 0, 1324162
Total [candidate] (10.863 s) : 0, 10863280
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.058 s -
Agent appsec 1.188 s 129.813 ms (12.3%)
Agent iast 1.179 s 120.537 ms (11.4%)
Agent profiling 1.274 s 215.58 ms (20.4%)
Total tracing 10.427 s -
Total appsec 10.68 s 252.965 ms (2.4%)
Total iast 10.935 s 508.098 ms (4.9%)
Total profiling 10.838 s 411.295 ms (3.9%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.092 s -
Agent appsec 1.233 s 140.605 ms (12.9%)
Agent iast 1.229 s 136.971 ms (12.5%)
Agent profiling 1.324 s 231.913 ms (21.2%)
Total tracing 10.419 s -
Total appsec 10.72 s 300.794 ms (2.9%)
Total iast 11.027 s 607.868 ms (5.8%)
Total profiling 10.863 s 444.357 ms (4.3%) 
gantt
    title petclinic - break down per module: candidate=1.45.0-SNAPSHOT~5b9454c79f, baseline=1.45.0-SNAPSHOT~c24db361be

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (716.852 ms) : 0, 716852
BytebuddyAgent [candidate] (695.223 ms) : 0, 695223
GlobalTracer [baseline] (258.14 ms) : 0, 258140
GlobalTracer [candidate] (317.132 ms) : 0, 317132
AppSec [baseline] (55.368 ms) : 0, 55368
AppSec [candidate] (54.787 ms) : 0, 54787
Remote Config [baseline] (715.945 µs) : 0, 716
Remote Config [candidate] (671.487 µs) : 0, 671
Telemetry [baseline] (12.121 ms) : 0, 12121
Telemetry [candidate] (10.718 ms) : 0, 10718
section appsec
BytebuddyAgent [baseline] (730.892 ms) : 0, 730892
BytebuddyAgent [candidate] (717.401 ms) : 0, 717401
GlobalTracer [baseline] (253.286 ms) : 0, 253286
GlobalTracer [candidate] (316.044 ms) : 0, 316044
AppSec [baseline] (170.457 ms) : 0, 170457
AppSec [candidate] (167.21 ms) : 0, 167210
Remote Config [baseline] (657.042 µs) : 0, 657
Remote Config [candidate] (639.46 µs) : 0, 639
Telemetry [baseline] (8.095 ms) : 0, 8095
Telemetry [candidate] (7.822 ms) : 0, 7822
IAST [baseline] (19.358 ms) : 0, 19358
IAST [candidate] (19.817 ms) : 0, 19817
section iast
BytebuddyAgent [baseline] (828.959 ms) : 0, 828959
BytebuddyAgent [candidate] (820.076 ms) : 0, 820076
GlobalTracer [baseline] (246.355 ms) : 0, 246355
GlobalTracer [candidate] (307.95 ms) : 0, 307950
AppSec [baseline] (58.011 ms) : 0, 58011
AppSec [candidate] (58.346 ms) : 0, 58346
Remote Config [baseline] (647.708 µs) : 0, 648
Remote Config [candidate] (614.333 µs) : 0, 614
Telemetry [baseline] (8.542 ms) : 0, 8542
Telemetry [candidate] (7.448 ms) : 0, 7448
IAST [baseline] (21.316 ms) : 0, 21316
IAST [candidate] (20.934 ms) : 0, 20934
section profiling
BytebuddyAgent [baseline] (701.805 ms) : 0, 701805
BytebuddyAgent [candidate] (694.197 ms) : 0, 694197
GlobalTracer [baseline] (372.201 ms) : 0, 372201
GlobalTracer [candidate] (434.838 ms) : 0, 434838
AppSec [baseline] (53.661 ms) : 0, 53661
AppSec [candidate] (53.639 ms) : 0, 53639
Remote Config [baseline] (690.225 µs) : 0, 690
Remote Config [candidate] (675.646 µs) : 0, 676
Telemetry [baseline] (7.797 ms) : 0, 7797
Telemetry [candidate] (7.663 ms) : 0, 7663
ProfilingAgent [baseline] (95.901 ms) : 0, 95901
ProfilingAgent [candidate] (93.76 ms) : 0, 93760
Profiling [baseline] (95.925 ms) : 0, 95925
Profiling [candidate] (93.784 ms) : 0, 93784
Loading
Startup time reports for insecure-bank 
gantt
    title insecure-bank - global startup overhead: candidate=1.45.0-SNAPSHOT~5b9454c79f, baseline=1.45.0-SNAPSHOT~c24db361be

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.07 s) : 0, 1069674
Total [baseline] (8.654 s) : 0, 8653505
Agent [candidate] (1.098 s) : 0, 1097820
Total [candidate] (8.734 s) : 0, 8734434
section iast
Agent [baseline] (1.185 s) : 0, 1185242
Total [baseline] (9.202 s) : 0, 9202163
Agent [candidate] (1.228 s) : 0, 1228392
Total [candidate] (9.203 s) : 0, 9202992
section iast_HARDCODED_SECRET_DISABLED
Agent [baseline] (1.181 s) : 0, 1181184
Total [baseline] (9.238 s) : 0, 9237895
Agent [candidate] (1.239 s) : 0, 1238710
Total [candidate] (9.221 s) : 0, 9220946
section iast_TELEMETRY_OFF
Agent [baseline] (1.181 s) : 0, 1180657
Total [baseline] (9.187 s) : 0, 9187254
Agent [candidate] (1.217 s) : 0, 1216799
Total [candidate] (9.165 s) : 0, 9164609
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.07 s -
Agent iast 1.185 s 115.567 ms (10.8%)
Agent iast_HARDCODED_SECRET_DISABLED 1.181 s 111.509 ms (10.4%)
Agent iast_TELEMETRY_OFF 1.181 s 110.982 ms (10.4%)
Total tracing 8.654 s -
Total iast 9.202 s 548.659 ms (6.3%)
Total iast_HARDCODED_SECRET_DISABLED 9.238 s 584.39 ms (6.8%)
Total iast_TELEMETRY_OFF 9.187 s 533.749 ms (6.2%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.098 s -
Agent iast 1.228 s 130.572 ms (11.9%)
Agent iast_HARDCODED_SECRET_DISABLED 1.239 s 140.891 ms (12.8%)
Agent iast_TELEMETRY_OFF 1.217 s 118.979 ms (10.8%)
Total tracing 8.734 s -
Total iast 9.203 s 468.558 ms (5.4%)
Total iast_HARDCODED_SECRET_DISABLED 9.221 s 486.511 ms (5.6%)
Total iast_TELEMETRY_OFF 9.165 s 430.175 ms (4.9%) 
gantt
    title insecure-bank - break down per module: candidate=1.45.0-SNAPSHOT~5b9454c79f, baseline=1.45.0-SNAPSHOT~c24db361be

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (725.948 ms) : 0, 725948
BytebuddyAgent [candidate] (700.375 ms) : 0, 700375
GlobalTracer [baseline] (259.637 ms) : 0, 259637
GlobalTracer [candidate] (318.767 ms) : 0, 318767
AppSec [baseline] (55.283 ms) : 0, 55283
AppSec [candidate] (54.897 ms) : 0, 54897
Remote Config [baseline] (714.822 µs) : 0, 715
Remote Config [candidate] (695.393 µs) : 0, 695
Telemetry [baseline] (12.81 ms) : 0, 12810
Telemetry [candidate] (9.253 ms) : 0, 9253
section iast
BytebuddyAgent [baseline] (833.826 ms) : 0, 833826
BytebuddyAgent [candidate] (819.571 ms) : 0, 819571
GlobalTracer [baseline] (247.2 ms) : 0, 247200
GlobalTracer [candidate] (308.05 ms) : 0, 308050
AppSec [baseline] (58.26 ms) : 0, 58260
AppSec [candidate] (57.102 ms) : 0, 57102
IAST [baseline] (21.548 ms) : 0, 21548
IAST [candidate] (21.792 ms) : 0, 21792
Remote Config [baseline] (665.539 µs) : 0, 666
Remote Config [candidate] (618.44 µs) : 0, 618
Telemetry [baseline] (8.695 ms) : 0, 8695
Telemetry [candidate] (7.459 ms) : 0, 7459
section iast_HARDCODED_SECRET_DISABLED
BytebuddyAgent [baseline] (829.509 ms) : 0, 829509
BytebuddyAgent [candidate] (826.308 ms) : 0, 826308
GlobalTracer [baseline] (247.283 ms) : 0, 247283
GlobalTracer [candidate] (309.925 ms) : 0, 309925
AppSec [baseline] (58.458 ms) : 0, 58458
AppSec [candidate] (57.823 ms) : 0, 57823
IAST [baseline] (21.58 ms) : 0, 21580
IAST [candidate] (22.542 ms) : 0, 22542
Remote Config [baseline] (662.878 µs) : 0, 663
Remote Config [candidate] (626.223 µs) : 0, 626
Telemetry [baseline] (8.672 ms) : 0, 8672
Telemetry [candidate] (7.634 ms) : 0, 7634
section iast_TELEMETRY_OFF
BytebuddyAgent [baseline] (830.68 ms) : 0, 830680
BytebuddyAgent [candidate] (810.693 ms) : 0, 810693
GlobalTracer [baseline] (247.338 ms) : 0, 247338
GlobalTracer [candidate] (305.894 ms) : 0, 305894
AppSec [baseline] (57.703 ms) : 0, 57703
AppSec [candidate] (56.511 ms) : 0, 56511
IAST [baseline] (20.86 ms) : 0, 20860
IAST [candidate] (22.102 ms) : 0, 22102
Remote Config [baseline] (632.062 µs) : 0, 632
Remote Config [candidate] (608.371 µs) : 0, 608
Telemetry [baseline] (8.47 ms) : 0, 8470
Telemetry [candidate] (7.279 ms) : 0, 7279
Loading

Load

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
end_time 2025-01-02T13:04:49 2025-01-02T13:11:52
git_branch master nikita-tkachenko/gradle-launcher-smoke-timeout
git_commit_date 1735815904 1735822331
git_commit_sha c24db36 5b9454c
release_version 1.45.0-SNAPSHOT~c24db361be 1.45.0-SNAPSHOT~5b9454c79f
start_time 2025-01-02T13:04:35 2025-01-02T13:11:38
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1735823868 1735823868
ci_job_id 752383399 752383399
ci_pipeline_id 52000883 52000883
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
variant iast iast

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 12 metrics, 16 unstable metrics.

Request duration reports for petclinic 
gantt
    title petclinic - request duration [CI 0.99] : candidate=1.45.0-SNAPSHOT~5b9454c79f, baseline=1.45.0-SNAPSHOT~c24db361be
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.362 ms) : 1343, 1381
.   : milestone, 1362,
appsec (1.768 ms) : 1744, 1792
.   : milestone, 1768,
appsec_no_iast (1.759 ms) : 1734, 1783
.   : milestone, 1759,
iast (1.508 ms) : 1485, 1530
.   : milestone, 1508,
profiling (1.562 ms) : 1537, 1588
.   : milestone, 1562,
tracing (1.521 ms) : 1496, 1546
.   : milestone, 1521,
section candidate
no_agent (1.376 ms) : 1355, 1396
.   : milestone, 1376,
appsec (1.758 ms) : 1734, 1782
.   : milestone, 1758,
appsec_no_iast (1.765 ms) : 1739, 1791
.   : milestone, 1765,
iast (1.509 ms) : 1487, 1532
.   : milestone, 1509,
profiling (1.535 ms) : 1510, 1559
.   : milestone, 1535,
tracing (1.505 ms) : 1481, 1529
.   : milestone, 1505,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.362 ms [1.343 ms, 1.381 ms] -
appsec 1.768 ms [1.744 ms, 1.792 ms] 405.932 µs (29.8%)
appsec_no_iast 1.759 ms [1.734 ms, 1.783 ms] 396.638 µs (29.1%)
iast 1.508 ms [1.485 ms, 1.53 ms] 145.854 µs (10.7%)
profiling 1.562 ms [1.537 ms, 1.588 ms] 200.543 µs (14.7%)
tracing 1.521 ms [1.496 ms, 1.546 ms] 159.017 µs (11.7%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.376 ms [1.355 ms, 1.396 ms] -
appsec 1.758 ms [1.734 ms, 1.782 ms] 382.291 µs (27.8%)
appsec_no_iast 1.765 ms [1.739 ms, 1.791 ms] 389.245 µs (28.3%)
iast 1.509 ms [1.487 ms, 1.532 ms] 133.501 µs (9.7%)
profiling 1.535 ms [1.51 ms, 1.559 ms] 158.96 µs (11.6%)
tracing 1.505 ms [1.481 ms, 1.529 ms] 129.118 µs (9.4%)
Request duration reports for insecure-bank 
gantt
    title insecure-bank - request duration [CI 0.99] : candidate=1.45.0-SNAPSHOT~5b9454c79f, baseline=1.45.0-SNAPSHOT~c24db361be
    dateFormat X
    axisFormat %s
section baseline
no_agent (379.258 µs) : 359, 399
.   : milestone, 379,
iast (493.814 µs) : 472, 515
.   : milestone, 494,
iast_FULL (663.304 µs) : 642, 685
.   : milestone, 663,
iast_GLOBAL (543.621 µs) : 520, 567
.   : milestone, 544,
iast_HARDCODED_SECRET_DISABLED (499.19 µs) : 478, 521
.   : milestone, 499,
iast_INACTIVE (462.82 µs) : 441, 484
.   : milestone, 463,
iast_TELEMETRY_OFF (487.629 µs) : 466, 509
.   : milestone, 488,
tracing (452.004 µs) : 431, 473
.   : milestone, 452,
section candidate
no_agent (379.978 µs) : 359, 401
.   : milestone, 380,
iast (501.866 µs) : 480, 524
.   : milestone, 502,
iast_FULL (661.589 µs) : 640, 683
.   : milestone, 662,
iast_GLOBAL (534.708 µs) : 511, 558
.   : milestone, 535,
iast_HARDCODED_SECRET_DISABLED (495.72 µs) : 474, 517
.   : milestone, 496,
iast_INACTIVE (458.308 µs) : 438, 479
.   : milestone, 458,
iast_TELEMETRY_OFF (483.183 µs) : 462, 504
.   : milestone, 483,
tracing (462.827 µs) : 441, 485
.   : milestone, 463,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 379.258 µs [359.459 µs, 399.057 µs] -
iast 493.814 µs [472.343 µs, 515.284 µs] 114.556 µs (30.2%)
iast_FULL 663.304 µs [641.576 µs, 685.033 µs] 284.046 µs (74.9%)
iast_GLOBAL 543.621 µs [520.198 µs, 567.045 µs] 164.364 µs (43.3%)
iast_HARDCODED_SECRET_DISABLED 499.19 µs [477.551 µs, 520.829 µs] 119.932 µs (31.6%)
iast_INACTIVE 462.82 µs [441.189 µs, 484.45 µs] 83.562 µs (22.0%)
iast_TELEMETRY_OFF 487.629 µs [465.948 µs, 509.31 µs] 108.372 µs (28.6%)
tracing 452.004 µs [431.247 µs, 472.761 µs] 72.747 µs (19.2%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 379.978 µs [359.0 µs, 400.957 µs] -
iast 501.866 µs [480.03 µs, 523.702 µs] 121.888 µs (32.1%)
iast_FULL 661.589 µs [640.043 µs, 683.134 µs] 281.61 µs (74.1%)
iast_GLOBAL 534.708 µs [511.086 µs, 558.331 µs] 154.73 µs (40.7%)
iast_HARDCODED_SECRET_DISABLED 495.72 µs [474.381 µs, 517.059 µs] 115.742 µs (30.5%)
iast_INACTIVE 458.308 µs [437.517 µs, 479.098 µs] 78.33 µs (20.6%)
iast_TELEMETRY_OFF 483.183 µs [461.966 µs, 504.4 µs] 103.205 µs (27.2%)
tracing 462.827 µs [440.922 µs, 484.732 µs] 82.849 µs (21.8%)

Dacapo

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master nikita-tkachenko/gradle-launcher-smoke-timeout
git_commit_date 1735815904 1735822331
git_commit_sha c24db36 5b9454c
release_version 1.45.0-SNAPSHOT~c24db361be 1.45.0-SNAPSHOT~5b9454c79f
See matching parameters
Baseline Candidate
application biojava biojava
ci_job_date 1735824425 1735824425
ci_job_id 752383400 752383400
ci_pipeline_id 52000883 52000883
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
variant appsec appsec

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 12 metrics, 0 unstable metrics.

Execution time for biojava 
gantt
    title biojava - execution time [CI 0.99] : candidate=1.45.0-SNAPSHOT~5b9454c79f, baseline=1.45.0-SNAPSHOT~c24db361be
    dateFormat X
    axisFormat %s
section baseline
no_agent (14.954 s) : 14954000, 14954000
.   : milestone, 14954000,
appsec (15.244 s) : 15244000, 15244000
.   : milestone, 15244000,
iast (18.8 s) : 18800000, 18800000
.   : milestone, 18800000,
iast_GLOBAL (17.936 s) : 17936000, 17936000
.   : milestone, 17936000,
profiling (14.906 s) : 14906000, 14906000
.   : milestone, 14906000,
tracing (15.171 s) : 15171000, 15171000
.   : milestone, 15171000,
section candidate
no_agent (14.884 s) : 14884000, 14884000
.   : milestone, 14884000,
appsec (14.809 s) : 14809000, 14809000
.   : milestone, 14809000,
iast (18.757 s) : 18757000, 18757000
.   : milestone, 18757000,
iast_GLOBAL (17.638 s) : 17638000, 17638000
.   : milestone, 17638000,
profiling (14.928 s) : 14928000, 14928000
.   : milestone, 14928000,
tracing (15.156 s) : 15156000, 15156000
.   : milestone, 15156000,
Loading
  • baseline results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 14.954 s [14.954 s, 14.954 s] -
appsec 15.244 s [15.244 s, 15.244 s] 290.0 ms (1.9%)
iast 18.8 s [18.8 s, 18.8 s] 3.846 s (25.7%)
iast_GLOBAL 17.936 s [17.936 s, 17.936 s] 2.982 s (19.9%)
profiling 14.906 s [14.906 s, 14.906 s] -48.0 ms (-0.3%)
tracing 15.171 s [15.171 s, 15.171 s] 217.0 ms (1.5%)
  • candidate results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 14.884 s [14.884 s, 14.884 s] -
appsec 14.809 s [14.809 s, 14.809 s] -75.0 ms (-0.5%)
iast 18.757 s [18.757 s, 18.757 s] 3.873 s (26.0%)
iast_GLOBAL 17.638 s [17.638 s, 17.638 s] 2.754 s (18.5%)
profiling 14.928 s [14.928 s, 14.928 s] 44.0 ms (0.3%)
tracing 15.156 s [15.156 s, 15.156 s] 272.0 ms (1.8%)
Execution time for tomcat 
gantt
    title tomcat - execution time [CI 0.99] : candidate=1.45.0-SNAPSHOT~5b9454c79f, baseline=1.45.0-SNAPSHOT~c24db361be
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.474 ms) : 1462, 1485
.   : milestone, 1474,
appsec (2.351 ms) : 2310, 2393
.   : milestone, 2351,
iast (2.097 ms) : 2044, 2150
.   : milestone, 2097,
iast_GLOBAL (2.141 ms) : 2088, 2194
.   : milestone, 2141,
profiling (1.959 ms) : 1916, 2001
.   : milestone, 1959,
tracing (1.942 ms) : 1902, 1983
.   : milestone, 1942,
section candidate
no_agent (1.472 ms) : 1461, 1484
.   : milestone, 1472,
appsec (2.359 ms) : 2317, 2401
.   : milestone, 2359,
iast (2.096 ms) : 2043, 2149
.   : milestone, 2096,
iast_GLOBAL (2.142 ms) : 2089, 2194
.   : milestone, 2142,
profiling (1.949 ms) : 1907, 1991
.   : milestone, 1949,
tracing (1.93 ms) : 1890, 1970
.   : milestone, 1930,
Loading
  • baseline results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 1.474 ms [1.462 ms, 1.485 ms] -
appsec 2.351 ms [2.31 ms, 2.393 ms] 877.581 µs (59.6%)
iast 2.097 ms [2.044 ms, 2.15 ms] 623.569 µs (42.3%)
iast_GLOBAL 2.141 ms [2.088 ms, 2.194 ms] 667.143 µs (45.3%)
profiling 1.959 ms [1.916 ms, 2.001 ms] 485.061 µs (32.9%)
tracing 1.942 ms [1.902 ms, 1.983 ms] 468.774 µs (31.8%)
  • candidate results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 1.472 ms [1.461 ms, 1.484 ms] -
appsec 2.359 ms [2.317 ms, 2.401 ms] 886.498 µs (60.2%)
iast 2.096 ms [2.043 ms, 2.149 ms] 623.936 µs (42.4%)
iast_GLOBAL 2.142 ms [2.089 ms, 2.194 ms] 669.291 µs (45.5%)
profiling 1.949 ms [1.907 ms, 1.991 ms] 476.595 µs (32.4%)
tracing 1.93 ms [1.89 ms, 1.97 ms] 457.527 µs (31.1%)

PerfectSlayer
PerfectSlayer approved these changes Jan 2, 2025
View reviewed changes
Copy link
Contributor

@PerfectSlayer PerfectSlayer left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there a way to setup extended timeout for Semuru or is it not worth it?
I'm wondering if we are not making things worst for the other cases.

@nikita-tkachenko-datadog nikita-tkachenko-datadog marked this pull request as ready for review January 2, 2025 14:20
@nikita-tkachenko-datadog nikita-tkachenko-datadog requested a review from a team as a code owner January 2, 2025 14:20
@nikita-tkachenko-datadog
Copy link
Contributor Author

nikita-tkachenko-datadog commented Jan 2, 2025

Is there a way to setup extended timeout for Semuru or is it not worth it? I'm wondering if we are not making things worst for the other cases.

It shouldn't be worth in the other cases because the builds don't timeout with the other JVMs, so the execution time should not change

@nikita-tkachenko-datadog nikita-tkachenko-datadog merged commit 1951e6f into master Jan 2, 2025
212 checks passed
@nikita-tkachenko-datadog nikita-tkachenko-datadog deleted the nikita-tkachenko/gradle-launcher-smoke-timeout branch January 2, 2025 15:05
@github-actions github-actions bot added this to the 1.45.0 milestone Jan 2, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@PerfectSlayer PerfectSlayer PerfectSlayer approved these changes

@anmarchenko anmarchenko Awaiting requested review from anmarchenko anmarchenko is a code owner automatically assigned from DataDog/ci-app-libraries-java

Assignees

No one assigned

Labels

comp: testing Testing run-tests: all Run all tests tag: no release notes Changes to exclude from release notes

Projects

None yet

Milestone

1.45.0

Development

Successfully merging this pull request may close these issues.

3 participants

@nikita-tkachenko-datadog @PerfectSlayer