Skip to content

Conversation

rbuckton
Copy link
Contributor

@rbuckton rbuckton commented May 7, 2021

This reverts #35877 due to the severe negative performance impact caused by the change. We may attempt to reland this in TS 4.4 either without a flag (if we can mitigate the performance cost), or with a flag (so that you can opt in to the more expensive behavior).

Fixes #43486

/cc @DanielRosenwasser

@typescript-bot typescript-bot added Author: Team For Milestone Bug PRs that fix a bug with a specific milestone labels May 7, 2021
@rbuckton
Copy link
Contributor Author

rbuckton commented May 7, 2021

@typescript-bot perf test

@typescript-bot
Copy link
Collaborator

typescript-bot commented May 7, 2021

Heya @rbuckton, I've started to run the perf test suite on this PR at 76b2930. You can monitor the build here.

Update: The results are in!

@rbuckton
Copy link
Contributor Author

rbuckton commented May 7, 2021

Re-running perf tests to verify this addresses the issue.

@typescript-bot
Copy link
Collaborator

@rbuckton
The results of the perf run you requested are in!

Here they are:

Comparison Report - master..43993

Metric master 43993 Delta Best Worst
Angular - node (v10.16.3, x64)
Memory used 344,758k (± 0.02%) 344,711k (± 0.02%) -47k (- 0.01%) 344,490k 344,828k
Parse Time 1.92s (± 0.69%) 1.94s (± 0.60%) +0.01s (+ 0.78%) 1.91s 1.97s
Bind Time 0.84s (± 0.35%) 0.84s (± 0.71%) +0.01s (+ 0.84%) 0.84s 0.86s
Check Time 5.26s (± 0.55%) 5.26s (± 0.45%) +0.00s (+ 0.00%) 5.20s 5.31s
Emit Time 5.92s (± 0.92%) 5.61s (± 0.67%) 🟩-0.31s (- 5.18%) 5.55s 5.70s
Total Time 13.95s (± 0.62%) 13.66s (± 0.45%) -0.29s (- 2.09%) 13.53s 13.77s
Compiler-Unions - node (v10.16.3, x64)
Memory used 200,721k (± 0.11%) 200,242k (± 0.03%) -479k (- 0.24%) 200,064k 200,366k
Parse Time 0.78s (± 0.97%) 0.79s (± 0.66%) +0.01s (+ 0.64%) 0.78s 0.80s
Bind Time 0.53s (± 1.27%) 0.53s (± 1.25%) -0.00s (- 0.75%) 0.52s 0.54s
Check Time 7.55s (± 0.43%) 7.59s (± 0.65%) +0.03s (+ 0.40%) 7.50s 7.74s
Emit Time 2.52s (± 0.91%) 2.46s (± 0.76%) -0.06s (- 2.58%) 2.43s 2.52s
Total Time 11.39s (± 0.24%) 11.36s (± 0.41%) -0.03s (- 0.28%) 11.28s 11.49s
Monaco - node (v10.16.3, x64)
Memory used 341,681k (± 0.01%) 341,678k (± 0.02%) -4k (- 0.00%) 341,559k 341,791k
Parse Time 1.55s (± 0.44%) 1.57s (± 0.90%) +0.02s (+ 1.03%) 1.54s 1.60s
Bind Time 0.74s (± 0.64%) 0.75s (± 0.80%) +0.00s (+ 0.40%) 0.73s 0.76s
Check Time 5.38s (± 0.56%) 5.39s (± 0.73%) +0.01s (+ 0.22%) 5.32s 5.49s
Emit Time 3.01s (± 0.72%) 2.99s (± 0.69%) -0.03s (- 0.86%) 2.94s 3.03s
Total Time 10.69s (± 0.47%) 10.70s (± 0.57%) +0.01s (+ 0.06%) 10.56s 10.82s
TFS - node (v10.16.3, x64)
Memory used 304,239k (± 0.02%) 304,219k (± 0.02%) -21k (- 0.01%) 304,106k 304,302k
Parse Time 1.22s (± 0.55%) 1.23s (± 0.56%) +0.01s (+ 0.74%) 1.21s 1.24s
Bind Time 0.71s (± 0.67%) 0.71s (± 0.84%) -0.00s (- 0.14%) 0.69s 0.72s
Check Time 4.80s (± 0.44%) 4.82s (± 0.72%) +0.02s (+ 0.46%) 4.73s 4.88s
Emit Time 3.19s (± 1.09%) 3.12s (± 1.70%) -0.07s (- 2.26%) 3.00s 3.21s
Total Time 9.91s (± 0.43%) 9.87s (± 0.75%) -0.04s (- 0.38%) 9.69s 10.02s
material-ui - node (v10.16.3, x64)
Memory used 473,833k (± 0.02%) 473,782k (± 0.01%) -51k (- 0.01%) 473,658k 473,858k
Parse Time 1.94s (± 0.73%) 1.96s (± 0.60%) +0.02s (+ 0.93%) 1.93s 1.99s
Bind Time 0.65s (± 1.23%) 0.66s (± 0.84%) +0.01s (+ 0.92%) 0.65s 0.67s
Check Time 14.76s (± 0.50%) 14.92s (± 0.79%) +0.16s (+ 1.11%) 14.69s 15.19s
Emit Time 0.00s (± 0.00%) 0.00s (± 0.00%) 0.00s ( NaN%) 0.00s 0.00s
Total Time 17.36s (± 0.49%) 17.54s (± 0.63%) +0.18s (+ 1.05%) 17.31s 17.79s
Angular - node (v12.1.0, x64)
Memory used 322,346k (± 0.01%) 322,248k (± 0.02%) -98k (- 0.03%) 322,178k 322,464k
Parse Time 1.91s (± 0.80%) 1.93s (± 0.78%) +0.02s (+ 0.78%) 1.91s 1.97s
Bind Time 0.83s (± 1.23%) 0.83s (± 1.09%) +0.00s (+ 0.24%) 0.81s 0.85s
Check Time 5.17s (± 0.53%) 5.18s (± 0.46%) +0.01s (+ 0.23%) 5.13s 5.24s
Emit Time 6.15s (± 0.48%) 5.81s (± 0.57%) 🟩-0.34s (- 5.56%) 5.72s 5.88s
Total Time 14.06s (± 0.36%) 13.74s (± 0.36%) -0.31s (- 2.24%) 13.60s 13.85s
Compiler-Unions - node (v12.1.0, x64)
Memory used 187,872k (± 0.22%) 187,687k (± 0.10%) -184k (- 0.10%) 187,110k 188,015k
Parse Time 0.77s (± 0.52%) 0.78s (± 0.71%) +0.01s (+ 1.17%) 0.76s 0.79s
Bind Time 0.53s (± 0.76%) 0.53s (± 1.37%) +0.00s (+ 0.57%) 0.52s 0.55s
Check Time 7.01s (± 0.48%) 7.06s (± 0.77%) +0.04s (+ 0.63%) 6.95s 7.20s
Emit Time 2.49s (± 1.31%) 2.42s (± 1.14%) -0.06s (- 2.53%) 2.38s 2.52s
Total Time 10.80s (± 0.55%) 10.79s (± 0.65%) -0.00s (- 0.04%) 10.62s 10.94s
Monaco - node (v12.1.0, x64)
Memory used 324,004k (± 0.06%) 324,074k (± 0.02%) +70k (+ 0.02%) 323,941k 324,187k
Parse Time 1.53s (± 0.90%) 1.55s (± 0.62%) +0.02s (+ 0.98%) 1.52s 1.56s
Bind Time 0.72s (± 0.90%) 0.72s (± 0.66%) +0.01s (+ 0.70%) 0.71s 0.73s
Check Time 5.20s (± 0.48%) 5.23s (± 0.44%) +0.03s (+ 0.56%) 5.18s 5.28s
Emit Time 3.07s (± 0.91%) 3.04s (± 0.57%) -0.03s (- 0.88%) 3.00s 3.07s
Total Time 10.52s (± 0.36%) 10.54s (± 0.35%) +0.02s (+ 0.23%) 10.45s 10.62s
TFS - node (v12.1.0, x64)
Memory used 288,692k (± 0.03%) 288,718k (± 0.02%) +26k (+ 0.01%) 288,578k 288,827k
Parse Time 1.21s (± 0.58%) 1.23s (± 1.06%) +0.02s (+ 1.41%) 1.20s 1.26s
Bind Time 0.70s (± 1.07%) 0.70s (± 0.82%) +0.00s (+ 0.43%) 0.69s 0.71s
Check Time 4.71s (± 0.69%) 4.71s (± 0.49%) +0.01s (+ 0.17%) 4.66s 4.76s
Emit Time 3.19s (± 0.93%) 3.13s (± 0.94%) -0.06s (- 1.79%) 3.07s 3.21s
Total Time 9.80s (± 0.52%) 9.77s (± 0.55%) -0.03s (- 0.30%) 9.65s 9.88s
material-ui - node (v12.1.0, x64)
Memory used 451,627k (± 0.06%) 451,756k (± 0.02%) +129k (+ 0.03%) 451,586k 451,884k
Parse Time 1.94s (± 0.42%) 1.96s (± 0.51%) +0.02s (+ 1.19%) 1.94s 1.99s
Bind Time 0.64s (± 0.46%) 0.65s (± 0.53%) +0.01s (+ 0.78%) 0.64s 0.65s
Check Time 13.30s (± 0.73%) 13.43s (± 0.74%) +0.13s (+ 0.98%) 13.24s 13.67s
Emit Time 0.00s (± 0.00%) 0.00s (± 0.00%) 0.00s ( NaN%) 0.00s 0.00s
Total Time 15.88s (± 0.64%) 16.04s (± 0.63%) +0.16s (+ 1.02%) 15.85s 16.29s
Angular - node (v14.15.1, x64)
Memory used 320,913k (± 0.08%) 320,994k (± 0.01%) +81k (+ 0.03%) 320,936k 321,049k
Parse Time 1.90s (± 0.41%) 1.92s (± 0.49%) +0.02s (+ 1.16%) 1.91s 1.94s
Bind Time 0.87s (± 0.60%) 0.87s (± 0.42%) +0.00s (+ 0.34%) 0.87s 0.88s
Check Time 5.18s (± 0.85%) 5.19s (± 0.31%) +0.01s (+ 0.25%) 5.15s 5.23s
Emit Time 6.27s (± 0.88%) 5.88s (± 0.38%) 🟩-0.40s (- 6.30%) 5.83s 5.93s
Total Time 14.22s (± 0.71%) 13.86s (± 0.23%) -0.36s (- 2.52%) 13.81s 13.93s
Compiler-Unions - node (v14.15.1, x64)
Memory used 189,475k (± 0.50%) 187,702k (± 0.62%) -1,773k (- 0.94%) 186,409k 189,660k
Parse Time 0.80s (± 0.50%) 0.81s (± 0.46%) +0.01s (+ 0.75%) 0.80s 0.81s
Bind Time 0.56s (± 0.61%) 0.56s (± 0.40%) +0.00s (+ 0.72%) 0.56s 0.57s
Check Time 7.17s (± 0.60%) 7.22s (± 0.51%) +0.05s (+ 0.67%) 7.15s 7.34s
Emit Time 2.50s (± 0.80%) 2.46s (± 0.98%) -0.05s (- 1.84%) 2.41s 2.52s
Total Time 11.03s (± 0.51%) 11.04s (± 0.53%) +0.01s (+ 0.10%) 10.92s 11.22s
Monaco - node (v14.15.1, x64)
Memory used 323,136k (± 0.01%) 323,122k (± 0.01%) -14k (- 0.00%) 323,093k 323,192k
Parse Time 1.56s (± 0.47%) 1.58s (± 0.48%) +0.03s (+ 1.67%) 1.57s 1.60s
Bind Time 0.75s (± 0.69%) 0.75s (± 0.90%) +0.00s (+ 0.40%) 0.74s 0.77s
Check Time 5.20s (± 0.49%) 5.21s (± 0.43%) +0.01s (+ 0.17%) 5.15s 5.27s
Emit Time 3.13s (± 0.66%) 3.09s (± 0.37%) -0.04s (- 1.31%) 3.06s 3.11s
Total Time 10.64s (± 0.34%) 10.64s (± 0.28%) -0.00s (- 0.04%) 10.54s 10.68s
TFS - node (v14.15.1, x64)
Memory used 287,672k (± 0.01%) 287,650k (± 0.01%) -22k (- 0.01%) 287,589k 287,684k
Parse Time 1.27s (± 0.71%) 1.28s (± 1.19%) +0.01s (+ 0.71%) 1.24s 1.31s
Bind Time 0.72s (± 0.69%) 0.72s (± 0.51%) 0.00s ( 0.00%) 0.71s 0.72s
Check Time 4.75s (± 0.53%) 4.74s (± 0.40%) -0.01s (- 0.17%) 4.71s 4.79s
Emit Time 3.28s (± 0.66%) 3.24s (± 0.66%) -0.04s (- 1.34%) 3.19s 3.30s
Total Time 10.02s (± 0.44%) 9.98s (± 0.24%) -0.04s (- 0.44%) 9.92s 10.02s
material-ui - node (v14.15.1, x64)
Memory used 449,880k (± 0.06%) 449,975k (± 0.01%) +95k (+ 0.02%) 449,864k 450,079k
Parse Time 1.98s (± 0.64%) 2.00s (± 0.58%) +0.02s (+ 0.81%) 1.98s 2.03s
Bind Time 0.70s (± 0.68%) 0.71s (± 0.53%) +0.01s (+ 1.15%) 0.70s 0.71s
Check Time 13.64s (± 0.77%) 13.62s (± 0.78%) -0.02s (- 0.14%) 13.40s 13.82s
Emit Time 0.00s (± 0.00%) 0.00s (± 0.00%) 0.00s ( NaN%) 0.00s 0.00s
Total Time 16.32s (± 0.66%) 16.32s (± 0.69%) +0.00s (+ 0.02%) 16.11s 16.54s
System
Machine Namets-ci-ubuntu
Platformlinux 4.4.0-206-generic
Architecturex64
Available Memory16 GB
Available Memory7 GB
CPUs4 × Intel(R) Core(TM) i7-4770 CPU @ 3.40GHz
Hosts
  • node (v10.16.3, x64)
  • node (v12.1.0, x64)
  • node (v14.15.1, x64)
Scenarios
  • Angular - node (v10.16.3, x64)
  • Angular - node (v12.1.0, x64)
  • Angular - node (v14.15.1, x64)
  • Compiler-Unions - node (v10.16.3, x64)
  • Compiler-Unions - node (v12.1.0, x64)
  • Compiler-Unions - node (v14.15.1, x64)
  • Monaco - node (v10.16.3, x64)
  • Monaco - node (v12.1.0, x64)
  • Monaco - node (v14.15.1, x64)
  • TFS - node (v10.16.3, x64)
  • TFS - node (v12.1.0, x64)
  • TFS - node (v14.15.1, x64)
  • material-ui - node (v10.16.3, x64)
  • material-ui - node (v12.1.0, x64)
  • material-ui - node (v14.15.1, x64)
Benchmark Name Iterations
Current 43993 10
Baseline master 10

Developer Information:

Download Benchmark

@rbuckton rbuckton marked this pull request as ready for review May 7, 2021 17:44
Copy link
Member

@weswigham weswigham left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should have some RWC baselines to accept once this gets merged~

@rbuckton
Copy link
Contributor Author

rbuckton commented May 7, 2021

@typescript-bot test this
@typescript-bot user test this

@typescript-bot
Copy link
Collaborator

typescript-bot commented May 7, 2021

Heya @rbuckton, I've started to run the extended test suite on this PR at 7b7a0ea. You can monitor the build here.

@typescript-bot
Copy link
Collaborator

typescript-bot commented May 7, 2021

Heya @rbuckton, I've started to run the parallelized community code test suite on this PR at 7b7a0ea. You can monitor the build here.

@typescript-bot
Copy link
Collaborator

The user suite test run you requested has finished and failed. I've opened a PR with the baseline diff from master.

@rbuckton
Copy link
Contributor Author

rbuckton commented May 7, 2021

Baselines seem fine, I can merge the RWC changes afterwards.

@rbuckton rbuckton merged commit cb9cd89 into master May 7, 2021
@DanielRosenwasser
Copy link
Member

Thank you!

@jakebailey jakebailey deleted the revert35877 branch November 7, 2022 17:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Author: Team For Milestone Bug PRs that fix a bug with a specific milestone
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Perf regression: emit time
4 participants