Shortcut relations in getNarrowedTypeWorker when comparing literals by jakebailey · Pull Request #56072 · microsoft/TypeScript

Jake Bailey (jakebailey) · 2023-10-11T17:23:13Z

When narrowing large unions of literals, we do four different relation calls which appear to be quite costly. By checking ahead of time that we're going to relate two literals, we can skip this work by comparing the values directly and only once (as the result shouldn't be order dependent).

This leads to a significant speedup on #55948:

Benchmark 1: main
  Time (mean ± σ):     12.456 s ±  0.851 s    [User: 12.929 s, System: 0.059 s]
  Range (min … max):   11.219 s … 13.428 s    10 runs
 
Benchmark 2: PR
  Time (mean ± σ):      2.973 s ±  0.053 s    [User: 3.434 s, System: 0.057 s]
  Range (min … max):    2.906 s …  3.073 s    10 runs
 
Summary
  'PR' ran
    4.19 ± 0.30 times faster than 'main'

This PR passes all of the tests, but I think my implementation is probably wrong for some edge cases. ~~Like, I don't validate that the literal types are the same between the two, just their values, which is wrong. But, doesn't break?~~ Fixed that edge case.

Jake Bailey (jakebailey) · 2023-10-11T17:23:57Z

TypeScript Bot (@typescript-bot) test top100
TypeScript Bot (@typescript-bot) user test this
TypeScript Bot (@typescript-bot) run dt
TypeScript Bot (@typescript-bot) perf test this
TypeScript Bot (@typescript-bot) pack this

TypeScript Bot (typescript-bot) · 2023-10-11T17:24:00Z

Heya Jake Bailey (@jakebailey), I've started to run the regular perf test suite on this PR at 54d0bc7. You can monitor the build here.

Update: The results are in!

TypeScript Bot (typescript-bot) · 2023-10-11T17:24:00Z

Heya Jake Bailey (@jakebailey), I've started to run the diff-based top-repos suite on this PR at 54d0bc7. You can monitor the build here.

Update: The results are in!

TypeScript Bot (typescript-bot) · 2023-10-11T17:24:00Z

Heya Jake Bailey (@jakebailey), I've started to run the parallelized Definitely Typed test suite on this PR at 54d0bc7. You can monitor the build here.

Update: The results are in!

TypeScript Bot (typescript-bot) · 2023-10-11T17:24:00Z

Heya Jake Bailey (@jakebailey), I've started to run the diff-based user code test suite on this PR at 54d0bc7. You can monitor the build here.

Update: The results are in!

TypeScript Bot (typescript-bot) · 2023-10-11T17:24:00Z

Heya Jake Bailey (@jakebailey), I've started to run the tarball bundle task on this PR at 54d0bc7. You can monitor the build here.

TypeScript Bot (typescript-bot) · 2023-10-11T17:25:54Z

Hey Jake Bailey (@jakebailey), I've packed this into an installable tgz. You can install it for testing by referencing it in your package.json like so:

{
    "devDependencies": {
        "typescript": "https://typescript.visualstudio.com/cf7ac146-d525-443c-b23c-0d58337efebc/_apis/build/builds/158179/artifacts?artifactName=tgz&fileId=130E9BE142C9389724046D51A9B44F05AB5048F38496A635A91CD40D59518E7802&fileName=/typescript-5.3.0-insiders.20231011.tgz"
    }
}

and then running npm install.

There is also a playground for this build and an npm module you can use via "typescript": "npm:@typescript-deploys/pr-build@5.3.0-pr-56072-6".;

TypeScript Bot (typescript-bot) · 2023-10-11T17:41:44Z

Jake Bailey (@jakebailey) Here are the results of running the user test suite comparing main and refs/pull/56072/merge:

There were infrastructure failures potentially unrelated to your change:

3 instances of "Package install failed"
1 instance of "Unknown failure"

Otherwise...

Everything looks good!

TypeScript Bot (typescript-bot) · 2023-10-11T17:43:42Z

Jake Bailey (@jakebailey)
The results of the perf run you requested are in!

Here they are:

Compiler

Comparison Report - baseline..pr

_Metric	_baseline	_pr	_Delta	_Best	_Worst	_p-value
_{Angular - node (v18.15.0, x64)}
_{Memory used}	_{295,117k (± 0.01%)}	_{295,130k (± 0.01%)}	_~	_295,096k	_295,184k	_{p=0.688 n=6}
_{Parse Time}	_{2.63s (± 0.57%)}	_{2.64s (± 0.31%)}	_~	_2.63s	_2.65s	_{p=0.236 n=6}
_{Bind Time}	_{0.84s (± 1.17%)}	_{0.84s (± 0.97%)}	_~	_0.83s	_0.85s	_{p=0.862 n=6}
_{Check Time}	_{8.07s (± 0.11%)}	_{8.08s (± 0.31%)}	_~	_8.04s	_8.11s	_{p=0.459 n=6}
_{Emit Time}	_{7.05s (± 0.11%)}	_{7.03s (± 0.42%)}	_~	_6.99s	_7.06s	_{p=0.406 n=6}
_{Total Time}	_{18.59s (± 0.10%)}	_{18.59s (± 0.26%)}	_~	_18.52s	_18.65s	_{p=0.747 n=6}
_{Compiler-Unions - node (v18.15.0, x64)}
_{Memory used}	_{192,611k (± 1.59%)}	_{194,495k (± 1.53%)}	_~	_190,608k	_196,470k	_{p=0.810 n=6}
_{Parse Time}	_{1.34s (± 0.47%)}	_{1.34s (± 0.56%)}	_~	_1.33s	_1.35s	_{p=0.718 n=6}
_{Bind Time}	_{0.73s (± 0.00%)}	_{0.73s (± 0.56%)}	_~	_0.72s	_0.73s	_{p=0.405 n=6}
_{Check Time}	_{9.13s (± 0.26%)}	_{9.10s (± 0.44%)}	_~	_9.06s	_9.15s	_{p=0.257 n=6}
_{Emit Time}	_{2.63s (± 0.20%)}	_{2.63s (± 0.44%)}	_~	_2.61s	_2.64s	_{p=0.195 n=6}
_{Total Time}	_{13.83s (± 0.10%)}	_{13.80s (± 0.30%)}	_~	_13.75s	_13.85s	_{p=0.512 n=6}
_{Monaco - node (v18.15.0, x64)}
_{Memory used}	_{347,305k (± 0.00%)}	_{347,317k (± 0.00%)}	_~	_347,304k	_347,321k	_{p=0.054 n=6}
_{Parse Time}	_{2.46s (± 0.31%)}	_{2.46s (± 0.33%)}	_~	_2.45s	_2.47s	_{p=0.306 n=6}
_{Bind Time}	_{0.94s (± 0.43%)}	_{0.94s (± 0.00%)}	_~	_0.94s	_0.94s	_{p=0.405 n=6}
_{Check Time}	_{6.91s (± 0.25%)}	_{6.92s (± 0.44%)}	_~	_6.87s	_6.96s	_{p=0.568 n=6}
_{Emit Time}	_{4.01s (± 0.34%)}	_{4.03s (± 0.53%)}	_~	_4.01s	_4.07s	_{p=0.391 n=6}
_{Total Time}	_{14.33s (± 0.21%)}	_{14.36s (± 0.33%)}	_~	_14.32s	_14.45s	_{p=0.627 n=6}
_{TFS - node (v18.15.0, x64)}
_{Memory used}	_{302,546k (± 0.01%)}	_{302,548k (± 0.01%)}	_~	_302,525k	_302,576k	_{p=0.936 n=6}
_{Parse Time}	_{1.99s (± 0.45%)}	_{1.99s (± 1.00%)}	_~	_1.97s	_2.02s	_{p=0.934 n=6}
_{Bind Time}	_{1.01s (± 0.97%)}	_{1.00s (± 0.41%)}	_~	_1.00s	_1.01s	_{p=0.213 n=6}
_{Check Time}	_{6.25s (± 0.22%)}	_{6.27s (± 0.41%)}	_~	_6.23s	_6.29s	_{p=0.462 n=6}
_{Emit Time}	_{3.56s (± 0.98%)}	_{3.56s (± 0.48%)}	_~	_3.54s	_3.59s	_{p=1.000 n=6}
_{Total Time}	_{12.82s (± 0.33%)}	_{12.82s (± 0.23%)}	_~	_12.77s	_12.85s	_{p=1.000 n=6}
_{material-ui - node (v18.15.0, x64)}
_{Memory used}	_{470,507k (± 0.00%)}	_{470,488k (± 0.00%)}	_~	_470,457k	_470,512k	_{p=0.066 n=6}
_{Parse Time}	_{2.57s (± 0.75%)}	_{2.58s (± 0.45%)}	_~	_2.57s	_2.60s	_{p=0.410 n=6}
_{Bind Time}	_{1.00s (± 1.33%)}	_{1.00s (± 1.60%)}	_~	_0.97s	_1.02s	_{p=0.720 n=6}
_{Check Time}	_{16.57s (± 0.19%)}	_{16.58s (± 0.43%)}	_~	_16.52s	_16.72s	_{p=1.000 n=6}
_{Emit Time}	_{0.00s (± 0.00%)}	_{0.00s (± 0.00%)}	_~	_0.00s	_0.00s	_{p=1.000 n=6}
_{Total Time}	_{20.14s (± 0.23%)}	_{20.16s (± 0.35%)}	_~	_20.10s	_20.29s	_{p=0.687 n=6}
_{xstate - node (v18.15.0, x64)}
_{Memory used}	_{512,620k (± 0.01%)}	_{512,637k (± 0.00%)}	_~	_512,615k	_512,674k	_{p=0.173 n=6}
_{Parse Time}	_{3.27s (± 0.42%)}	_{3.27s (± 0.25%)}	_~	_3.26s	_3.28s	_{p=0.933 n=6}
_{Bind Time}	_{1.55s (± 0.48%)}	_{1.55s (± 0.41%)}	_~	_1.54s	_1.56s	_{p=0.718 n=6}
_{Check Time}	_{2.86s (± 0.84%)}	_{2.86s (± 0.49%)}	_~	_2.84s	_2.88s	_{p=0.935 n=6}
_{Emit Time}	_{0.08s (± 4.99%)}	_{0.08s (± 4.99%)}	_~	_0.08s	_0.09s	_{p=1.000 n=6}
_{Total Time}	_{7.76s (± 0.25%)}	_{7.76s (± 0.26%)}	_~	_7.73s	_7.79s	_{p=1.000 n=6}

System info unknown

Hosts

node (v18.15.0, x64)

Scenarios

Angular - node (v18.15.0, x64)
Compiler-Unions - node (v18.15.0, x64)
Monaco - node (v18.15.0, x64)
TFS - node (v18.15.0, x64)
material-ui - node (v18.15.0, x64)
xstate - node (v18.15.0, x64)

_Benchmark	_Name	_Iterations
_Current	_pr	₆
_Baseline	_baseline	₆

tsserver

Comparison Report - baseline..pr

_Metric	_baseline	_pr	_Delta	_Best	_Worst	_p-value
_{Compiler-UnionsTSServer - node (v18.15.0, x64)}
_{Req 1 - updateOpen}	_{2,354ms (± 0.94%)}	_{2,341ms (± 0.21%)}	_~	_2,336ms	_2,350ms	_{p=0.575 n=6}
_{Req 2 - geterr}	_{5,393ms (± 1.66%)}	_{5,404ms (± 1.51%)}	_~	_5,285ms	_5,475ms	_{p=0.575 n=6}
_{Req 3 - references}	_{328ms (± 0.86%)}	_{326ms (± 0.36%)}	_~	_325ms	_328ms	_{p=0.558 n=6}
_{Req 4 - navto}	_{276ms (± 1.14%)}	_{277ms (± 1.51%)}	_~	_273ms	_282ms	_{p=0.805 n=6}
_{Req 5 - completionInfo count}	_{1,356 (± 0.00%)}	_{1,356 (± 0.00%)}	_~	_1,356	_1,356	_{p=1.000 n=6}
_{Req 5 - completionInfo}	_{85ms (± 8.94%)}	_{85ms (± 8.98%)}	_~	_75ms	_90ms	_{p=0.491 n=6}
_{CompilerTSServer - node (v18.15.0, x64)}
_{Req 1 - updateOpen}	_{2,490ms (± 0.67%)}	_{2,452ms (± 1.41%)}	_~	_2,410ms	_2,505ms	_{p=0.054 n=6}
_{Req 2 - geterr}	_{4,086ms (± 1.47%)}	_{4,185ms (± 1.44%)}	_~	_4,063ms	_4,225ms	_{p=0.066 n=6}
_{Req 3 - references}	_{341ms (± 1.21%)}	_{335ms (± 1.06%)}	_{-7ms (- 1.90%)}	_333ms	_342ms	_{p=0.039 n=6}
_{Req 4 - navto}	_{285ms (± 0.37%)}	_{283ms (± 0.62%)}	_{-2ms (- 0.70%)}	_279ms	_284ms	_{p=0.024 n=6}
_{Req 5 - completionInfo count}	_{1,518 (± 0.00%)}	_{1,518 (± 0.00%)}	_~	_1,518	_1,518	_{p=1.000 n=6}
_{Req 5 - completionInfo}	_{85ms (± 5.32%)}	_{77ms (± 6.31%)}	_{🟩-8ms (- 9.39%)}	_74ms	_87ms	_{p=0.032 n=6}
_{xstateTSServer - node (v18.15.0, x64)}
_{Req 1 - updateOpen}	_{2,597ms (± 0.38%)}	_{2,595ms (± 0.62%)}	_~	_2,562ms	_2,604ms	_{p=0.521 n=6}
_{Req 2 - geterr}	_{1,700ms (± 3.31%)}	_{1,682ms (± 2.45%)}	_~	_1,610ms	_1,728ms	_{p=0.936 n=6}
_{Req 3 - references}	_{111ms (± 7.25%)}	_{112ms (± 7.96%)}	_~	_105ms	_126ms	_{p=1.000 n=6}
_{Req 4 - navto}	_{361ms (± 1.29%)}	_{359ms (± 0.33%)}	_~	_358ms	_361ms	_{p=0.803 n=6}
_{Req 5 - completionInfo count}	_{2,073 (± 0.00%)}	_{2,073 (± 0.00%)}	_~	_2,073	_2,073	_{p=1.000 n=6}
_{Req 5 - completionInfo}	_{305ms (± 1.85%)}	_{306ms (± 1.13%)}	_~	_300ms	_310ms	_{p=1.000 n=6}

System info unknown

Hosts

node (v18.15.0, x64)

Scenarios

CompilerTSServer - node (v18.15.0, x64)
Compiler-UnionsTSServer - node (v18.15.0, x64)
xstateTSServer - node (v18.15.0, x64)

_Benchmark	_Name	_Iterations
_Current	_pr	₆
_Baseline	_baseline	₆

Startup

Comparison Report - baseline..pr

_Metric	_baseline	_pr	_Delta	_Best	_Worst	_p-value
_{tsc-startup - node (v18.15.0, x64)}
_{Execution time}	_{152.38ms (± 0.18%)}	_{152.26ms (± 0.18%)}	_{-0.13ms (- 0.08%)}	_151.26ms	_155.17ms	_{p=0.000 n=600}
_{tsserver-startup - node (v18.15.0, x64)}
_{Execution time}	_{227.82ms (± 0.16%)}	_{227.77ms (± 0.17%)}	_{-0.05ms (- 0.02%)}	_226.26ms	_233.24ms	_{p=0.046 n=600}
_{tsserverlibrary-startup - node (v18.15.0, x64)}
_{Execution time}	_{228.64ms (± 0.17%)}	_{228.59ms (± 0.19%)}	_~	_226.96ms	_240.71ms	_{p=0.169 n=600}
_{typescript-startup - node (v18.15.0, x64)}
_{Execution time}	_{228.89ms (± 0.17%)}	_{228.94ms (± 0.16%)}	_~	_227.44ms	_233.23ms	_{p=0.229 n=600}

System info unknown

Hosts

node (v18.15.0, x64)

Scenarios

tsc-startup - node (v18.15.0, x64)
tsserver-startup - node (v18.15.0, x64)
tsserverlibrary-startup - node (v18.15.0, x64)
typescript-startup - node (v18.15.0, x64)

_Benchmark	_Name	_Iterations
_Current	_pr	₆
_Baseline	_baseline	₆

Developer Information:

Download Benchmarks

TypeScript Bot (typescript-bot) · 2023-10-11T18:45:15Z

Jake Bailey (@jakebailey) Here are the results of running the top-repos suite comparing main and refs/pull/56072/merge:

Everything looks good!

TypeScript Bot (typescript-bot) · 2023-10-11T18:58:08Z

Hey Jake Bailey (@jakebailey), the results of running the DT tests are ready.
Everything looks the same!
You can check the log here.

Jake Bailey (jakebailey) · 2023-10-12T05:51:37Z

TypeScript Bot (@typescript-bot) perf test this

TypeScript Bot (typescript-bot) · 2023-10-12T05:51:40Z

Heya Jake Bailey (@jakebailey), I've started to run the regular perf test suite on this PR at 629de0d. You can monitor the build here.

Update: The results are in!

TypeScript Bot (typescript-bot) · 2023-10-12T06:13:33Z

Jake Bailey (@jakebailey)
The results of the perf run you requested are in!

Here they are:

Compiler

Comparison Report - baseline..pr

_Metric	_baseline	_pr	_Delta	_Best	_Worst	_p-value
_{Angular - node (v18.15.0, x64)}
_{Memory used}	_{295,131k (± 0.01%)}	_{295,104k (± 0.02%)}	_~	_294,993k	_295,136k	_{p=0.575 n=6}
_{Parse Time}	_{2.63s (± 0.21%)}	_{2.64s (± 0.61%)}	_~	_2.61s	_2.65s	_{p=0.357 n=6}
_{Bind Time}	_{0.83s (± 0.62%)}	_{0.84s (± 0.97%)}	_~	_0.83s	_0.85s	_{p=0.523 n=6}
_{Check Time}	_{8.06s (± 0.36%)}	_{8.06s (± 0.31%)}	_~	_8.02s	_8.09s	_{p=0.683 n=6}
_{Emit Time}	_{7.03s (± 0.49%)}	_{7.05s (± 0.17%)}	_~	_7.03s	_7.06s	_{p=0.466 n=6}
_{Total Time}	_{18.57s (± 0.23%)}	_{18.58s (± 0.13%)}	_~	_18.56s	_18.63s	_{p=0.418 n=6}
_{Compiler-Unions - node (v18.15.0, x64)}
_{Memory used}	_{191,611k (± 1.24%)}	_{190,628k (± 0.01%)}	_~	_190,603k	_190,656k	_{p=0.229 n=6}
_{Parse Time}	_{1.35s (± 1.11%)}	_{1.35s (± 0.90%)}	_~	_1.34s	_1.37s	_{p=0.343 n=6}
_{Bind Time}	_{0.73s (± 0.00%)}	_{0.73s (± 0.00%)}	_~	_0.73s	_0.73s	_{p=1.000 n=6}
_{Check Time}	_{9.13s (± 0.28%)}	_{9.13s (± 0.54%)}	_~	_9.06s	_9.21s	_{p=0.624 n=6}
_{Emit Time}	_{2.63s (± 0.44%)}	_{2.63s (± 0.61%)}	_~	_2.62s	_2.66s	_{p=0.438 n=6}
_{Total Time}	_{13.85s (± 0.11%)}	_{13.83s (± 0.30%)}	_~	_13.77s	_13.90s	_{p=0.226 n=6}
_{Monaco - node (v18.15.0, x64)}
_{Memory used}	_{347,316k (± 0.01%)}	_{347,324k (± 0.01%)}	_~	_347,289k	_347,362k	_{p=0.575 n=6}
_{Parse Time}	_{2.45s (± 0.21%)}	_{2.46s (± 0.54%)}	_~	_2.44s	_2.47s	_{p=0.546 n=6}
_{Bind Time}	_{0.94s (± 0.43%)}	_{0.94s (± 0.00%)}	_~	_0.94s	_0.94s	_{p=0.405 n=6}
_{Check Time}	_{6.92s (± 0.46%)}	_{6.93s (± 0.73%)}	_~	_6.88s	_7.00s	_{p=1.000 n=6}
_{Emit Time}	_{4.02s (± 0.41%)}	_{4.00s (± 0.26%)}	_{-0.02s (- 0.50%)}	_3.99s	_4.02s	_{p=0.049 n=6}
_{Total Time}	_{14.33s (± 0.26%)}	_{14.33s (± 0.38%)}	_~	_14.26s	_14.41s	_{p=0.748 n=6}
_{TFS - node (v18.15.0, x64)}
_{Memory used}	_{302,568k (± 0.01%)}	_{302,538k (± 0.00%)}	_{-31k (- 0.01%)}	_302,524k	_302,550k	_{p=0.013 n=6}
_{Parse Time}	_{2.01s (± 1.03%)}	_{2.00s (± 0.94%)}	_~	_1.96s	_2.01s	_{p=0.514 n=6}
_{Bind Time}	_{1.01s (± 0.97%)}	_{1.00s (± 0.75%)}	_~	_0.99s	_1.01s	_{p=0.102 n=6}
_{Check Time}	_{6.25s (± 0.29%)}	_{6.25s (± 0.35%)}	_~	_6.22s	_6.28s	_{p=0.685 n=6}
_{Emit Time}	_{3.56s (± 0.42%)}	_{3.55s (± 0.53%)}	_~	_3.53s	_3.58s	_{p=0.934 n=6}
_{Total Time}	_{12.82s (± 0.20%)}	_{12.80s (± 0.29%)}	_~	_12.75s	_12.85s	_{p=0.470 n=6}
_{material-ui - node (v18.15.0, x64)}
_{Memory used}	_{470,499k (± 0.01%)}	_{470,489k (± 0.00%)}	_~	_470,465k	_470,517k	_{p=0.630 n=6}
_{Parse Time}	_{2.59s (± 1.02%)}	_{2.57s (± 0.62%)}	_~	_2.55s	_2.59s	_{p=0.120 n=6}
_{Bind Time}	_{0.99s (± 0.64%)}	_{0.99s (± 0.84%)}	_~	_0.99s	_1.01s	_{p=0.340 n=6}
_{Check Time}	_{16.61s (± 0.55%)}	_{16.58s (± 0.13%)}	_~	_16.56s	_16.62s	_{p=0.258 n=6}
_{Emit Time}	_{0.00s (± 0.00%)}	_{0.00s (± 0.00%)}	_~	_0.00s	_0.00s	_{p=1.000 n=6}
_{Total Time}	_{20.19s (± 0.54%)}	_{20.15s (± 0.18%)}	_~	_20.12s	_20.21s	_{p=0.199 n=6}
_{xstate - node (v18.15.0, x64)}
_{Memory used}	_{512,615k (± 0.01%)}	_{512,658k (± 0.01%)}	_~	_512,617k	_512,717k	_{p=0.128 n=6}
_{Parse Time}	_{3.26s (± 0.27%)}	_{3.27s (± 0.30%)}	_~	_3.26s	_3.28s	_{p=0.082 n=6}
_{Bind Time}	_{1.55s (± 0.53%)}	_{1.54s (± 0.35%)}	_~	_1.54s	_1.55s	_{p=0.088 n=6}
_{Check Time}	_{2.88s (± 0.90%)}	_{2.88s (± 0.87%)}	_~	_2.86s	_2.93s	_{p=0.687 n=6}
_{Emit Time}	_{0.08s (± 4.99%)}	_{0.08s (± 4.99%)}	_~	_0.08s	_0.09s	_{p=1.000 n=6}
_{Total Time}	_{7.77s (± 0.41%)}	_{7.78s (± 0.42%)}	_~	_7.75s	_7.84s	_{p=0.327 n=6}

System info unknown

Hosts

node (v18.15.0, x64)

Scenarios

Angular - node (v18.15.0, x64)
Compiler-Unions - node (v18.15.0, x64)
Monaco - node (v18.15.0, x64)
TFS - node (v18.15.0, x64)
material-ui - node (v18.15.0, x64)
xstate - node (v18.15.0, x64)

_Benchmark	_Name	_Iterations
_Current	_pr	₆
_Baseline	_baseline	₆

tsserver

Comparison Report - baseline..pr

_Metric	_baseline	_pr	_Delta	_Best	_Worst	_p-value
_{Compiler-UnionsTSServer - node (v18.15.0, x64)}
_{Req 1 - updateOpen}	_{2,381ms (± 0.94%)}	_{2,366ms (± 1.62%)}	_~	_2,318ms	_2,407ms	_{p=0.810 n=6}
_{Req 2 - geterr}	_{5,316ms (± 1.45%)}	_{5,370ms (± 1.58%)}	_~	_5,287ms	_5,453ms	_{p=0.229 n=6}
_{Req 3 - references}	_{331ms (± 1.59%)}	_{327ms (± 0.95%)}	_~	_324ms	_333ms	_{p=0.373 n=6}
_{Req 4 - navto}	_{280ms (± 1.42%)}	_{277ms (± 1.36%)}	_~	_273ms	_282ms	_{p=0.224 n=6}
_{Req 5 - completionInfo count}	_{1,356 (± 0.00%)}	_{1,356 (± 0.00%)}	_~	_1,356	_1,356	_{p=1.000 n=6}
_{Req 5 - completionInfo}	_{79ms (± 7.24%)}	_{83ms (± 9.69%)}	_~	_75ms	_91ms	_{p=0.462 n=6}
_{CompilerTSServer - node (v18.15.0, x64)}
_{Req 1 - updateOpen}	_{2,492ms (± 0.53%)}	_{2,480ms (± 0.93%)}	_~	_2,440ms	_2,500ms	_{p=0.423 n=6}
_{Req 2 - geterr}	_{4,111ms (± 1.79%)}	_{4,107ms (± 1.78%)}	_~	_4,052ms	_4,204ms	_{p=0.471 n=6}
_{Req 3 - references}	_{340ms (± 1.46%)}	_{340ms (± 1.61%)}	_~	_332ms	_344ms	_{p=0.870 n=6}
_{Req 4 - navto}	_{284ms (± 0.44%)}	_{284ms (± 0.36%)}	_~	_282ms	_285ms	_{p=0.560 n=6}
_{Req 5 - completionInfo count}	_{1,518 (± 0.00%)}	_{1,518 (± 0.00%)}	_~	_1,518	_1,518	_{p=1.000 n=6}
_{Req 5 - completionInfo}	_{83ms (± 7.02%)}	_{83ms (± 6.52%)}	_~	_76ms	_87ms	_{p=1.000 n=6}
_{xstateTSServer - node (v18.15.0, x64)}
_{Req 1 - updateOpen}	_{2,588ms (± 0.69%)}	_{2,599ms (± 0.41%)}	_~	_2,587ms	_2,611ms	_{p=0.422 n=6}
_{Req 2 - geterr}	_{1,692ms (± 1.81%)}	_{1,715ms (± 2.38%)}	_~	_1,657ms	_1,763ms	_{p=0.423 n=6}
_{Req 3 - references}	_{106ms (± 0.52%)}	_{118ms (± 7.85%)}	_{🔻+12ms (+11.53%)}	_106ms	_127ms	_{p=0.015 n=6}
_{Req 4 - navto}	_{359ms (± 0.25%)}	_{360ms (± 1.09%)}	_~	_357ms	_368ms	_{p=0.804 n=6}
_{Req 5 - completionInfo count}	_{2,073 (± 0.00%)}	_{2,073 (± 0.00%)}	_~	_2,073	_2,073	_{p=1.000 n=6}
_{Req 5 - completionInfo}	_{305ms (± 2.33%)}	_{308ms (± 2.88%)}	_~	_296ms	_319ms	_{p=0.748 n=6}

System info unknown

Hosts

node (v18.15.0, x64)

Scenarios

CompilerTSServer - node (v18.15.0, x64)
Compiler-UnionsTSServer - node (v18.15.0, x64)
xstateTSServer - node (v18.15.0, x64)

_Benchmark	_Name	_Iterations
_Current	_pr	₆
_Baseline	_baseline	₆

Startup

Comparison Report - baseline..pr

_Metric	_baseline	_pr	_Delta	_Best	_Worst	_p-value
_{tsc-startup - node (v18.15.0, x64)}
_{Execution time}	_{152.49ms (± 0.18%)}	_{152.27ms (± 0.17%)}	_{-0.22ms (- 0.15%)}	_151.27ms	_155.68ms	_{p=0.000 n=600}
_{tsserver-startup - node (v18.15.0, x64)}
_{Execution time}	_{227.89ms (± 0.14%)}	_{227.89ms (± 0.17%)}	_~	_226.49ms	_232.25ms	_{p=0.251 n=600}
_{tsserverlibrary-startup - node (v18.15.0, x64)}
_{Execution time}	_{228.93ms (± 0.15%)}	_{229.02ms (± 0.20%)}	_~	_227.43ms	_240.23ms	_{p=0.185 n=600}
_{typescript-startup - node (v18.15.0, x64)}
_{Execution time}	_{228.19ms (± 0.15%)}	_{228.21ms (± 0.16%)}	_~	_226.82ms	_232.43ms	_{p=0.680 n=600}

System info unknown

Hosts

node (v18.15.0, x64)

Scenarios

tsc-startup - node (v18.15.0, x64)
tsserver-startup - node (v18.15.0, x64)
tsserverlibrary-startup - node (v18.15.0, x64)
typescript-startup - node (v18.15.0, x64)

_Benchmark	_Name	_Iterations
_Current	_pr	₆
_Baseline	_baseline	₆

Developer Information:

Download Benchmarks

Jake Bailey (jakebailey) · 2023-10-12T21:17:53Z

TypeScript Bot (@typescript-bot) test top200
TypeScript Bot (@typescript-bot) user test this
TypeScript Bot (@typescript-bot) perf test this

TypeScript Bot (typescript-bot) · 2023-10-12T21:17:56Z

Heya Jake Bailey (@jakebailey), I've started to run the regular perf test suite on this PR at e68e177. You can monitor the build here.

Update: The results are in!

TypeScript Bot (typescript-bot) · 2023-10-12T21:17:56Z

Heya Jake Bailey (@jakebailey), I've started to run the diff-based top-repos suite on this PR at e68e177. You can monitor the build here.

Update: The results are in!

TypeScript Bot (typescript-bot) · 2023-10-12T21:17:56Z

Heya Jake Bailey (@jakebailey), I've started to run the diff-based user code test suite on this PR at e68e177. You can monitor the build here.

Update: The results are in!

TypeScript Bot (typescript-bot) · 2023-10-12T21:36:29Z

Jake Bailey (@jakebailey) Here are the results of running the user test suite comparing main and refs/pull/56072/merge:

There were infrastructure failures potentially unrelated to your change:

3 instances of "Package install failed"
1 instance of "Unknown failure"

Otherwise...

Everything looks good!

TypeScript Bot (typescript-bot) · 2023-10-12T21:38:22Z

Jake Bailey (@jakebailey)
The results of the perf run you requested are in!

Here they are:

Compiler

Comparison Report - baseline..pr

_Metric	_baseline	_pr	_Delta	_Best	_Worst	_p-value
_{Angular - node (v18.15.0, x64)}
_{Memory used}	_{295,149k (± 0.01%)}	_{295,134k (± 0.02%)}	_~	_295,082k	_295,192k	_{p=1.000 n=6}
_{Parse Time}	_{2.65s (± 0.32%)}	_{2.63s (± 0.71%)}	_~	_2.60s	_2.65s	_{p=0.065 n=6}
_{Bind Time}	_{0.83s (± 0.66%)}	_{0.84s (± 1.30%)}	_~	_0.83s	_0.85s	_{p=0.487 n=6}
_{Check Time}	_{8.05s (± 0.33%)}	_{8.08s (± 0.34%)}	_{+0.03s (+ 0.41%)}	_8.06s	_8.12s	_{p=0.035 n=6}
_{Emit Time}	_{7.07s (± 0.33%)}	_{7.06s (± 0.39%)}	_~	_7.03s	_7.10s	_{p=0.683 n=6}
_{Total Time}	_{18.60s (± 0.16%)}	_{18.61s (± 0.20%)}	_~	_18.57s	_18.66s	_{p=0.625 n=6}
_{Compiler-Unions - node (v18.15.0, x64)}
_{Memory used}	_{192,550k (± 1.54%)}	_{191,599k (± 1.23%)}	_~	_190,592k	_196,433k	_{p=0.872 n=6}
_{Parse Time}	_{1.34s (± 0.38%)}	_{1.36s (± 1.27%)}	_~	_1.34s	_1.38s	_{p=0.142 n=6}
_{Bind Time}	_{0.73s (± 0.00%)}	_{0.73s (± 0.00%)}	_~	_0.73s	_0.73s	_{p=1.000 n=6}
_{Check Time}	_{9.14s (± 0.37%)}	_{9.20s (± 0.52%)}	_{+0.06s (+ 0.66%)}	_9.13s	_9.26s	_{p=0.045 n=6}
_{Emit Time}	_{2.63s (± 0.39%)}	_{2.63s (± 0.46%)}	_~	_2.61s	_2.64s	_{p=0.735 n=6}
_{Total Time}	_{13.85s (± 0.35%)}	_{13.93s (± 0.34%)}	_{+0.08s (+ 0.58%)}	_13.84s	_13.97s	_{p=0.036 n=6}
_{Monaco - node (v18.15.0, x64)}
_{Memory used}	_{347,312k (± 0.01%)}	_{347,345k (± 0.00%)}	_{+33k (+ 0.01%)}	_347,338k	_347,358k	_{p=0.030 n=6}
_{Parse Time}	_{2.45s (± 0.87%)}	_{2.45s (± 0.61%)}	_~	_2.44s	_2.47s	_{p=0.869 n=6}
_{Bind Time}	_{0.95s (± 1.68%)}	_{0.94s (± 0.00%)}	_~	_0.94s	_0.94s	_{p=0.074 n=6}
_{Check Time}	_{6.95s (± 0.46%)}	_{6.96s (± 0.82%)}	_~	_6.90s	_7.03s	_{p=0.936 n=6}
_{Emit Time}	_{4.03s (± 0.49%)}	_{4.01s (± 0.37%)}	_~	_4.00s	_4.04s	_{p=0.251 n=6}
_{Total Time}	_{14.37s (± 0.34%)}	_{14.37s (± 0.27%)}	_~	_14.32s	_14.42s	_{p=0.685 n=6}
_{TFS - node (v18.15.0, x64)}
_{Memory used}	_{302,546k (± 0.01%)}	_{302,591k (± 0.00%)}	_{+45k (+ 0.01%)}	_302,567k	_302,601k	_{p=0.008 n=6}
_{Parse Time}	_{2.00s (± 0.60%)}	_{1.99s (± 0.88%)}	_~	_1.98s	_2.02s	_{p=0.251 n=6}
_{Bind Time}	_{1.00s (± 0.81%)}	_{1.01s (± 0.54%)}	_{+0.01s (+ 1.16%)}	_1.01s	_1.02s	_{p=0.027 n=6}
_{Check Time}	_{6.27s (± 0.23%)}	_{6.27s (± 0.35%)}	_~	_6.23s	_6.29s	_{p=0.869 n=6}
_{Emit Time}	_{3.56s (± 0.38%)}	_{3.57s (± 0.46%)}	_~	_3.55s	_3.59s	_{p=0.088 n=6}
_{Total Time}	_{12.83s (± 0.08%)}	_{12.85s (± 0.25%)}	_~	_12.81s	_12.90s	_{p=0.744 n=6}
_{material-ui - node (v18.15.0, x64)}
_{Memory used}	_{470,487k (± 0.00%)}	_{470,494k (± 0.00%)}	_~	_470,467k	_470,512k	_{p=0.470 n=6}
_{Parse Time}	_{2.57s (± 0.29%)}	_{2.57s (± 0.40%)}	_~	_2.56s	_2.59s	_{p=0.931 n=6}
_{Bind Time}	_{0.99s (± 0.52%)}	_{1.00s (± 0.84%)}	_~	_0.99s	_1.01s	_{p=0.070 n=6}
_{Check Time}	_{16.56s (± 0.39%)}	_{16.67s (± 0.61%)}	_~	_16.51s	_16.78s	_{p=0.065 n=6}
_{Emit Time}	_{0.00s (± 0.00%)}	_{0.00s (± 0.00%)}	_~	_0.00s	_0.00s	_{p=1.000 n=6}
_{Total Time}	_{20.13s (± 0.33%)}	_{20.23s (± 0.52%)}	_~	_20.08s	_20.36s	_{p=0.078 n=6}
_{xstate - node (v18.15.0, x64)}
_{Memory used}	_{512,638k (± 0.01%)}	_{512,630k (± 0.02%)}	_~	_512,533k	_512,730k	_{p=0.936 n=6}
_{Parse Time}	_{3.27s (± 0.45%)}	_{3.27s (± 0.36%)}	_~	_3.25s	_3.28s	_{p=1.000 n=6}
_{Bind Time}	_{1.55s (± 0.48%)}	_{1.55s (± 0.26%)}	_~	_1.54s	_1.55s	_{p=0.389 n=6}
_{Check Time}	_{2.87s (± 1.44%)}	_{2.86s (± 0.76%)}	_~	_2.84s	_2.90s	_{p=0.629 n=6}
_{Emit Time}	_{0.08s (± 4.99%)}	_{0.08s (± 0.00%)}	_~	_0.08s	_0.08s	_{p=0.405 n=6}
_{Total Time}	_{7.77s (± 0.47%)}	_{7.76s (± 0.28%)}	_~	_7.74s	_7.80s	_{p=0.258 n=6}

System info unknown

Hosts

node (v18.15.0, x64)

Scenarios

Angular - node (v18.15.0, x64)
Compiler-Unions - node (v18.15.0, x64)
Monaco - node (v18.15.0, x64)
TFS - node (v18.15.0, x64)
material-ui - node (v18.15.0, x64)
xstate - node (v18.15.0, x64)

_Benchmark	_Name	_Iterations
_Current	_pr	₆
_Baseline	_baseline	₆

tsserver

Comparison Report - baseline..pr

_Metric	_baseline	_pr	_Delta	_Best	_Worst	_p-value
_{Compiler-UnionsTSServer - node (v18.15.0, x64)}
_{Req 1 - updateOpen}	_{2,374ms (± 1.43%)}	_{2,351ms (± 0.86%)}	_~	_2,322ms	_2,381ms	_{p=0.336 n=6}
_{Req 2 - geterr}	_{5,364ms (± 1.52%)}	_{5,437ms (± 1.58%)}	_~	_5,344ms	_5,520ms	_{p=0.173 n=6}
_{Req 3 - references}	_{328ms (± 1.28%)}	_{328ms (± 1.33%)}	_~	_324ms	_335ms	_{p=1.000 n=6}
_{Req 4 - navto}	_{277ms (± 1.36%)}	_{277ms (± 0.81%)}	_~	_274ms	_279ms	_{p=0.746 n=6}
_{Req 5 - completionInfo count}	_{1,356 (± 0.00%)}	_{1,356 (± 0.00%)}	_~	_1,356	_1,356	_{p=1.000 n=6}
_{Req 5 - completionInfo}	_{84ms (± 8.24%)}	_{84ms (± 8.09%)}	_~	_75ms	_91ms	_{p=0.616 n=6}
_{CompilerTSServer - node (v18.15.0, x64)}
_{Req 1 - updateOpen}	_{2,474ms (± 1.37%)}	_{2,468ms (± 1.40%)}	_~	_2,402ms	_2,498ms	_{p=0.423 n=6}
_{Req 2 - geterr}	_{4,135ms (± 1.83%)}	_{4,158ms (± 2.02%)}	_~	_4,099ms	_4,272ms	_{p=0.378 n=6}
_{Req 3 - references}	_{338ms (± 1.62%)}	_{341ms (± 1.71%)}	_~	_333ms	_347ms	_{p=0.325 n=6}
_{Req 4 - navto}	_{283ms (± 0.63%)}	_{284ms (± 0.41%)}	_~	_282ms	_285ms	_{p=0.458 n=6}
_{Req 5 - completionInfo count}	_{1,518 (± 0.00%)}	_{1,518 (± 0.00%)}	_~	_1,518	_1,518	_{p=1.000 n=6}
_{Req 5 - completionInfo}	_{82ms (± 7.61%)}	_{84ms (± 7.48%)}	_~	_76ms	_90ms	_{p=0.498 n=6}
_{xstateTSServer - node (v18.15.0, x64)}
_{Req 1 - updateOpen}	_{2,590ms (± 0.41%)}	_{2,595ms (± 0.67%)}	_~	_2,572ms	_2,612ms	_{p=0.378 n=6}
_{Req 2 - geterr}	_{1,705ms (± 1.75%)}	_{1,679ms (± 1.07%)}	_~	_1,658ms	_1,709ms	_{p=0.128 n=6}
_{Req 3 - references}	_{109ms (± 7.48%)}	_{110ms (± 8.14%)}	_~	_105ms	_128ms	_{p=0.932 n=6}
_{Req 4 - navto}	_{359ms (± 0.18%)}	_{359ms (± 0.38%)}	_~	_357ms	_361ms	_{p=0.236 n=6}
_{Req 5 - completionInfo count}	_{2,073 (± 0.00%)}	_{2,073 (± 0.00%)}	_~	_2,073	_2,073	_{p=1.000 n=6}
_{Req 5 - completionInfo}	_{302ms (± 0.97%)}	_{307ms (± 0.56%)}	_{+5ms (+ 1.66%)}	_305ms	_309ms	_{p=0.014 n=6}

System info unknown

Hosts

node (v18.15.0, x64)

Scenarios

CompilerTSServer - node (v18.15.0, x64)
Compiler-UnionsTSServer - node (v18.15.0, x64)
xstateTSServer - node (v18.15.0, x64)

_Benchmark	_Name	_Iterations
_Current	_pr	₆
_Baseline	_baseline	₆

Startup

Comparison Report - baseline..pr

_Metric	_baseline	_pr	_Delta	_Best	_Worst	_p-value
_{tsc-startup - node (v18.15.0, x64)}
_{Execution time}	_{152.04ms (± 0.21%)}	_{152.04ms (± 0.18%)}	_~	_150.98ms	_155.38ms	_{p=0.877 n=600}
_{tsserver-startup - node (v18.15.0, x64)}
_{Execution time}	_{227.90ms (± 0.16%)}	_{227.73ms (± 0.19%)}	_{-0.17ms (- 0.07%)}	_226.40ms	_234.08ms	_{p=0.000 n=600}
_{tsserverlibrary-startup - node (v18.15.0, x64)}
_{Execution time}	_{228.85ms (± 0.15%)}	_{228.98ms (± 0.20%)}	_{+0.12ms (+ 0.05%)}	_227.22ms	_239.82ms	_{p=0.013 n=600}
_{typescript-startup - node (v18.15.0, x64)}
_{Execution time}	_{228.86ms (± 0.16%)}	_{228.85ms (± 0.16%)}	_~	_227.40ms	_232.04ms	_{p=0.642 n=600}

System info unknown

Hosts

node (v18.15.0, x64)

Scenarios

tsc-startup - node (v18.15.0, x64)
tsserver-startup - node (v18.15.0, x64)
tsserverlibrary-startup - node (v18.15.0, x64)
typescript-startup - node (v18.15.0, x64)

_Benchmark	_Name	_Iterations
_Current	_pr	₆
_Baseline	_baseline	₆

Developer Information:

Download Benchmarks

Jake Bailey (jakebailey) · 2023-10-12T21:57:55Z

Hm, not very good, probably because if we're already willing to pay the cost to do relations, it's not going to be very expensive to go through simpleTypeRelated or whatever. But in the other place, we're guaranteed to call relation 4 times when the types aren't related, which is expensive on large unions.

TypeScript Bot (typescript-bot) · 2023-10-12T23:31:06Z

Jake Bailey (@jakebailey) Here are the results of running the top-repos suite comparing main and refs/pull/56072/merge:

Everything looks good!

Jake Bailey (jakebailey) · 2024-01-19T21:01:15Z

TypeScript Bot (@typescript-bot) perf test this

TypeScript Bot (typescript-bot) · 2024-01-19T21:01:18Z

Heya Jake Bailey (@jakebailey), I've started to run the regular perf test suite on this PR at f993810. You can monitor the build here.

Update: The results are in!

TypeScript Bot (typescript-bot) · 2024-01-20T00:29:45Z

Jake Bailey (@jakebailey)
The results of the perf run you requested are in!

Here they are:

tsc

Comparison Report - baseline..pr

_Metric	_baseline	_pr	_Delta	_Best	_Worst	_p-value
_{Angular - node (v18.15.0, x64)}
_{Memory used}	_{295,651k (± 0.01%)}	_{295,652k (± 0.01%)}	_~	_295,626k	_295,680k	_{p=0.689 n=6}
_{Parse Time}	_{2.66s (± 0.00%)}	_{2.66s (± 0.24%)}	_~	_2.65s	_2.67s	_{p=1.000 n=6}
_{Bind Time}	_{0.83s (± 0.90%)}	_{0.83s (± 1.08%)}	_~	_0.82s	_0.84s	_{p=0.798 n=6}
_{Check Time}	_{8.19s (± 0.30%)}	_{8.19s (± 0.31%)}	_~	_8.16s	_8.22s	_{p=1.000 n=6}
_{Emit Time}	_{7.11s (± 0.31%)}	_{7.11s (± 0.23%)}	_~	_7.09s	_7.14s	_{p=1.000 n=6}
_{Total Time}	_{18.79s (± 0.05%)}	_{18.80s (± 0.20%)}	_~	_18.73s	_18.84s	_{p=0.164 n=6}
_{Compiler-Unions - node (v18.15.0, x64)}
_{Memory used}	_{192,535k (± 1.28%)}	_{192,979k (± 1.27%)}	_~	_191,508k	_197,462k	_{p=0.873 n=6}
_{Parse Time}	_{1.36s (± 0.72%)}	_{1.36s (± 0.46%)}	_~	_1.35s	_1.37s	_{p=1.000 n=6}
_{Bind Time}	_{0.72s (± 0.00%)}	_{0.72s (± 0.00%)}	_~	_0.72s	_0.72s	_{p=1.000 n=6}
_{Check Time}	_{9.36s (± 0.42%)}	_{9.35s (± 0.41%)}	_~	_9.31s	_9.40s	_{p=0.936 n=6}
_{Emit Time}	_{2.62s (± 0.59%)}	_{2.60s (± 1.36%)}	_~	_2.54s	_2.64s	_{p=0.225 n=6}
_{Total Time}	_{14.06s (± 0.22%)}	_{14.03s (± 0.36%)}	_~	_13.98s	_14.11s	_{p=0.296 n=6}
_{Monaco - node (v18.15.0, x64)}
_{Memory used}	_{347,462k (± 0.01%)}	_{347,459k (± 0.00%)}	_~	_347,450k	_347,476k	_{p=0.809 n=6}
_{Parse Time}	_{2.48s (± 0.47%)}	_{2.48s (± 0.61%)}	_~	_2.47s	_2.51s	_{p=1.000 n=6}
_{Bind Time}	_{0.93s (± 0.55%)}	_{0.93s (± 0.00%)}	_~	_0.93s	_0.93s	_{p=0.174 n=6}
_{Check Time}	_{6.90s (± 0.39%)}	_{6.89s (± 0.22%)}	_~	_6.87s	_6.91s	_{p=0.408 n=6}
_{Emit Time}	_{4.06s (± 0.62%)}	_{4.06s (± 0.19%)}	_~	_4.05s	_4.07s	_{p=1.000 n=6}
_{Total Time}	_{14.38s (± 0.32%)}	_{14.36s (± 0.13%)}	_~	_14.32s	_14.37s	_{p=0.290 n=6}
_{TFS - node (v18.15.0, x64)}
_{Memory used}	_{302,829k (± 0.01%)}	_{302,840k (± 0.01%)}	_~	_302,819k	_302,859k	_{p=0.298 n=6}
_{Parse Time}	_{2.01s (± 1.07%)}	_{2.00s (± 0.88%)}	_~	_1.99s	_2.04s	_{p=0.317 n=6}
_{Bind Time}	_{1.00s (± 0.81%)}	_{1.01s (± 1.20%)}	_~	_0.99s	_1.02s	_{p=0.652 n=6}
_{Check Time}	_{6.33s (± 0.12%)}	_{6.31s (± 0.23%)}	_{-0.02s (- 0.26%)}	_6.29s	_6.33s	_{p=0.046 n=6}
_{Emit Time}	_{3.59s (± 0.42%)}	_{3.59s (± 0.65%)}	_~	_3.56s	_3.62s	_{p=0.871 n=6}
_{Total Time}	_{12.94s (± 0.18%)}	_{12.91s (± 0.28%)}	_~	_12.88s	_12.97s	_{p=0.195 n=6}
_{material-ui - node (v18.15.0, x64)}
_{Memory used}	_{511,295k (± 0.00%)}	_{511,295k (± 0.01%)}	_~	_511,250k	_511,321k	_{p=0.688 n=6}
_{Parse Time}	_{2.66s (± 0.19%)}	_{2.65s (± 0.74%)}	_~	_2.62s	_2.67s	_{p=0.797 n=6}
_{Bind Time}	_{1.00s (± 1.37%)}	_{1.00s (± 0.54%)}	_~	_1.00s	_1.01s	_{p=0.154 n=6}
_{Check Time}	_{17.27s (± 0.36%)}	_{17.25s (± 0.31%)}	_~	_17.19s	_17.35s	_{p=0.573 n=6}
_{Emit Time}	_{0.00s (± 0.00%)}	_{0.00s (± 0.00%)}	_~	_0.00s	_0.00s	_{p=1.000 n=6}
_{Total Time}	_{20.93s (± 0.33%)}	_{20.91s (± 0.29%)}	_~	_20.82s	_21.01s	_{p=0.470 n=6}
_{mui-docs - node (v18.15.0, x64)}
_{Memory used}	_{1,695,944k (± 0.00%)}	_{1,695,962k (± 0.00%)}	_~	_1,695,913k	_1,696,005k	_{p=0.575 n=6}
_{Parse Time}	_{6.53s (± 0.40%)}	_{6.54s (± 0.60%)}	_~	_6.50s	_6.61s	_{p=0.747 n=6}
_{Bind Time}	_{2.35s (± 0.35%)}	_{2.34s (± 0.50%)}	_~	_2.33s	_2.36s	_{p=0.066 n=6}
_{Check Time}	_{55.93s (± 3.01%)}	_{55.37s (± 0.36%)}	_~	_55.16s	_55.69s	_{p=0.630 n=6}
_{Emit Time}	_{0.16s (± 0.00%)}	_{0.16s (± 0.00%)}	_~	_0.16s	_0.16s	_{p=1.000 n=6}
_{Total Time}	_{64.97s (± 2.61%)}	_{64.42s (± 0.37%)}	_~	_64.18s	_64.83s	_{p=0.748 n=6}
_{self-build-src - node (v18.15.0, x64)}
_{Memory used}	_{2,413,387k (± 0.03%)}	_{2,413,106k (± 0.02%)}	_~	_2,412,589k	_2,413,969k	_{p=0.298 n=6}
_{Parse Time}	_{4.95s (± 1.25%)}	_{4.96s (± 0.88%)}	_~	_4.92s	_5.02s	_{p=0.521 n=6}
_{Bind Time}	_{1.87s (± 0.53%)}	_{1.88s (± 0.52%)}	_~	_1.87s	_1.89s	_{p=0.151 n=6}
_{Check Time}	_{33.39s (± 0.17%)}	_{33.26s (± 0.18%)}	_{-0.13s (- 0.39%)}	_33.20s	_33.34s	_{p=0.013 n=6}
_{Emit Time}	_{2.66s (± 1.75%)}	_{2.73s (± 1.32%)}	_{+0.08s (+ 2.88%)}	_2.69s	_2.79s	_{p=0.025 n=6}
_{Total Time}	_{42.90s (± 0.29%)}	_{42.85s (± 0.10%)}	_~	_42.78s	_42.89s	_{p=0.230 n=6}
_{self-compiler - node (v18.15.0, x64)}
_{Memory used}	_{419,682k (± 0.01%)}	_{419,744k (± 0.01%)}	_{+61k (+ 0.01%)}	_419,693k	_419,776k	_{p=0.031 n=6}
_{Parse Time}	_{2.77s (± 3.57%)}	_{2.75s (± 2.90%)}	_~	_2.68s	_2.85s	_{p=1.000 n=6}
_{Bind Time}	_{1.12s (± 6.38%)}	_{1.14s (± 6.74%)}	_~	_1.07s	_1.22s	_{p=0.864 n=6}
_{Check Time}	_{15.09s (± 0.27%)}	_{15.09s (± 0.41%)}	_~	_15.01s	_15.19s	_{p=1.000 n=6}
_{Emit Time}	_{1.17s (± 1.37%)}	_{1.16s (± 2.08%)}	_~	_1.12s	_1.19s	_{p=0.805 n=6}
_{Total Time}	_{20.14s (± 0.24%)}	_{20.14s (± 0.33%)}	_~	_20.03s	_20.23s	_{p=1.000 n=6}
_{vscode - node (v18.15.0, x64)}
_{Memory used}	_{2,807,322k (± 0.00%)}	_{2,807,291k (± 0.00%)}	_~	_2,807,192k	_2,807,361k	_{p=0.689 n=6}
_{Parse Time}	_{10.63s (± 0.44%)}	_{10.63s (± 0.40%)}	_~	_10.59s	_10.69s	_{p=1.000 n=6}
_{Bind Time}	_{3.38s (± 0.22%)}	_{3.40s (± 0.71%)}	_~	_3.36s	_3.43s	_{p=0.118 n=6}
_{Check Time}	_{59.72s (± 0.23%)}	_{59.77s (± 0.44%)}	_~	_59.47s	_60.10s	_{p=1.000 n=6}
_{Emit Time}	_{16.17s (± 0.49%)}	_{16.19s (± 0.34%)}	_~	_16.12s	_16.26s	_{p=0.689 n=6}
_{Total Time}	_{89.90s (± 0.07%)}	_{89.99s (± 0.24%)}	_~	_89.77s	_90.33s	_{p=0.810 n=6}
_{webpack - node (v18.15.0, x64)}
_{Memory used}	_{392,386k (± 0.02%)}	_{392,418k (± 0.03%)}	_~	_392,307k	_392,614k	_{p=0.575 n=6}
_{Parse Time}	_{3.05s (± 0.72%)}	_{3.06s (± 0.60%)}	_~	_3.04s	_3.08s	_{p=0.511 n=6}
_{Bind Time}	_{1.40s (± 0.45%)}	_{1.41s (± 0.39%)}	_~	_1.40s	_1.41s	_{p=0.201 n=6}
_{Check Time}	_{14.01s (± 0.40%)}	_{14.01s (± 0.46%)}	_~	_13.95s	_14.13s	_{p=0.936 n=6}
_{Emit Time}	_{0.00s (±244.70%)}	_{0.00s (± 0.00%)}	_~	_0.00s	_0.00s	_{p=0.405 n=6}
_{Total Time}	_{18.46s (± 0.36%)}	_{18.47s (± 0.34%)}	_~	_18.44s	_18.60s	_{p=0.868 n=6}
_{xstate - node (v18.15.0, x64)}
_{Memory used}	_{513,429k (± 0.02%)}	_{513,432k (± 0.01%)}	_~	_513,337k	_513,494k	_{p=0.471 n=6}
_{Parse Time}	_{3.28s (± 0.16%)}	_{3.28s (± 0.45%)}	_~	_3.26s	_3.30s	_{p=1.000 n=6}
_{Bind Time}	_{1.54s (± 0.35%)}	_{1.54s (± 0.27%)}	_~	_1.53s	_1.54s	_{p=0.054 n=6}
_{Check Time}	_{2.85s (± 0.52%)}	_{2.85s (± 0.75%)}	_~	_2.83s	_2.89s	_{p=0.683 n=6}
_{Emit Time}	_{0.08s (± 0.00%)}	_{0.08s (± 0.00%)}	_~	_0.08s	_0.08s	_{p=1.000 n=6}
_{Total Time}	_{7.76s (± 0.30%)}	_{7.75s (± 0.19%)}	_~	_7.73s	_7.77s	_{p=0.460 n=6}

System info unknown

Hosts

node (v18.15.0, x64)

Scenarios

Angular - node (v18.15.0, x64)
Compiler-Unions - node (v18.15.0, x64)
Monaco - node (v18.15.0, x64)
TFS - node (v18.15.0, x64)
material-ui - node (v18.15.0, x64)
mui-docs - node (v18.15.0, x64)
self-build-src - node (v18.15.0, x64)
self-compiler - node (v18.15.0, x64)
vscode - node (v18.15.0, x64)
webpack - node (v18.15.0, x64)
xstate - node (v18.15.0, x64)

_Benchmark	_Name	_Iterations
_Current	_pr	₆
_Baseline	_baseline	₆

tsserver

Comparison Report - baseline..pr

_Metric	_baseline	_pr	_Delta	_Best	_Worst	_p-value
_{Compiler-UnionsTSServer - node (v18.15.0, x64)}
_{Req 1 - updateOpen}	_{2,342ms (± 0.70%)}	_{2,353ms (± 0.89%)}	_~	_2,318ms	_2,372ms	_{p=0.297 n=6}
_{Req 2 - geterr}	_{5,529ms (± 1.55%)}	_{5,529ms (± 1.61%)}	_~	_5,430ms	_5,633ms	_{p=0.810 n=6}
_{Req 3 - references}	_{323ms (± 0.36%)}	_{325ms (± 1.13%)}	_~	_320ms	_330ms	_{p=0.121 n=6}
_{Req 4 - navto}	_{274ms (± 1.24%)}	_{273ms (± 1.38%)}	_~	_270ms	_278ms	_{p=0.935 n=6}
_{Req 5 - completionInfo count}	_{1,356 (± 0.00%)}	_{1,356 (± 0.00%)}	_~	_1,356	_1,356	_{p=1.000 n=6}
_{Req 5 - completionInfo}	_{88ms (± 7.36%)}	_{90ms (± 7.31%)}	_~	_79ms	_95ms	_{p=0.743 n=6}
_{CompilerTSServer - node (v18.15.0, x64)}
_{Req 1 - updateOpen}	_{2,487ms (± 0.69%)}	_{2,488ms (± 0.86%)}	_~	_2,460ms	_2,509ms	_{p=0.872 n=6}
_{Req 2 - geterr}	_{4,212ms (± 1.70%)}	_{4,208ms (± 1.91%)}	_~	_4,124ms	_4,288ms	_{p=0.378 n=6}
_{Req 3 - references}	_{331ms (± 0.80%)}	_{337ms (± 1.69%)}	_{+6ms (+ 1.81%)}	_331ms	_343ms	_{p=0.044 n=6}
_{Req 4 - navto}	_{285ms (± 1.28%)}	_{284ms (± 0.26%)}	_~	_283ms	_285ms	_{p=1.000 n=6}
_{Req 5 - completionInfo count}	_{1,518 (± 0.00%)}	_{1,518 (± 0.00%)}	_~	_1,518	_1,518	_{p=1.000 n=6}
_{Req 5 - completionInfo}	_{77ms (± 3.34%)}	_{84ms (± 7.85%)}	_~	_77ms	_90ms	_{p=0.105 n=6}
_{xstateTSServer - node (v18.15.0, x64)}
_{Req 1 - updateOpen}	_{2,601ms (± 0.29%)}	_{2,604ms (± 0.59%)}	_~	_2,584ms	_2,626ms	_{p=0.688 n=6}
_{Req 2 - geterr}	_{1,704ms (± 2.75%)}	_{1,705ms (± 2.19%)}	_~	_1,669ms	_1,741ms	_{p=0.688 n=6}
_{Req 3 - references}	_{124ms (± 6.28%)}	_{124ms (± 5.47%)}	_~	_110ms	_128ms	_{p=0.565 n=6}
_{Req 4 - navto}	_{370ms (± 0.22%)}	_{371ms (± 0.33%)}	_~	_370ms	_373ms	_{p=0.126 n=6}
_{Req 5 - completionInfo count}	_{2,078 (± 0.00%)}	_{2,078 (± 0.00%)}	_~	_2,078	_2,078	_{p=1.000 n=6}
_{Req 5 - completionInfo}	_{308ms (± 1.73%)}	_{308ms (± 2.38%)}	_~	_297ms	_317ms	_{p=0.936 n=6}

System info unknown

Hosts

node (v18.15.0, x64)

Scenarios

CompilerTSServer - node (v18.15.0, x64)
Compiler-UnionsTSServer - node (v18.15.0, x64)
xstateTSServer - node (v18.15.0, x64)

_Benchmark	_Name	_Iterations
_Current	_pr	₆
_Baseline	_baseline	₆

startup

Comparison Report - baseline..pr

_Metric	_baseline	_pr	_Delta	_Best	_Worst	_p-value
_{tsc-startup - node (v18.15.0, x64)}
_{Execution time}	_{153.87ms (± 0.22%)}	_{153.95ms (± 0.22%)}	_{+0.09ms (+ 0.06%)}	_152.60ms	_160.27ms	_{p=0.001 n=600}
_{tsserver-startup - node (v18.15.0, x64)}
_{Execution time}	_{229.94ms (± 0.16%)}	_{229.78ms (± 0.17%)}	_{-0.16ms (- 0.07%)}	_228.53ms	_236.76ms	_{p=0.000 n=600}
_{tsserverlibrary-startup - node (v18.15.0, x64)}
_{Execution time}	_{231.55ms (± 0.19%)}	_{231.46ms (± 0.20%)}	_{-0.09ms (- 0.04%)}	_229.88ms	_237.82ms	_{p=0.033 n=600}
_{typescript-startup - node (v18.15.0, x64)}
_{Execution time}	_{231.18ms (± 0.19%)}	_{231.30ms (± 0.18%)}	_{+0.12ms (+ 0.05%)}	_229.55ms	_234.45ms	_{p=0.005 n=600}

System info unknown

Hosts

node (v18.15.0, x64)

Scenarios

tsc-startup - node (v18.15.0, x64)
tsserver-startup - node (v18.15.0, x64)
tsserverlibrary-startup - node (v18.15.0, x64)
typescript-startup - node (v18.15.0, x64)

_Benchmark	_Name	_Iterations
_Current	_pr	₆
_Baseline	_baseline	₆

Developer Information:

Download Benchmarks

Nathan Shively-Sanders (sandersn) · 2025-04-03T13:28:31Z

Jake Bailey (@jakebailey) did the results indicate that this is worth pursuing? It is worth keeping open?
A quick glance at the latest perf run didn't show anything substantial.

Jake Bailey (jakebailey) · 2025-04-03T15:13:55Z

You can see the relevant benchmark in the original post; our current test suite doesn't have a case like this.

Jake Bailey (jakebailey) added 2 commits October 11, 2023 08:53

Optimize getNarrowedTypeWorker for literals

b19c64d

revert unrelated

54d0bc7

TypeScript Bot (typescript-bot) assigned Jake Bailey (jakebailey) Oct 11, 2023

TypeScript Bot (typescript-bot) added Author: Team For Uncommitted Bug PR for untriaged, rejected, closed or missing bug labels Oct 11, 2023

Jake Bailey (jakebailey) mentioned this pull request Oct 11, 2023

Performance regression from #49625 and #52984 #55948

Open

Fixup

629de0d

Wesley Wigham (weswigham) reviewed Oct 12, 2023

View reviewed changes

Comment thread src/compiler/checker.ts Outdated

Jake Bailey (jakebailey) added 2 commits October 12, 2023 13:59

Fix bools, leave todo

53f60d1

Try out new thing for all relations

e68e177

Revert change

247724d

Merge branch 'main' into fix-55948

f993810

Jake Bailey (jakebailey) mentioned this pull request Jul 1, 2024

[experiment] trying to short circuit relations in more places #59086

Closed

Jake Bailey (jakebailey) closed this Dec 11, 2025

Microsoft (microsoft) locked as resolved and limited conversation to collaborators Jun 10, 2026

Jake Bailey (jakebailey) deleted the fix-55948 branch August 23, 2026 01:57

Conversation

Jake Bailey (jakebailey) commented Oct 11, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Jake Bailey (jakebailey) commented Oct 11, 2023

Uh oh!

TypeScript Bot (typescript-bot) commented Oct 11, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

TypeScript Bot (typescript-bot) commented Oct 11, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

TypeScript Bot (typescript-bot) commented Oct 11, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

TypeScript Bot (typescript-bot) commented Oct 11, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

TypeScript Bot (typescript-bot) commented Oct 11, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

TypeScript Bot (typescript-bot) commented Oct 11, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

TypeScript Bot (typescript-bot) commented Oct 11, 2023

Uh oh!

TypeScript Bot (typescript-bot) commented Oct 11, 2023

Compiler

tsserver

Startup

Uh oh!

TypeScript Bot (typescript-bot) commented Oct 11, 2023

Uh oh!

TypeScript Bot (typescript-bot) commented Oct 11, 2023

Uh oh!

Jake Bailey (jakebailey) commented Oct 12, 2023

Uh oh!

TypeScript Bot (typescript-bot) commented Oct 12, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

TypeScript Bot (typescript-bot) commented Oct 12, 2023

Compiler

tsserver

Startup

Uh oh!

Uh oh!

Jake Bailey (jakebailey) commented Oct 12, 2023

Uh oh!

TypeScript Bot (typescript-bot) commented Oct 12, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

TypeScript Bot (typescript-bot) commented Oct 12, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

TypeScript Bot (typescript-bot) commented Oct 12, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

TypeScript Bot (typescript-bot) commented Oct 12, 2023

Uh oh!

TypeScript Bot (typescript-bot) commented Oct 12, 2023

Compiler

tsserver

Startup

Uh oh!

Jake Bailey (jakebailey) commented Oct 12, 2023

Uh oh!

TypeScript Bot (typescript-bot) commented Oct 12, 2023

Uh oh!

Jake Bailey (jakebailey) commented Jan 19, 2024

Uh oh!

TypeScript Bot (typescript-bot) commented Jan 19, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

TypeScript Bot (typescript-bot) commented Jan 20, 2024

tsc

tsserver

startup

Uh oh!

Nathan Shively-Sanders (sandersn) commented Apr 3, 2025

Uh oh!

Jake Bailey (jakebailey) commented Oct 11, 2023 •

edited

Loading

TypeScript Bot (typescript-bot) commented Oct 11, 2023 •

edited

Loading

TypeScript Bot (typescript-bot) commented Oct 11, 2023 •

edited

Loading

TypeScript Bot (typescript-bot) commented Oct 11, 2023 •

edited

Loading

TypeScript Bot (typescript-bot) commented Oct 11, 2023 •

edited

Loading

TypeScript Bot (typescript-bot) commented Oct 11, 2023 •

edited

Loading

TypeScript Bot (typescript-bot) commented Oct 11, 2023 •

edited

Loading

TypeScript Bot (typescript-bot) commented Oct 12, 2023 •

edited

Loading

TypeScript Bot (typescript-bot) commented Oct 12, 2023 •

edited

Loading

TypeScript Bot (typescript-bot) commented Oct 12, 2023 •

edited

Loading

TypeScript Bot (typescript-bot) commented Oct 12, 2023 •

edited

Loading

TypeScript Bot (typescript-bot) commented Jan 19, 2024 •

edited

Loading