vm.compileFunction(source) is much slower than new vm.Script(source) #35375

I added a quick and dirty produceCachedData, and it went down to about 180 ms. Which is a great improvement, albeit still 5x slower than new Script. And harder to manage (I'd prefer the implicit caching of the new Script APIs)

cc @nodejs/v8, this might not be something fixable on node's side.

This is about v8::ScriptCompiler::CompileFunctionInContext() not using the code cache, right?

I don't think that's V8's issue to fix. V8 can't know whether it's safe to use the cache because the argument names and context extensions affect how the function is parsed.

Not that it's easy for Node.js. options.contextExtensions is a free-form list of mutable objects. Hard to cache on that unless object identity is good enough - but it probably isn't.

What Node.js (or perhaps V8) could optimize for is the presumably common case of no context extensions. TBD how to cache without introducing memory leaks.

In Jest's case we'll be passing a different parsingContext all the time (Jest uses a different Context per test file for sandboxing), so if the cached data is dependent on the Context we'd probably still get a painful performance regression vs using vm.Script. If it's not possible to get caching based on the source code string (and the arguments) alone, we'll probably have to drop usage of compileFunction and stay on vm.Script.

Using the args array to vary the cache is no issue for our use case, though - 99% of the time they'll be the same, and for the cases it's not I guess it'd just create some more? Same as we have today with the manual function(some, args) {USER_SOURCE_CODE_HERE} wrapper we add which changes its source code via the arguments we pass in the manual function wrapper.

(That said, improving performance for compileFunction in the case of not passing parsingContext seems like a good thing even if it wouldn't be enough for Jest's use case 🙂)

If it's not possible to get caching based on the source code string (and the arguments) alone

I don't think that can work so you're probably better off sticking with vm.Script, yes. Argument names aren't insurmountable but context extensions are.

When you pass in context extensions, what's evaluated looks like this in pseudo code:

// assume a = { y: 42 } and b = { z: 1337 }
with (a)
  with (b)
    function f(x) {
      return x * y * z
    }

I say "pseudo code" but in fact it's really close to the runtime representation.

@bnoordhuis reading your comment more closely, I see you're talking about contextExtensions. We don't use that, we use parsingContext - does that have the same limitations you think?

I think this issue should be upstreamed.

Fwiw, Node 16 has both Script and compileFunction performing similarly:

rishi vm-script-vs-compile-function % nvm use 14
Now using node v14.18.2 (npm v6.14.15)

rishi vm-script-vs-compile-function % node index.js
Running with Script took 51 ms
Running with compileFunction took 4379 ms
Running with compileFunction and cached data took 264 ms

rishi vm-script-vs-compile-function % nvm use 16
Now using node v16.13.1 (npm v8.1.2)

rishi vm-script-vs-compile-function % node index.js
Running with Script took 4280 ms
Running with compileFunction took 4350 ms
Running with compileFunction and cached data took 321 ms

yes... in more recent versions of v8 the compilation cache is effectively broken. see v8:10284

Just as a follow up here, these are the numbers I'm seeing with the latest 20.8 (nothing new, just posting to say it's still an issue).

$ nvm run 16.10 index.js
Running node v16.10.0 (npm v7.24.0)
Running with Script took 19 ms
Running with compileFunction took 1799 ms
Running with compileFunction and cached data took 145 ms

$ nvm run 16.11 index.js
Running node v16.11.1 (npm v8.0.0)
Running with Script took 1790 ms
Running with compileFunction took 1798 ms
Running with compileFunction and cached data took 147 ms

$ nvm run 20 index.js
Running node v20.8.0 (npm v10.1.0)
Running with Script took 1804 ms
Running with compileFunction took 1826 ms
Running with compileFunction and cached data took 141 ms

I think we can at least not have any host-defined options at all when the importModuleDynamically isn't used. That should make it possible for the cache to get hit again in that case at least.

The performance hit would comeback again if you intent to support import() though, because until https://bugs.chromium.org/p/v8/issues/detail?id=10284 gets fixed, the host defined options is going to be serialized as part of the script, and since you might still want different host-defined options for different script even if they have the same source, V8 needs to reject the cache. The fix would be to either move the host defined options to somewhere more sensible (as proposed by the V8 issue initially), or as later comments in that issue suggested, just don't serialize it and provide some kind of callback for the embedder to compute the host-defined options (personally I'd prefer that, I think that provides more flexibility).

Ooh, any workaround, even with caveats, would be awesome! 😃

The performance hit would comeback again if you intent to support import()

We do support that, but I believe that only happens if vm Modules are active (which is behind a flag)? If it's not behind a node flag, I'm happy to put it behind a Jest flag or some such so people who don't need it won't get the perf hit.

I opened #49950 to address the "no import() needed" case which is a fairly simple change and can backport to earlier release lines (v20.x at least, not 100% sure about v18.x yet). Locally the in-isolate cache is getting hit again (I used the repro from OP but switched the script being compiled to test/fixtures/snapshot/typescript.js which is also big).

❯ ./node_main bench.js
Running with Script took 5428 ms
Running with compileFunction took 5332 ms
Running with compileFunction and cached data took 1010 ms

❯ ./node_pr bench.js
Running with Script took 56 ms
Running with compileFunction took 5369 ms
Running with compileFunction and cached data took 1013 ms

This issue has been marked as stale due to 210 days of inactivity.
It will be automatically closed in 30 days if no further activity occurs. If this is still relevant, please leave a comment or update it to keep it open.

Still relevant I would think, but possibly better tracked in #62720? Seeing as the issues have been fixed EXCEPT for when using ESM (IIRC)

Yes, I am still consolidating the new design but on a high level, it should stick to "one host-defined option for one loader", therefore all ESM compiled using a loader should hit the cache when compiled again using the same loader (and in the new design, the loader should be an instance of a class that stays constant for a context, not different closures that will keep changing for each compiled unit). So moving to #62720 for now.