node >=17: experimental esm loader doesnt respect --experimental-specifier-resolution=node

Version

17.3.0

Platform

Linux spaceship 5.13.0-22-generic #22-Ubuntu SMP Fri Nov 5 13:21:36 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux

Subsystem

loaders, experimental node options

What steps will reproduce the bug?

create an esm loader for .mjs files
create two files, main.mjs which imports somefile.mjs
run the main.mjs with experimental-specifier-resolution=node via node 17.3.0

esm loader works under the following conditions

import poop from './somefile.mjs'
import poop from '@someNamespace/somePackage''

esm loader fails under the following conditions

import poop from './somefile'

How often does it reproduce? Is there a required condition?

everyday, except sundays. thats the lords day

What is the expected behavior?

i can use experimental loader + experimental specifier resolution because Ive spent many days reading the nodejs docs, and the nodejs docs never lie

What do you see instead?

poop@spaceship:~/git/foss/nodeproto/packages/libraries/testproto
06:57 AM (test-everything *$=) 
$ pnpm repo:test

> @nodeproto/testproto@0.0.0 repo:test /home/poop/git/foss/nodeproto/packages/libraries/testproto
> NODE_OPTIONS="$npm_package_config_NODE_OPTIONS" uvu ./src "test.(m|c)?js"

(node:1377741) ExperimentalWarning: --experimental-loader is an experimental feature. This feature could change at any time
(Use `node --trace-warnings ...` to show where the warning was created)
TypeError [ERR_INVALID_FILE_URL_HOST]: File URL host must be "localhost" or empty on linux
    at statSync (node:fs:1531:10)
    at fileExists (node:internal/modules/esm/resolve:260:10)
    at resolveExtensions (node:internal/modules/esm/resolve:335:9)
    at resolveExtensionsWithTryExactName (node:internal/modules/esm/resolve:322:10)
    at finalizeResolution (node:internal/modules/esm/resolve:375:16)
    at moduleResolve (node:internal/modules/esm/resolve:915:10)
    at defaultResolve (node:internal/modules/esm/resolve:1005:11)
    at ESMLoader.resolve (node:internal/modules/esm/loader:530:30)
    at ESMLoader.getModuleJob (node:internal/modules/esm/loader:251:18)
    at ModuleWrap.<anonymous> (node:internal/modules/esm/module_job:79:40)
 ELIFECYCLE  Command failed with exit code 1.

poop@spaceship:~/git/foss/nodeproto/packages/libraries/testproto
06:57 AM (test-everything *$=) 
$

Additional information

06:57 AM (test-everything *$=) 
$ pnpm repo:about

> @nodeproto/testproto@0.0.0 repo:about /home/poop/git/foss/nodeproto/packages/libraries/testproto
> npm run-script && npm --versions && pnpm c list && echo "NODE OPTIONS set to: $npm_package_config_NODE_OPTIONS"

npm info using npm@8.3.0
npm info using node@v17.3.0
npm timing npm:load:whichnode Completed in 0ms
npm timing config:load:defaults Completed in 1ms
npm timing config:load:file:/opt/nodejs/node-v17.3.0-linux-x64/lib/node_modules/npm/npmrc Completed in 1ms
npm timing config:load:builtin Completed in 1ms
npm timing config:load:cli Completed in 1ms
npm timing config:load:env Completed in 1ms
npm timing config:load:file:/home/poop/git/foss/nodeproto/packages/libraries/testproto/.npmrc Completed in 0ms
npm timing config:load:project Completed in 1ms
npm timing config:load:file:/home/poop/.npmrc Completed in 0ms
npm timing config:load:user Completed in 1ms
npm timing config:load:file:/opt/nodejs/node-v17.3.0-linux-x64/etc/npmrc Completed in 0ms
npm timing config:load:global Completed in 0ms
npm timing config:load:validate Completed in 1ms
npm timing config:load:credentials Completed in 1ms
npm timing config:load:setEnvs Completed in 1ms
npm timing config:load Completed in 9ms
npm timing npm:load:configload Completed in 9ms
npm timing npm:load:setTitle Completed in 1ms
npm timing config:load:flatten Completed in 2ms
npm timing npm:load:display Completed in 6ms
npm timing npm:load:logFile Completed in 4ms
npm timing npm:load:timers Completed in 0ms
npm timing npm:load:configScope Completed in 0ms
npm timing npm:load Completed in 21ms
Scripts available in @nodeproto/testproto@0.0.0 via `npm run-script`:
  build
    pnpm repo:rm:dist && pnpm repo:cp:cjs && swc $npm_package_config_DIRS_TO_TRANSPILE -d $npm_package_config_PATH_DIST --config-file ./node_modules/@nodeproto/configproto/src/swc/node.swcrc --copy-files
  proto:bin
    pnpm -r exec $*
  proto:script
    pnpm -r $*
  proto:slice
    NODE_OPTIONS="$npm_package_config_NODE_OPTIONS" pnpm exec ultra -r --filter
  proto
    pnpm exec ultra -r
  repo:about
    npm run-script && npm --versions && pnpm c list && echo "NODE OPTIONS set to: $npm_package_config_NODE_OPTIONS"
  repo:cp:browserslist
    rm -f .browserslistrc && cp ./node_modules/@nodeproto/configproto/.browserslistrc .
  repo:cp:cjs
    rm -f ./dist/package.json && mkdir -p dist && cp ./node_modules/@nodeproto/configproto/src/node/cjs.json ./dist/package.json
  repo:cp:configproto
    pnpm repo:cp:flow; pnpm repo:cp:cjs; pnpm repo:cp:browserslist
  repo:cp:flow
    rm -f .flowconfig && cp ./node_modules/@nodeproto/configproto/.flowconfig .
  repo:deps
    ultra --info
  repo:eslint:fix
    pnpm repo:eslint -- --fix
  repo:eslint
    eslint './**/*'
  repo:flowtyped:install
    flow-typed install
  repo:nuke
    pnpm repo:rm:dist; pnpm repo:rm:nodemodules
  repo:rm:dist
    rm -rf dist/*
  repo:rm:nodemodules
    rm -rf node_modules/*
  repo:scripts:v
    npm run-script
  repo:scripts
    ultra --list
  repo:test:file
    node --experimental-specifier-resolution=node --experimental-import-meta-resolve $*
  repo:test:help
    uvu --help
  repo:test
    NODE_OPTIONS="$npm_package_config_NODE_OPTIONS" uvu ./src "test.(m|c)?js"
  repo:testing
    pnpm repo:test; chokidar "src/**/*(?:.test)?.(m|c)?js" -c "pnpm repo:test"

npm timing command:run-script Completed in 9ms
npm timing npm Completed in 117ms
npm info ok 
npm info using npm@8.3.0
npm info using node@v17.3.0
npm timing npm:load:whichnode Completed in 0ms
npm timing config:load:defaults Completed in 1ms
npm timing config:load:file:/opt/nodejs/node-v17.3.0-linux-x64/lib/node_modules/npm/npmrc Completed in 0ms
npm timing config:load:builtin Completed in 1ms
npm timing config:load:cli Completed in 1ms
npm timing config:load:env Completed in 1ms
npm timing config:load:file:/home/poop/git/foss/nodeproto/packages/libraries/testproto/.npmrc Completed in 0ms
npm timing config:load:project Completed in 2ms
npm timing config:load:file:/home/poop/.npmrc Completed in 0ms
npm timing config:load:user Completed in 0ms
npm timing config:load:file:/opt/nodejs/node-v17.3.0-linux-x64/etc/npmrc Completed in 0ms
npm timing config:load:global Completed in 0ms
npm timing config:load:validate Completed in 1ms
npm timing config:load:credentials Completed in 1ms
npm timing config:load:setEnvs Completed in 1ms
npm timing config:load Completed in 9ms
npm timing npm:load:configload Completed in 9ms
npm timing npm:load:setTitle Completed in 0ms
npm timing config:load:flatten Completed in 3ms
npm timing npm:load:display Completed in 7ms
npm timing npm:load:logFile Completed in 4ms
npm timing npm:load:timers Completed in 0ms
npm timing npm:load:configScope Completed in 0ms
npm timing npm:load Completed in 22ms
npm timing config:load:flatten Completed in 1ms
{
  '@nodeproto/testproto': '0.0.0',
  npm: '8.3.0',
  node: '17.3.0',
  v8: '9.6.180.15-node.12',
  uv: '1.42.0',
  zlib: '1.2.11',
  brotli: '1.0.9',
  ares: '1.18.1',
  modules: '102',
  nghttp2: '1.45.1',
  napi: '8',
  llhttp: '6.0.4',
  openssl: '3.0.1+quic',
  cldr: '40.0',
  icu: '70.1',
  tz: '2021a3',
  unicode: '14.0',
  ngtcp2: '0.1.0-DEV',
  nghttp3: '0.1.0-DEV'
}
npm timing command:version Completed in 8ms
npm timing npm Completed in 116ms
npm info ok 
npm info using npm@8.3.0
npm info using node@v17.3.0
npm timing npm:load:whichnode Completed in 0ms
npm timing config:load:defaults Completed in 1ms
npm timing config:load:file:/opt/nodejs/node-v17.3.0-linux-x64/lib/node_modules/npm/npmrc Completed in 1ms
npm timing config:load:builtin Completed in 1ms
npm timing config:load:cli Completed in 1ms
npm timing config:load:env Completed in 1ms
npm timing config:load:file:/home/poop/git/foss/nodeproto/packages/libraries/testproto/.npmrc Completed in 0ms
npm timing config:load:project Completed in 1ms
npm timing config:load:file:/home/poop/.npmrc Completed in 0ms
npm timing config:load:user Completed in 0ms
npm timing config:load:file:/opt/nodejs/node-v17.3.0-linux-x64/etc/npmrc Completed in 0ms
npm timing config:load:global Completed in 1ms
npm timing config:load:validate Completed in 1ms
npm timing config:load:credentials Completed in 1ms
npm timing config:load:setEnvs Completed in 1ms
npm timing config:load Completed in 9ms
npm timing npm:load:configload Completed in 9ms
npm timing npm:load:setTitle Completed in 0ms
npm timing config:load:flatten Completed in 2ms
npm timing npm:load:display Completed in 7ms
npm timing npm:load:logFile Completed in 4ms
npm timing npm:load:timers Completed in 0ms
npm timing npm:load:configScope Completed in 0ms
npm timing npm:load Completed in 21ms
; "env" config from environment

_noahehall-registry = (protected) 
argv = "{\"remain\":[\"run\",\"repo:about\"],\"cooked\":[\"run\",\"repo:about\"],\"original\":[\"run\",\"repo:about\"]}" 
cache-dir = "/var/.nodeproto/pnpm" 
child-concurrency = "2" 
color = true 
enable-modules-dir = true 
fetch-retry-maxtimeout = 60000 
fetch-retry-mintimeout = 10000 
fetch-timeout = 60000 
global-bin-dir = "/var/.nodeproto/pnpm-global/bin" 
global-dir = "/var/.nodeproto/pnpm-global" 
hoist = true 
hoist-pattern = "['*']" 
link-workspace-packages = true 
lockfile = true 
loglevel = "info" 
modules-cache-max-age = "2880" 
modules-dir = "node_modules" 
node-gyp = "/opt/nodejs/node-v17.3.0-linux-x64/pnpm-global/5/node_modules/.pnpm/pnpm@6.24.2/node_modules/pnpm/dist/node_modules/node-gyp/bin/node-gyp.js" 
package-import-method = "copy" 
prefer-frozen-lockfile = true 
prefer-offline = true 
prefer-workspace-packages = true 
recursive-install = true 
registry = "https://registry.npmjs.org/" 
save-exact = true 
save-prefix = "" 
shared-workspace-lockfile = true 
state-dir = "/var/.nodeproto/pnpm" 
store-dir = "/var/.nodeproto/.pnpm-store" 
symlink = true 
tag = "latest" 
use-beta-cli = true 
user-agent = "pnpm/6.24.2 npm/? node/v17.3.0 linux x64" 
virtual-store-dir = "node_modules/.pnpm" 

; node bin location = /opt/nodejs/node-v17.3.0-linux-x64/bin/node
; cwd = /home/poop/git/foss/nodeproto/packages/libraries/testproto
; HOME = /home/poop
; Run `npm config ls -l` to show all defaults.
npm timing command:c Completed in 7ms
npm timing npm Completed in 113ms
npm info ok 
NODE OPTIONS set to: --experimental-loader="./node_modules/@nodeproto/configproto/src/node/loaders/flow.mjs" --experimental-specifier-resolution=node --experimental-json-modules --experimental-top-level-await --experimental-vm-modules --experimental-import-meta-resolve node --enable-source-maps --heapsnapshot-near-heap-limit=3 --use-largepages=on --pending-deprecation --trace-sigint --trace-warnings --trace-exit --report-uncaught-exception --report-on-fatalerror --abort-on-uncaught-exception

poop@spaceship:~/git/foss/nodeproto/packages/libraries/testproto
06:57 AM (test-everything *$=)

use fully qualified name until nodejs/node#41321

not sure if this matters, but here is the relevant section of my pkg json

"exports": {
    ".": {
      "import": "./src/index.mjs",
      "default": "./dist/index.js"
    },
    "./t": {
      "import": "./src/t.mjs",
      "default": "./dist/t.js"
    }
  },

this is the line in the esm loader that throws the err

if (!isFlow) return isCjs 
  ? { format, source: rawSource } 
   : defaultLoad(url, { format });  <--- this procedure

/cc @nodejs/loaders

Need loader source text

…

On Sat, Dec 25, 2021, 11:32 AM Antoine du Hamel ***@***.***> wrote: /cc @nodejs/loaders <https://github.com/orgs/nodejs/teams/loaders> — Reply to this email directly, view it on GitHub <#41321 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AABZJIYAYZOYIERHGF6KYCTUSX5ZLANCNFSM5KXX5NKA> . Triage notifications on the go with GitHub Mobile for iOS <https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675> or Android <https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>. You are receiving this because you are on a team that was mentioned.Message ID: ***@***.***>

@bmeck

@bmeck loader source text

use fully qualified name until nodejs/node#41321

still broken in 17.4

in /src/t.mjs it exports a number of things
in /src/t.test.mjs it imports all things under the name space t

import * as t from ./t fails
import * as t from ./t.mjs works
import * as t from @nodeproto/testproto works

console log

poop@spaceship:~/git/foss/nodeproto/packages/libraries/testproto
10:48 AM (rework-testproto *$=) 
$ pnpm repo:test

> @nodeproto/testproto@0.0.0 repo:test /home/poop/git/foss/nodeproto/packages/libraries/testproto
> NODE_OPTIONS="$npm_package_config_NODE_OPTIONS" uvu ./src "test.(m|c)?js"

(node:614409) ExperimentalWarning: --experimental-loader is an experimental feature. This feature could change at any time
    at emitExperimentalWarning (node:internal/util:224:11)
    at initializeLoader (node:internal/process/esm_loader:59:3)
    at loadESM (node:internal/process/esm_loader:87:11)
    at runMainESM (node:internal/modules/run_main:51:21)
    at Function.executeUserEntryPoint [as runMain] (node:internal/modules/run_main:74:5)
    at node:internal/main/run_main_module:17:47
TypeError [ERR_INVALID_FILE_URL_HOST]: File URL host must be "localhost" or empty on linux
    at __node_internal_captureLargerStackTrace (node:internal/errors:464:5)
    at new NodeError (node:internal/errors:371:5)
    at getPathFromURLPosix (node:internal/url:1459:11)
    at fileURLToPath (node:internal/url:1482:50)
    at toPathIfFileURL (node:internal/url:1561:10)
    at __node_internal_ (node:internal/fs/utils:681:16)
    at statSync (node:fs:1531:10)
    at fileExists (node:internal/modules/esm/resolve:260:10)
    at resolveExtensions (node:internal/modules/esm/resolve:335:9)
    at resolveExtensionsWithTryExactName (node:internal/modules/esm/resolve:322:10)
    at finalizeResolution (node:internal/modules/esm/resolve:375:16)
    at moduleResolve (node:internal/modules/esm/resolve:944:10)
    at defaultResolve (node:internal/modules/esm/resolve:1041:11)
    at ESMLoader.resolve (node:internal/modules/esm/loader:530:30)
    at ESMLoader.getModuleJob (node:internal/modules/esm/loader:251:18)
    at ModuleWrap.<anonymous> (node:internal/modules/esm/module_job:79:40)
    at link (node:internal/modules/esm/module_job:78:36)
(node:614409) WARNING: Exited the environment with code 1
    at exit (node:internal/process/per_thread:194:13)
    at /home/poop/git/foss/nodeproto/node_modules/.pnpm/uvu@0.5.3/node_modules/uvu/bin.js:32:12
 ELIFECYCLE  Command failed with exit code 1.

poop@spaceship:~/git/foss/nodeproto/packages/libraries/testproto
10:49 AM (rework-testproto *$=) 
$

Seems to still be broken in v18.0

The plan is to remove --experimental-specifier-resolution=node once the equivalent functionality can be achieved via a custom loader without an unreasonable amount of boilerplate. See https://nodejs.org/api/esm.html#customizing-esm-specifier-resolution-algorithm.

Therefore, this issue is unlikely to be fixed unless it can be demonstrated to have wider implications than behavior under --experimental-specifier-resolution=node.

--experimental-specifier-resolution=node has been removed. I'll go ahead and close this.

Navigation Menu

Repository navigation

node >=17: experimental esm loader doesnt respect --experimental-specifier-resolution=node #41321

Version

Platform

Subsystem

What steps will reproduce the bug?

How often does it reproduce? Is there a required condition?

What is the expected behavior?

What do you see instead?

Additional information

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

node >=17: experimental esm loader doesnt respect --experimental-specifier-resolution=node #41321

Description

Version

Platform

Subsystem

What steps will reproduce the bug?

How often does it reproduce? Is there a required condition?

What is the expected behavior?

What do you see instead?

Additional information

Activity

noahehall commented on Dec 25, 2021

aduh95 commented on Dec 25, 2021

bmeck commented on Dec 25, 2021

noahehall commented on Dec 26, 2021

noahehall commented on Jan 31, 2022

Nick-Mazuk commented on May 1, 2022

GeoffreyBooth commented on May 1, 2022

bnoordhuis commented on Aug 19, 2023

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions