WRT PR #3 - ES module path resolution

This is the place to discuss specifics of how the EcmaScript (ES) module imports will resolve paths. The proposal itself can be seen in PR #3.

Discussions here should regard:

What differences module resolution in ES module import syntax will have from require's module resolution algorithm.

It should not discuss import/export conversion, evaluation ordering, or detecting file mode.

@domenic

To be clear, this is about import and export statements, whether they are declarative or imperative calls, it doesn't really matter because they are reflective somehow.

Additionally, we should, somehow, try to weight-in the resolution process used in browsers, to facilitate portability, I think @domenic can give us a quick overview of what we have so far in HTML.

Questions to be ask:

should the importer always include the file extension?
should we disallow the ambitious mechanism we do today in node to resolve index.js? specifically, the fact that require('./something') could resolve in ./something.js or ./something/index.js.
should we allow introspection into the module's internals? e.g.: import foo from "mod/lib/util"; or should we consider this a legacy behavior, and if you want to access the internals of the module, you should use require(), while an npm pkg will have a single es6 module that represents its api? (this somehow leaks into the discussion about .jsm vs package.json out of band signal)

What browsers are implementing is documented at https://html.spec.whatwg.org/#resolve-a-module-specifier. The answer to your questions for browsers are:

Yes, file extension is always necessary; import "./foo" looks for a file named foo, not one named foo.js.
Yes, index.js automagic is disallowed. import "./something" looks for a file named something, not one named something/index.js.
Not applicable yet, as any non-absolute-URL import specifiers not prefixed with /, ./, or ../ fail. Both seem possible in the future; when a mapping table API is created (manifest etc.) it could either be based on prefix-matching (allowing inspection of "internals") or just string equality (only allow inspection of "internals" if explicitly exposed in the manifest).

The proposal currently has these specified. Currently the behavior is:

yes, importer must include file extension
this is disallowed if not looking for a package, but when searching for package main we use the current require algorithm.
currently this is allowed, it is used by various packages on npm and would be a hard sell. Currently the proposal does require file extensions for inner files, that would only affect file extension (keep that discussion contained to WRT PR #3 - ES module detection #13).

@domenic

@domenic yes, 3 does not apply to the browser, although you could enforce that in the future via loader hooks.

@bmeck excellent, 1 and 2 seems to be very straight forward. As for 3, I think the various packages on npm today that are doing so are fat packages, and the transpiled version of their source are doing require() anyways. Once node supports ES modules, they will have to rework their pkgs a little bit, and should be fine to introduce this restriction. For years, we have been battling this, it is a refactor hazard for module authors, and signaling that to consumers of the modules by forcing them to use an alternative api to import those internals, seems like a good tradeoff here.

@caridy

@caridy banning inner files will cause some issues, but can probably smooth that over. Going to say we ban inner imports for packages.

I think it would be good to give people an opt-in way to expose their internals. I agree it's a hazard that they're automatically exposed, but for e.g. packages like especially or IIUC lodash it's an important part of their API.

@caridy

@caridy banning inner files will cause some issues, but can probably smooth that over. Going to say we ban inner imports for packages.

@bmeck I'm not proposing banning them, obivously, inside your pkg you can do whatever you want since you can do ./path/to/relative/files.js. My proposal is to ban it when using import "packageName/fileName.js";

packages like especially or IIUC lodash it's an important part of their API.

@domenic Yes, we have few popular pkgs using that technique, and those will continue to work for people who are using require() to import them, but if you want to use an import statement, you will have to do something like: import {Math} from "especially"; which is itself a namespace produce by: export * as Math from './math.js'; in your index.js which will describe the API of the pkg.

@caridy

@caridy We have packages that include service clients and stubs to use for testing. We really don't want to load the service stubbing code into our production apps. Disallowing separate entry points sounds like a pretty big restriction. "You can keep using require" is another way of saying "if you want consistency in your code base and you're doing anything beyond toy examples, ignore import". Which would be a rather unfortunate outcome.

@domenic

@domenic would the reexport work if the loader does not evaluate any dependency that only has reexported bindings being imported (words @_@)?

// especially
export * as Math from './math.js';

// consumer
import {Math} from 'especially';

If we specify that especially does not evaluate since it has no direct bindings requested by consumer. Would that work? (me is rereading 262 since this would be a change it looks like, feel free to correct me).

I don't think there's any need to change the execution strategy. Reexporting would work I guess. Just kind of ugly.

Forget what I said, side effects could mean the execution order needs preservation. We can't skip.

@caridy

@caridy We have packages that include service clients and stubs to use for testing. We really don't want to load the service stubbing code into our production apps. Disallowing separate entry points sounds like a pretty big restriction.

@jkrems this is precisely what this discussion is for, to surface use cases, like yours. Can you elaborate more about the structure of your pkg? maybe a link that we can look at?

"You can keep using require" is another way of saying "if you want consistency in your code base and you're doing anything what toy examples, ignore import". Which would be a rather unfortunate outcome.

to reiterate, it is the consumer or existing pkgs using require() that will continue using require() for their convenience. If, as a consumer, you really want to use import, then you follow the new API provided by the author of the pkg.

We have packages that include service clients and stubs to use for testing. We really don't want to load the service stubbing code into our production apps.

This is a use case I've hit and pondered with ES6 modules a few times. While it's possible to control what gets exported, there doesn't seem to be any way to conditionally prevent unneeded code from being loaded at all.

@timoxley

@timoxley you will need to use an imperative function to load things conditionally:

// declare what can be exported:
export if_debug;

// conditionally load using require
let if_debug = IS_DEBUG ? require('foo') : null;

// conditionally load using System.loader.import
let if_debug = null;

// block completion using `await` , this would not affect require
// after this point if_debug is defined
await System.loader.import('foo').then(ns => if_debug = ns;)

additionally, once we settle on the api for the import() imperative api, you should be able to do something like:

export let if_debug = null;

if (something === true) {
    if_debug = await import('./something');
}

and that is not that far from what you do today in node.

@ljharb

@ljharb: You mean client-side? What's the issue there then?

@pakastin

@pakastin no, i mean requiring any file from any folder. Only npm-installed packages have a package.json, but any file can be required.

I don't understand. If you use import don't you expect the file being an ES6-module? Wouldn't it be better to be other way around: if you want to import legacy commonjs file it would have the suffix '.cjs' for example?

@pakastin

@pakastin no. you should be able to import a CJS module and require an ES6 module without having to know the module format that was used.

Why need for .mjs then? Don't you know the format if you define it to be .mjs?

@pakastin

@pakastin please read this comment (and the rest of the thread): #3 (comment)

ok, thanks! 👍

@ljharb

@ljharb we discussed changing the resolution mode when using [bare paths], that apparently did not get written down into spec, bug, will fix when I get a chance. Def needed.

@bmeck

@bmeck thanks. does the same problem exist for import x from 'no_package_json_but_in_node_modules'?

@ljharb

@ljharb nope, see examples under https://github.com/nodejs/node-eps/blob/master/002-es6-modules.md#es-import-path-resolution

Sorry to chime in so late, I don't think the file extension issue is fixable for web servers with rewrite rules, a rewrite rule is just a rewrite rule, this is an algorithm:

try FILENAME
try FILENAME.js
is FILENAME === "package", if so try "package.json"
TRY FILENAME/index.js

Maybe I'm just super-ignorant about what nginx/apache can do with rewrite rules, but I'm thinking a web server plugin is likely needed to achieve this.

@matthewp

@matthewp a rewrite rule (certainly an apache one, i'm not sure about others) should trivially, given the request path /foo, be able to check for the existence of foo.mjs, foo.js, foo/index.mjs, and foo/index.js, selecting the first one that is found, without the browser having to know either the extension or whether the path is a directory or not. To me that seems sufficient to address any issues. What seems problematic to you?

Ok, I'll take your word for it, I assuming you would use things like RewriteCond, I didn't realize rewriting was that sophisticated. So the remaining issue would be that the server needs to conditionally apply this rewrite only to module requests, and I don't see a signal of that from the browser as currently specced.

It would apply it to everything without an extension explicitly included, with the JS mime type. The browser spec has nothing to do with that.

Navigation Menu

Repository navigation

WRT PR #3 - ES module path resolution #11

Questions to be ask:

37 remaining items

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

WRT PR #3 - ES module path resolution #11

Description

Activity

caridy commented on Mar 10, 2016

Questions to be ask:

domenic commented on Mar 10, 2016

bmeck commented on Mar 10, 2016

caridy commented on Mar 10, 2016

bmeck commented on Mar 10, 2016

domenic commented on Mar 10, 2016

caridy commented on Mar 10, 2016

hybrist commented on Mar 10, 2016

bmeck commented on Mar 10, 2016

domenic commented on Mar 10, 2016

bmeck commented on Mar 10, 2016

caridy commented on Mar 10, 2016

timoxley commented on Mar 10, 2016

bmeck commented on Mar 10, 2016

caridy commented on Mar 10, 2016

37 remaining items

pakastin commented on Apr 28, 2016

ljharb commented on Apr 28, 2016

pakastin commented on Apr 28, 2016

ljharb commented on Apr 28, 2016

pakastin commented on Apr 28, 2016

mariusGundersen commented on Apr 28, 2016

pakastin commented on Apr 28, 2016

bmeck commented on Apr 28, 2016

ljharb commented on Apr 28, 2016

bmeck commented on Apr 28, 2016

matthewp commented on Jul 7, 2016

ljharb commented on Jul 7, 2016

matthewp commented on Jul 7, 2016

ljharb commented on Jul 7, 2016

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions