Repository navigation
Improve API for configuring a high number of arguments #45
Description
Activity
That seems like a useful semver-minor addition in the future, when usage patterns are well-understood.
Reacted by John GeeFair question. I think the initial focus is on keeping it simple, and large scale CLI is not a concern for now. (And a large scale program could repackage the one-object configuration fairly easily.)
I think the initial focus is on keeping it simple
Totally understand the desire to keep things simple. However, I don't believe the proposed suggestion adds complexity to the implementation or configuration. For example, I modified a few lines in the current implementation and enabled individual configuration objects: main...aaronccasanova:feat/improve-options-api
const argv = ['--f', 'foo1', '--foo', 'foo2', '--bar', 'baz']; const parsed = parseArgs(argv, { args: { foo: { short: 'f', withValue: true, multiples: true, }, }, }); console.log('parsed:', parsed);
large scale CLI is not a concern for now
Being that this is intended for Node core, I would have thought otherwise.
Appreciate the quick response and feedback!
My concern about including it now isn't about the code complexity, it's about the API surface, and the design space.
For background, the Intended Audience from the "Initial Proposal". (May or may not reflect what eventuates!)
Intended Audience
It is already possible to build great arg parsing modules on top of what Node.js provides; the prickly API is abstracted away by these modules. Thus, process.parseArgs() is not necessarily intended for library authors; it is intended for developers of simple CLI tools, ad-hoc scripts, deployed Node.js applications, and learning materials.
It is exceedingly difficult to provide an API which would both be friendly to these Node.js users while being extensible enough for libraries to build upon. We chose to prioritize these use cases because these are currently not well-served by Node.js' API.
That's a helpful bit of context! Thanks for the reference @shadowspawn
@aaronccasanova, I'm with @ljharb on erring on the side of a smaller API for MVP. With the goal initially being to support simple throw away CLIs and code samples.
I'm hopeful we might be able to target Node 18, or Node 19 with the MVP, so the tighter we can keep the API the better, hence trying to keep behavior constrained.
I think this work could immediate be a great follow up, and a good place to contribute to the Node.s codebase.
Reacted by Aaron Casanova@bcoe What about making this the only way to configure options? It seems more natural, to me: it means you're describing each argument in one place, rather than splitting up the configuration for each option across several fields. And if it's the only way to do it, it doesn't increase the API surface.
const parsed = parseArgs(argv, { args: { foo: { withValue: true, }, }, });
does not seem noticeably harder to use than
const parsed = parseArgs(argv, { withValue: ['foo'], });
and has the advantage of being a lot more natural as soon as you get at all beyond that:
const parsed = parseArgs(argv, { args: { foo: { withValue: true, multiple: true, }, bar: { withValue: true, }, }, });
is much easier to understand than
const parsed = parseArgs(argv, { withValue: ['foo', 'bar'], multiple: ['foo'], });
since the latter requires you to scan multiple lists to figure out how
foois going to behave. More importantly, it also matches how people are likely to be accustomed to doing configuration: the properties for each field are set within that field, rather than having lists of fields-with-this-property.does not seem noticeably harder to use than
I think using nested objects is noticeably harder. The simplest configuration is a triply nested object. I think that is more challenging for a beginner to scan, or indeed type.
Smaller comments:
- Multiples are going to be relatively rare. But there will still often be two properties in place as I think short options are pretty routine. So your comments about how to slice still apply.
- I personally slightly prefer the objects from a purist point of view, but I still prefer the arrays from a pragmatic point of view for the initial target audience.
I don't think "nested objects" as a concept is terribly confusing for most beginners. In my experience that's not the kind of complexity which usually trips people up when starting out. "These two things are implicitly related despite having no syntactic connection", on the other hand, definitely is.
So I think the above approach is better even from a strictly pragmatic point of view when optimizing for the experience of beginners.
That said, you can quite reasonably drop a level of indentation by having the second argument just be
args, and having remaining options be in an optional third parameter. Then you have only doubly-nested objects, which is the same level of nesting as in the current API.That is:
const parsed = parseArgs(argv, { foo: { withValue: true, multiple: true, }, bar: { withValue: true, }, }, /* any other options */);
What about making this the only way to configure options?
@shadowspawn @darcyclarke @ljharb, if we went with the nested object approach for configuration, I feel like strict becomes more natural, since all your valid arguments are defined in one place ... For flags, we could allow any empty object for configuration, or flag: true?
I’m ambivalent on either of the two ways to configure, but i feel strongly that non-strict is a very bad default, so if this argument form comes with default-strict, I’m super on board.
I think using nested objects is noticeably harder.
I used a nested object structure for three reasons:
- Allows for more top level configurations
- Minimizes the chances of introducing breaking changes in future iterations
- Easier to static type check
- Allows for more top level configurations
e.g.
parseArgs(argv, { strict: true, ...moreTopLevelConfigs, args: {...} })
- Easier to static type check
e.g.
type ParseArgsOptions = { strict?: boolean args?: { [arg: string]: { // 👈 Isolated from top level configs short?: string withValue?: boolean multiples?: boolean } } }
That said, you can quite reasonably drop a level of indentation by having the second argument just be args
We may not necessarily care about this here, but dropping a level of indentation will result in flaky types if we want to allow more top level configurations:
e.g. How do you accurately type this example:
TSPlayground - with nested object structure
TSPlayground - without nested object structureparseArgs(argv, { strict: true, foo: { short: 'f', withValue: true, }, bar: false, // Should error, not a valid arg value or top level config }) // This would pass, but is no longer valid or representative of the implementation: type ParseArgsOptions = { strict?: boolean [arg: string]: boolean | { short?: string withValue?: boolean multiples?: boolean } }
We may not necessarily care about this here, but dropping a level of indentation will result in flaky types if we want to allow more top level configurations:
The suggestion was that options like
strict, which are not intended as CLI arguments but rather configure the parser itself, would go in a new third parameter. I agree it would be bad to mix those with arguments in the same object.4 remaining items
I like the suggestion of:
const parsed = parseArgs(argv, { options: { foo: { withValue: true, }, }, });
Personally, for whatever reason ergonomically I like passing first the arguments that you're parsing, then the description of how the parse shoudl take place.
Personally, for whatever reason ergonomically I like passing first the arguments that you're parsing, then the description of how the parse should take place.
Me too! Although, while I have no data to back this up, I imagine the majority of users will leverage the default argv = getMainArgs(). As a result, we may see a lot of this out in the wild:
parseArgs(undefined, { options }). Thoughts?I agree with both viewpoints - passing the args first is better, but being able to omit them is the many-nines use case.
Short of providing two APIs - one that takes required args, and one that defaults them - i'm not sure how to reconcile those.
Personally I think it makes more sense for the thing-to-be-parsed to come after the specification-of-how-to-parse. (Compare, for example, a tool like ajv, where you build a parser and then apply it.) I'm having trouble wrapping my head around the other viewpoint; can someone say more about why they prefer that?
That said, I note that if you use an options bag, as in #63, there's no issues with ordering.
"parse args" is a verb, "args" is the noun i want to parse, and "options" is how i want it parsed.
I agree with that but don't understand why it implies anything about the arguments order. If anything it's the reverse: "use this specification to parse these args".
Certainly one can form a sentence in either direction - "parse these args this way" versus "parse, this way, these args".
@bakkot another argument I'd make for arguments and then options bag, is that it's what the community is accustomed to, as it's the approach taken by commander, yargs, minimist, args (worth noting I did find one exception meow).
I'm pretty convinced that the combined popularlity of yargs, minimist, commander, and args, has enough momentum, that if we switch the order on people the community would constantly be screwing up the argument ordering.
Reacted by Jordan HarbandI'm pretty convinced that the combined popularlity of yargs, minimist, commander, and args, has enough momentum, that if we switch the order on people the community would constantly be screwing up the argument ordering.
Are people constantly screwing up the argument ordering with
meow? I would be kinda surprised to learn that was a common problem.I also note that the way I've usually seen commander and yargs used is as
yargs().variousBuilderMethods().parse(argv)oryargs(argv).variousBuilderMethods().parse(), where the variousBuilderMethods are functioning like the options bag parameter here. That's the style used by both projects in their introductions. That does not seem directly analogous to what we're discussing.args
command-line-args is much more popular than args and takes the options bag approach (its first parameter is an options bag with an optional
argvkey).(I should mention that, while it seems more natural to me to have the configuration parameter first, I'm not all that worried either way. I care more about the general shape of the configuration discussed in this issue than about the precise order of the arguments.)
Reacted by John Gee, Jordan Harband and Aaron Casanovaanother argument I'd make for arguments and then options bag, is that it's what the community is accustomed to, as it's the approach taken by commander, yargs, minimist, args (worth noting I did find one exception meow).
Short version: Commander only superficially follows the pattern, and I don't think is prior art supporting "arguments and then option bag" expectations.
Long version
For many years in Commander,
.parse()had a single required argument which was the array of arguments to parse. Then the argument was made optional with a default ofprocess.argv, and support was added to allow specifying the source of the arguments to override automatic decision about which ones to ignore. The only configuration in the option bag is how to interpret the array of arguments, and the expectation is in practice the options-bag is only used when the first argument is supplied.program.parse(process.argv); // classic program.parse(); // modern program.parse(['--demo'], { from: 'user' }); // common pattern in unit tests for exampleI'm fine with @aaronccasanova approach of providing
argvas an option on the config object, rather than taking two arguments, as we see in #63. Withargvdefaulting toprocess.argv-- this seems to side step disagreement about argument order.
I'm not seeing much argument in this thread against the spirit of #63, so I propose we land the work soon, since it seems to get us close to our goal of an MVP that we could put in Node 18.
Reacted by Aaron Casanova- added a commit that references this issue
on Mar 2, 2022 Closing as #63 has landed.
Reacted by Aaron CasanovaReacted by Aaron Casanova
I wanted to play devils advocate on the design of the
parseArgsoptionsobject. Are there any concerns that theoptionsobject will become unwieldy as the number of arguments grow? While I am still catching up on prior discussions and understanding the target audience of this tool, I imagine creating large scale CLIs (e.g.aws-cli,git,docker, etc.) will be challenging to configure as configuring a single argument requires setting multiple top level keys (e.g.withValues,multiples,short).I wanted to gauge folks opinion on allowing users to set all the configurations for a single argument in one object.
(Current) Example configuring large scale CLI options
(Proposed) Example configuring large scale CLI options