Proposal: support pathof along with current keyof

Problem - What are users having difficulty with?

While using popular libraries for their day to day development needs, developers can find them disjoint from typesafe philosophy even if they do have the needed typings. This is true for frequently used libraries such as lodash and immutablejs that access or set properties via paths. It is up to the developers to to preserve typesafety by expressing complex types using the amazing power that typescript gives them. The only obstacle is the absence of a way to express the type that represnts legit object paths for a specific type.

Current state

We can currently do this only for shallow objects where paths can be simply expressed as the keys of a specific type.

Example Issue

In an effort to play with stricter typings for immutablejs map I created this tiny project :
https://github.com/agalazis/typed-map/
(as a side note my personal view is that immutable js map is not a conventional map that should be represented with map<keyType, valueType> since it represents a map that matches a specific type rather than just a key,value data structure as demonstarated in src/examples)

The type I created was just this (only playing with get and set):

export type TypedMap<T> = {
  get: <K extends keyof T >(k:K) => T[K];
  set: <K extends keyof T >(k:K, v:T[K]) => TypedMap<T>;
}

Simple enough, leveraging all the expressiveness of typescript.

Example usage of the proposed solution:

If I could replace keyof with pathof in order to express possible path strings and also used T[P] as the path type I would be able to completely cover this use-case :

export type TypedMap<T> = {
  get: <P extends pathof T >(p:P) => T[P];
  set: <P extends pathof T>(p:P, v:T[P]) => TypedMap<T>;
}

Possible usage in the following libraries:

lodash
immutable
ramda
anything built on top of them

Why bother since this does not involve facilitating any feature of ES standard?(and is somewhat an unconventional feature)

We will be able to perform type-safe updates using existing js immutability libraries similar to what you can achieve in other languages (eg. skala lense).
This feature complies with typescript goals:

Statically identify constructs that are likely to be errors.
Provide a structuring mechanism for larger pieces of code (the above-mentioned libraries are used every day in big projects imagine the chaos that can be created. Despite exhaustive unit testing, intellisense is a must ;) ).
Strike a balance between correctness and productivity.

This feature improves compile time checks and sticks to the mindset of not having to deal with any runtime functionality
There are no side effects in the generated js code

Alternatives

While digging further into the issue an alternative solution is to be able to spread keyof recursively. This will allow us to be creative and build our own solution as per:
#20423 (comment)
The drawbacks of the alternative solution if doable are:

performance
lack of cyclic reference support

Implementation Suggestions

The optimal (if implemented in the language) would be to not compute the full nested object paths but only the paths used via pathof ie when declaring something as path of x just accept it as pathof x then when a value is assigned just validate it is pathOf x( the compiler could also have some sort of caching so that it doesn't recalculate paths).
This will also solve the issue with cyclic references since paths will be finite anw.

So, if I understand correctly, the idea is that you could have something like

var m: TypedMap<{ a: { b: number } }>;
var m2 = m.set('a.b', 42);

where 'a.b' would be of type pathof { a: { b: number } }.

You can already have type safety for a similar function

m.set('a', 'b', 42);

with the following type definition for TypedMap:

interface TypedMap<T> {
  get<K extends keyof T>(k: K): T[K];
  get<K extends keyof T, K2 extends keyof T[K]>(k: K, k2: K2): T[K][K2];
  get<K extends keyof T, K2 extends keyof T[K], K3 extends keyof T[K][K2]>(k: K, k2: K2, k3: K3): T[K][K2][K3];

  set<K extends keyof T>(k:K, v:T[K]): TypedMap<T>;
  set<K extends keyof T, K2 extends keyof T[K]>(k:K, k2: K2, v:T[K][K2]): TypedMap<T>;
  set<K extends keyof T, K2 extends keyof T[K], K3 extends keyof T[K][K2]>(k:K, k2: K2, v:T[K][K2][K3]): TypedMap<T>;
}

Now, this approach does have a few drawbacks, of course; the API isn't quite as neat and the supported depth of the objects is limited by the number of overloaded function definitions you are willing to write.

Still, most of the use cases you mention could be achieved with the current state of the type system.

Playground link.

One problem is that the following is completely legal:

interface A {
    a: boolean;
    b: 2;
    'a.b': string;
    '"a.b"': number;
}

type T1 = A['a']; // boolean
type T2 = A['a.b']; // string
type T3 = A["'a.b'"]; // string[]
type T4 = A['"a.b"']; // number

In fact, any path string you can possibly come up with is a fully legal keyof T for some T, so you'll run into collisions between the two.

Oskari Noppa (@noppa)

Oskari Noppa (@noppa) Thank you for contributing your thoughts yes the above is another way to approach this issue but as you mentioned, it has the depth limitation but also a productivity* issue(if we manually have to do all the overloading or rewrite typings for every type since depth is variable).
*TS goals: Strike a balance between correctness and productivity
You just demonstrated another way that cannot be used for shipping plug and play typings for the mentioned use case/libraries and confirmed my suspicion that this is not currently doable( a strong indication is also that none of the above-mentioned libraries supports such strict checks).
On the other hand, based on the comments, I started looking into a way of expressing the path as an array since path as array of keys is something commonly used:

type PathSetter<T, K extends keyof T ,P=K,M=TypedMap<T>> =  ( p:P, t: T[keyof T]) => M | PathSetter<T[K], keyof T[K], [P, ...keyof T[K]], M>
export type TypedMap<T> = {
  get: <K extends keyof T >(k:K) => T[K];
  set: <K extends keyof T >(k:K, v:T[K]) => TypedMap<T>;
  setIn: PathedSetter<T, keyof T>;
  
}

which won't work since I can't use spread operator on keys

My example would also work with tuple types, which might be closer to what you are looking for

interface TypedMap<T> {
  set<K extends keyof T>(k:K, v:T[K]): TypedMap<T>;
  set<K extends keyof T, K2 extends keyof T[K]>(k: [K, K2], v:T[K][K2]): TypedMap<T>;
  set<K extends keyof T, K2 extends keyof T[K], K3 extends keyof T[K][K2]>(k: [K, K2, K3], v:T[K][K2][K3]): TypedMap<T>;
}

declare var m: TypedMap<{ a: { b: number } }>;

var m2 = m.set(['a', 'b'], 42);

Yes that's true but it should happen in a recursive way that's what I tried in my example above but since we cannot have spread operator on the array, it's not doable I guess.

I think a possible solution could be blocked by:
#17884

In addition to the libraries mentioned in the initial comment, something like this proposal is absolutely necessary in order to provide any sort of type checking on queries in the elasticsearch.js client driver.

The other very common use case I keep stumbling on is dot notation in query string parameters.

This is an eagerly awaited feature for me! It would allow for much less headaches when dealing with utility functions designed to pull nested properties out of other types!

Just implemented this feature
https://github.com/Morglod/ts-pathof

const c = { z: { y: { bb: 123 }}};
const path = pathOf(c, 'z', 'y', 'bb');
// path now is typeof [ 'z', 'y', 'bb' ]

const path2 = pathOf(c, 'z', 'y', 'gg'); // error, because no 'gg' field in c.z.y

Also types only approach:

let path: pathOf3<typeof c, 'z', 'y', 'bb'>;
path = pathOf(c, 'z', 'y', 'bb');

Vladimir Kruzhkov (@Morglod)

Thanks for posting in the other thread, Vladimir Kruzhkov (@Morglod). For reference to others, #12290 also discusses tackling similar functions.

fwiw, tackling m.set('a.b', 42) is far off currently -- the only operation we can do using string literal types is pretty much direct object navigation. for type-safety, it may currently be more realistic to stick with tuple-based variants (-> ['a', 'b']) like in e.g. Ramda.

I feel this problem is more general than just path-based navigation though -- similar challenges exist for lenses/traversals as well. Given that, I would advocate a more generic approach based on recursion, which I hope will gain further support.

Andreas Galazis (@agalazis): I think #17884 actually ended up superseded by a recently merged PR by ahejlsberg! :)

@tycho01 was it released? looking forward to giving it a shot

@tycho01 just updated ts-pathof.

Now you can:

import { hasPath } from 'ts-pathof';

const c = { z: { y: { bb: 123 }}};
const path = hasPath(c, [ 'z', 'y', 'bb' ]);
path -> [ 'z', 'y', 'bb' ]

const path2 = hasPath(c, [ 'z', 'y', 'gg' ]); // no error
path2 -> value is false, type is never

or

import { PathOf } from 'ts-pathof';

const o = {x: { y: 10 }};

type xy = PathOf<typeof o, ['x', 'y']>;
xy -> ['x', 'y']

type xyz = PathOf<typeof o, ['x', 'y', 'z']>;
xyz -> never

Andreas Galazis (@agalazis)

Andreas Galazis (@agalazis): check out #24897. while technically it doesn't go as far, Concat from #24897 (comment) (not officially supported) does get you there as a ... alternative.

Vladimir Kruzhkov (@Morglod)

Vladimir Kruzhkov (@Morglod) still you need 277 lines of code to achieve what we could achieve in just a few (in my example) if spreading keyOf was supported (and to be fair I am not sure how your code copes with arbitrarily nested objects or if it has some sort of limit)

Andreas Galazis (@agalazis)

Andreas Galazis (@agalazis)
There will be much more code in typescript compiler, server etc to achive this)
I posted it as temporary solution for this case. Better wait 2+ years?

Actually you are right, this hack works only for 2 levels deep.
Rewrited implementation without hacks.

Vladimir Kruzhkov (@Morglod)

Vladimir Kruzhkov (@Morglod) Your work is amazing(I would use your library any time), my point was that the proposal is still valid, I am sure you agree

Andreas Galazis (@agalazis)

Andreas Galazis (@agalazis) ye it should be in language

This would solve a lot of problems.

we do it like this: (in a class query)

 Filter<Y>(fieldFunc:  (part: T) => Y, filter: FilterDataGridFilterType | "==" | "!=" | "<" | ">" | "in" | "<=" | ">=", value: Y, level?: number) {
    let field = fieldFunc.toString().split('=>')[1].split('.').splice(1).join('.');
    ....

so we could use like this:

    let qry = (new QueryT(BasicActionDTO)).Service(this.mfcname);
    qry = qry.Filter(x => x.Archive, "==", 0);

so we could convert our c# code to typescript nearly the same...
we know that it's not safe if someone uses a complex function instead of only property access, but it works for us

Any news or related thread?

That's really problematic in some kind of libraries, such as MongoDB. Where dot notation is used to atomically change fields on a document.

Also on Lodash get/set, ramda and various other libraries.

I've been working on a solution to this problem using template string types. Playground link

It's a little convoluted but I was trying to avoid full-on recursion in hopes of suffering from less of a performance penalty. I think it works but may still be slow.

I asked on the TS Gitter and was given a much simpler solution by user @webstrand

type PathOf<T, K extends string, P extends string = ""> =
  K extends `${infer U}.${infer V}`
    ? U extends keyof T ? PathOf<T[U], V, `${P}${U}.`> : `${P}${keyof T & (string | number)}`
    : K extends keyof T ? `${P}${K}` : `${P}${keyof T & (string | number)}`;

declare function consumer<K extends string>(path: PathOf<KnownObjectType, K>);

Note that the second generic parameter is the path string itself, so explicitly passing it is basically redundant -- const myPath: PathOf<SomeObject, "a.b.c"> = "a.b.c". But when you use it to type a function argument, as with consumer above, the path-string type K can be inferred contextually and you can just call consumer("a.b.c") by itself.

I think the pros and cons between my original solution come down to capabilities and performance. My solution fully computes four-deep paths, which I'm sure is slow -- possibly awful if the input type has lots of index signatures etc. Webstrand's solution requires partially-specified generic params to be useful, which is always kind of a hassle, but likely performs a lot better because it's not forcing the compiler to compute the full set of possible key names.

(As an aside, I'd still love to see an officially supported solution for this problem.)

In case anybody is tracking this, here's a refined Path type that should be more robust:

type StringableKey<T> =
    T extends readonly unknown[]
        ? number extends T['length']
            ? number : `${number}`
        : string | number;

type Path<T> =
  T extends object
    ? {
        [P in keyof T & StringableKey<T>]: `${P}` | `${P}.${Path<T[P]>}`;
      }[keyof T & StringableKey<T>]
    : never;

Like the other one, I would expect this to perform poorly when used on a very deep or complex input type, and to error out completely if the type is recursive.

James Bromwell (@thw0rted)

James Bromwell (@thw0rted) I didn't know about template string types, when was it introduced? Can it be used as a discriminant property in a tagged Union type?

It's a relatively new feature. I haven't tried it with tagged unions. I think in most cases it's supposed to work pretty similarly to a literal union, i.e. type Digit = 0|1|2|3|4|5|6|7|8|9; type FooDigit = `Foo${Digit}`; expands to type FooDigit = "Foo0" | "Foo1" | "Foo2" | "Foo3" | "Foo4" | "Foo5" | "Foo6" | "Foo7" | "Foo8" | "Foo9". So, I'd expect it to behave the same way as a literal union when used as a discriminant property.

I played with it a little and it looks like a pretty good result. You can try it for yourself if you like.

In case anybody tracking this issue is curious, I asked for a built-in helper type in #46337 and got shot down because helper types can be kind of controversial. As I said over there, maybe a keyword would be better in the long run since the team could really think about how to make it performant, how to handle recursion, how to improve error messages, etc etc. Still watching with interest!

Navigation Menu

Repository navigation

Proposal: support pathof along with current keyof #20423

Problem - What are users having difficulty with?

Current state

Example Issue

Example usage of the proposed solution:

Possible usage in the following libraries:

Why bother since this does not involve facilitating any feature of ES standard?(and is somewhat an unconventional feature)

Alternatives

Implementation Suggestions

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Proposal: support pathof along with current keyof #20423

Description

Problem - What are users having difficulty with?

Current state

Example Issue

Example usage of the proposed solution:

Possible usage in the following libraries:

Why bother since this does not involve facilitating any feature of ES standard?(and is somewhat an unconventional feature)

Alternatives

Implementation Suggestions

Activity

noppa commented on Dec 3, 2017

dmichon-msft commented on Dec 6, 2017

agalazis commented on Dec 6, 2017

noppa commented on Dec 7, 2017

agalazis commented on Dec 7, 2017

agalazis commented on Dec 14, 2017

marshall007 commented on Jun 15, 2018

Nimelrian commented on Sep 20, 2018

Morglod commented on Sep 20, 2018

KiaraGrouwstra commented on Sep 20, 2018

agalazis commented on Sep 21, 2018

Morglod commented on Sep 21, 2018

KiaraGrouwstra commented on Sep 22, 2018

agalazis commented on Sep 23, 2018

Morglod commented on Sep 23, 2018

agalazis commented on Sep 24, 2018

Morglod commented on Sep 25, 2018

diegohaz commented on Apr 9, 2019

jogibear9988 commented on Nov 5, 2019

ScreamZ commented on Jan 5, 2020

thw0rted commented on Apr 2, 2021

thw0rted commented on Apr 6, 2021

agalazis commented on Apr 16, 2021

thw0rted commented on Apr 16, 2021

thw0rted commented on Oct 14, 2021

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions