Discussion: (Reflective) Type Model

Hi all,

We currently have a AST model for TypeScript which is useful for compilers, editors, and linters.

However, it would be extremely useful to have a type model, a la Java's reflective model. There are a huge number of use-cases supporting this, but I will list a few here:

I have a REST interface and I define a data model for it as a TypeScript interface. I use this interface for the function definition of the REST implementation, but then use the type model to validate the input at runtime.
I have a service model that abstracts an interface for some system (database, or application, etc.). I define this as a TypeScript interface (maybe with some decorators as meta-data), and have a generator library that extracts this information to either generate code - or create a runtime implementation - that actually implements the service.
I have an injection (DI) system. I use injection decorators on class properties/constructor parameters, and use the type model to reflectively wire the correct instance.

A good example of what could be achieved with this, is something like the Spring platform for Java where our applications are built as decorated components and most of the boilerplate code is abstracted away by the platform thanks to reflection and decorators.

I have a "first stab" at implementing such a thing: typescript-schema. It's not feature complete and the design isn't finished either. But hopefully it gives a feel for what a reflective model could look like.

Is this something the TypeScript team would be interested in having as a first class citizen?

You might be interested in #2577. Reflection type functionality has been an oft requested but controversial issue for us.

Another related topic would be #3136.

I use injection decorators on class properties/constructor parameters, and use the type model to reflectively wire the correct instance.

can you elaborate on how this is done.

Dan Quirk (@danquirk)

Dan Quirk (@danquirk) So when jonathandturner said at http://blogs.msdn.com/b/typescript/archive/2015/03/05/angular-2-0-built-on-typescript.aspx

We've also added a way to retrieve type information at runtime. When enabled, this will enable developers to do a simple type introspection.

he meant metadata generated only for decorators, not full-scale reflection (for all public symbols, for example)?

he meant metadata generated only for decorators, not full-scale reflection (for all public symbols, for example)?

correct.

What he said :)

I'm happy to see:

A few notes on metadata:

Type metadata uses the metadata key "design:type".

Parameter type metadata uses the metadata key "design:paramtypes".

Return type metadata uses the metadata key "design:returntype".

From #2589

What other metadata are you planing to support in the future? There are a few things that I would like to see:

Implemented interfaces uses the metadata key "design:implements".
Parameter name metadata uses the metadata key "design:paramnames".

Of course "design:implements" would require the interface names to be serialized using the interfaces names not Object (or complex type serialization). The same any case in which an interface is serialized.

Parameter name metadata uses the metadata key "design:paramnames".

this sounds interesting. and should be doable

The same any case in which an interface is serialized.

Interfaces are not serialized today. only names with a value attached to them (i.e. classes) and built-ins are serialized.

About:

Interfaces are not serialized today

I'm aware of a gist (by Ron Buckton) about complex type serialization, is that going ahead? Are you guys considering adding interface serialization support at some point?

As mentioned in my original post, I've got a project called typescript-schema that provides this kind of serialisation, as well as a reflective type model:

It has two types of model: A serialisable model, and a runtime reflective model.

It works by: AST -> Serialisable Model -> Runtime reflective model

It currently only supports 1.5, and only exported types. It's not 100% complete, but is able to process the entire TypeScript library itself, and the node type library too (as well as a number of other large libraries).

What I'd really like is to understand is:

a) if the TypeScript team would want any of this code, and if so
b) what would be required to make it ready for inclusion? I.e.: as it stands it's largely standalone (and probably doesn't follow all the code standards)

To give you an example of it at work, imagine this TypeScript code:

import {X} from 'my-module'
export interface MyInterface<T> {
  f(x: X, s: string):T
}

This will serialise to:

{
  "instanceType": {
    "typeKind": "COMPOSITE",
    "members": {
      "f": {
        "optional": false,
        "type": {
          "typeKind": "FUNCTION",
          "parameters": [{
            "name": "x",
            "type": {
              "module": "my-module"
              "name": "X"
            },
            {
              "name": "s",
              "type": {
                "typeKind": "PRIMITIVE",
                "primitiveTypeKind": "STRING"
              }
            }],
            "type": {
              "module": "@",
              "name": "T"
            }
          }
        }
      }
    }
  }
}

This can then be converted into a reflective model, which can be used like:

  let myInterfaceConstructor:InterfaceConstructor //Called an interface constructor because it has type parameters

  myInterfaceConstructor.parent // The parent module or namespace
  myInterfaceConstructor.name // "MyInterface"
  myInterfaceConstructor.typeParameters[0].name // "T"
  myInterfaceConstructor.instanceType.members['f'].type.name // "T"
  myInterfaceConstructor.instanceType.members['f'].parameters[0].name // "x"

You can then "close" that reflective interface by providing a type argument, and the output will be the corresponding interface with all type parameters correctly substituted:

  let myInterface = closeInterfaceConstructor(myInterfaceConstructor, [someType])
  myInterface.typeConstructor // myInterfaceConstructor
  myInterface.instanceType.members['f'].type // someType

To check it out, go to:

typescript-schema

and

typescript-package (which provides the code for converting a package of TypeScript into the serialisable form - including typings and nested packages via the node resolve algorithm)

Mohamed Hegazy (@mhegazy)

When I asked for a "design:paramnames" metadata key.

Mohamed Hegazy (@mhegazy) said:

this sounds interesting. and should be doable

So I have created a new issue to request it: #4905

Interfaces are not serialized today. only names with a value attached to them (i.e. classes) and built-ins are serialized.

What if you guys will introduce new language semantic? Something like realtime interface Name {...} which will remain realtime (with empty method bodies for easing mocking in testing frameworks)?
I doubt the interface serialization will be implemented somewhen, since there is a much code written which assuming that interfaces are not realtime objects.

I forced to use classes as interfaces now to make DI work

Is this issue getting at being able to do something like the following for using interface annotations as DI tokens?

interface IAlgorithm {
    execute(): void; 
}

class FancyAlgorithm implements IAlgorithm {
    execute(): void {
        let i = 123 * 456;
    }
}

class AlgorithmExecuter {
    constructor(private algorithm: IAlgorithm) { }

    execute(): void {
        this.algorithm.execute();
    }
}

class Program {
    main() {
        // DI API based on autofac DI for .NET
        // container could be populated any number of ways just imperative for illustration
        let builder: Container = new ContainerBuilder();
        builder.RegisterType(FancyAlgorithm).As(IAlgorithm);
        builder.RegisterType(AlgorithmExecuter).AsSelf();

        // now invoke the composition root
        let executer = builder.Resolve(AlgorithmExecuter);
        executer.execute();
    }
}

The specific reason I'm asking is because when using a DI system for TypeScript that uses type annotations as metadata (such as what exists for Angular 2) this is currently not possible because no runtime "Reflect" unique metadata is emitted for interfaces. In DI circles this is basically the way DI based apps are built. It's extremely powerful and makes for very flexible and easily testable apps.

If this issue doesn't involve this kind of metadata please let me know as I'd like to create a separate proposal issue. 😄

Mike Haas (@mikehaas763)

Mike Haas (@mikehaas763) nothing stops you today from doing the same thing but using strings in place of types. i am assuming this would be sufficient. your DI system needs to have a key to store and locate these, if you use strings, then nothing in this issue should block you.

@export("IAlgorithm")
class FancyAlgorithm implements IAlgorithm {
    execute(): void {
        let i = 123 * 456;
    }
}

The v0.0.20 release of typescript-rtti supports reflecting on both classes (ie "by value") and interfaces ("by type") via reflect(MyClass) or reflect<MyInterface>(), respectively. I think it covers both use cases nicely. As a bonus it can validate object values against the emitted types-- while more extensive solutions (possibly built on top of these features) will always make sense, the basic validation cases are covered.

My thought is to take the reflection API (reflect() mainly) itself out of the transformer project so that it can objectively implement any/all of the relevant RTTI formats in one place which should help to guard against dependency on a single one while we collectively work out what the format itself should look like long term, and create opportunities for others to come up with other solutions if they want to. It already supports both emitDecoratorMetadata and typescript-rtti's format, it should be possible to map more to it.

I do think the format typescript-rtti uses right now has some merits, but gawking at the "weirdness" of it is not unexpected- to be compact and also retain some level of intentional awkwardness when trying to access the metadata directly (instead of via API), it is not something you'd ever want to interact with directly. That was indeed the point.

I'd considered symbols prior but thought it was pointless considering the code being generated itself needs access to the symbols. It also complicates the transformer because those imports need to be independently managed. On the other hand, at the very least it eliminates the runtime visibility of it (unless you're looking for it), and better matches the (for lack of a better term) WebIDL style, so moving to that might make sense to clean it up in preparation for wider usage.

Edit: On the other hand, any dependency on a symbol necessitates a reference to the source library to interpret it's metadata format, and I'm not sure that's reasonable.

Andrew Schmadel (@schmod)

Andrew Schmadel (@schmod), Andy Kutruff (@akutruff), Jiri Spac (@capaj)

Released version 0.7.0 of tst-reflect is now able to get type of runtime value. getType(someValue)

import { getType } from "tst-reflect";

class A
{
	constructor(public foo: string)
	{
	}
}

const someValue: unknown = new A("Lorem ipsum");
console.log(getType(someValue).is(getType<A>())); // > true

const someValue2: unknown = { foo: "dolor sit amet" };
console.log(getType(someValue2).is(getType<A>())); // > false
console.log(getType(someValue2).isAssignableTo(getType<A>())); // > true

The runtime value in memory can be class or some native type (object, array, string, etc..).

Then it is possible to call existing .isAssignableTo(), so I've rewritten REPL I've posted few comments above to this REPL which use just .isAssignableTo(). Object validation out of the box...

Lot of progress on typescript-rtti lately including reflection on functions, shape matching, and more. We've also been battle testing it with complex codebases, both those dependent on emitDecoratorMetadata and not. You can try it out on this spiffy website I made that runs TS, the transformer and the reflect() API in the browser: https://typescript-rtti.org

There are true technical limitations

I think we can easily say that some reflection is better than no reflection. It doesn't have to be a complete solution that sees all types. It can stop within a reasonable subset of types for which reflection will help meet geater use cases, which is inversion of control and dependency injection. And that mostly means reflection for simple classes and interfaces. When these types can't be reasonably expressed, typescript can emit a no-op, just like it emits Object now on emitDecoratorMetadata

A system can't take an arbitrary TypeScript type, flush it to an output of any sort, and then reason about whether or not a value matches that type without effectively being tsc. And at the point that you've reimplemented tsc.

You are assuming the reason for flushing a type is to see whether it fits structurally. But let's say for a common use case of reflection like model building. a la entity framework, the only question that the consumer needs to ask a type is what are your inherited public properties. I mean how hard it is to ask a question like that from the type checker? It's hard. So that's what we want, we want a "Reflection API" on top of the type information that is already available in typescript.

And that's why I use tst-reflect, you can ask sensible questions from it at runtime instead of having to deal with the compilation oriented typechecker api.

So basically what you are doing is looking at the entire problem and saying that it is not solvable. But a subset of the problem is solvable. And we know from prior experience that solving a subset of the problem (emitDecoratorMetadata) helps meet a subset of use cases. (Angular and TypeORM). So yeah, this can absolutely be done to a reasonably useful level.

DeepKit is doing full-on reflection, and they've built a complete framework making use of these reflection facilities: ORM, dependency injection, configuration, routing, RPC, validation, etc. - what's interesting about this is they've not only built it, they're using it, and it has lots of interesting applications.

One major down side to this though, is the fact it's only going to work with the official TypeScript compiler - whereas probably most projects these days are actually built with Babel, ESBuild, SWC, etc... expecting all of these projects to implement full scale reflection is probably asking a lot. It might put the official compiler back in the game though?

Daniel Rosenwasser (@DanielRosenwasser)

Daniel Rosenwasser (@DanielRosenwasser) Belated 8th anniversary of this issue, and an update just as an FYI - I just received a flurry of PR's for https://github.com/akutruff/typescript-needs-types. The list of projects grows and grows. Is there really no way that the TS team could do something?

I've already resigned waiting for this. 👎

I've already started building new robust version of my tst-reflect which will support all build tools (typescript itself with all TS based bundlers as well as Vite, esbuild and SWC).

What's extra frustrating is that the language also prevents us from building our own in userspace properly. Recursive types are almost impossible to implement without the hackery Zod has to resort to. The other one is being able to conditionally add property modifiers in mapped types. #44261. The lack of first class union-detection as a utility type leads to a very deep rabbit hole as well.

I agree. I personally miss the Ability to decorate abstract members/classes/interfaces; it would be great feature for developers that use the reflection.

On the other hand I have no problem with recursive types. It wasn't difficult to handle it so that generated metadata would work as expected.

Roman Jámbor (@Hookyns)

I personally miss the Ability to decorate abstract members/classes/interfaces; it would be great feature for developers that use the reflection.

Yes, it feels like the language is inconsistent on this point. It would be natural for new developers to ask "why does this feature only work on certain types of members", and of course, if you're already a JavaScript expert, you would know why - but I'm sure an increasing number of new developers are jumping head-first into TypeScript, and this would be surprising if, for example, they've been taught Java or C# in school.

Arguably, TS is a superset of JS, so not really intended to stand alone - but still, as a language in and of itself, I think it's also arguable that TS is inheriting more limitations from JS than if the same language had been designed from the ground up.

Regardless of static typing being largely a solved problem with TS, the lack of good DI support on the server-side remains a sore point, and the main reason I still can't confidently recommend TS as a server-side language for anything large. Great for the client-side, but not so great on the server-side, in practice, at scale. More type checking improvements can't fix that.

Side note, for those who don't know, Roman Jámbor (@Hookyns) is building a reflection library for TS with ergonomics similar to C# - I'm personally pretty excited about this! 🤩

Ryan Cavanaugh (@RyanCavanaugh)

Hey all,

While I understand the appeal and potential use cases for purely-reflective or intentionally runtime-validated types, I have to say be explicit and state that this is not the direction of the language.

First and foremost, TypeScript is primarily designed as a static type checker. Its main goal is to catch type errors during development, providing early feedback and improving code quality. Adding runtime-reified types (even just for reflective purposes) would introduce a significant overhead, both in terms of compiler performance and code size. The TypeScript team has made a conscious decision to prioritize static typing over other approaches.

Second, over time we have intentionally been making room for alternative compilers. These alternative compilers have varying levels of cross-file analysis, ranging from full analysis across implementation files and declaration files, to none at all. Realistically, the only compiler that has type information available to it is the TypeScript compiler itself; but over time, our focus on emit has only waned. We believe most of the power of TypeScript comes from type-checking and tooling, and developers benefit from erasable emit that works in alternative (often faster!) compilers than ours.

While part of the discussion in this issue is in regards to a reflective type representation, many here are discussing runtime-validated types. Some discussion here is a mix, with the notion that some runtime type validation could be performed through the use of a reflective model. The high-level idea is that metadata is injected so that libraries can perform runtime checks based on the types themselves. As some point out, there is a repo listing all the runtime type-checking libraries as a sort of "proof" that this is necessary; however, the fact that each library varies in needs and scope means that some theoretically emitted metadata needs to be maximally inclusive of whatever tool might ask for it. This is a nightmare for bundle size and might be a fool's errand to implement. As Ryan Cavanaugh (@RyanCavanaugh) mentioned here, maybe tools are better off using the TypeScript API as part of this since these tools know precisely what they need.

As I've mentioned in the same thread, libraries have taken 2 approaches - either use our API to generate runtime validation code out of a set of types, or leverage TypeScript's type system to derive static types from the construction of the runtime validator itself! We think that these directions are a lot more promising than any sort of philosophical change to our emit.

We know that there's been a lot of feedback on this issue, and as such believe that there is not much new ground left to cover with further comments on the topic. To prevent a flood of notifications on everyone's inbox, we're temporarily locking this issue for 2 weeks for some pre-emptive cooldown, but further discussion can pick up at that time if needed.

Navigation Menu

Repository navigation

Discussion: (Reflective) Type Model #3628

113 remaining items

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Discussion: (Reflective) Type Model #3628

Description

Activity

danquirk commented on Jun 25, 2015

mhegazy commented on Jun 25, 2015

tobich commented on Aug 16, 2015

mhegazy commented on Aug 17, 2015

sophiajt commented on Aug 17, 2015

remojansen commented on Sep 14, 2015

mhegazy commented on Sep 14, 2015

remojansen commented on Sep 15, 2015

christyharagan commented on Sep 15, 2015

remojansen commented on Sep 21, 2015

asvetliakov commented on Sep 26, 2015

mikehaas763 commented on Oct 3, 2015

mhegazy commented on Oct 5, 2015

113 remaining items

rezonant commented on Feb 7, 2022

Hookyns commented on Feb 14, 2022

rezonant commented on Feb 23, 2022

avin-kavish commented on Sep 2, 2022

mindplay-dk commented on Oct 1, 2022

akutruff commented on Jul 10, 2023

Hookyns commented on Jul 10, 2023

akutruff commented on Jul 11, 2023

Hookyns commented on Jul 11, 2023

mindplay-dk commented on Jul 12, 2023

DanielRosenwasser commented on Jul 13, 2023

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions