Skip to content
This repository was archived by the owner on Apr 18, 2020. It is now read-only.
This repository was archived by the owner on Apr 18, 2020. It is now read-only.

Move models.json into /models #13

Closed

Description

@ritch

/to @bajtos @raymondfeng @seanbrookes

From a previous discussion

There should be a convention for adding new things by adding a new file, without modifying any existing code.

I really like this approach. I think we should adopt it for models as well. This would mean all model definitions would exist as models/*.json.

For this app it would look like:

/models
  /todo.json
  /todo.js
  /user.json
  ...

todo.json would be the typical models.json definition just in a separate file.

todo.js would be the extensions to the todo definition. One simplification we might want to do is the todo.js file could define the model by requiring the todo.json file.

// todo.js
var Todo = module.exports = loopback.PersistedModel.extend('Todo', require('./todo.json'));

This would close the gap between hand written models and generated models. There isn't a lot of boilerplate that we would have to generate either. We would have to generate the additional files. This should be simple to add to the workspace.

.extend() could support the definition format of todo.json without much modification.

Activity

  1. seanbrookes commented on May 29, 2014

    @seanbrookes

    this makes sense to me as well as it makes it cleaner how to extend your
    model definitions in an elegant fashion

    On Thu, May 29, 2014 at 11:01 AM, Ritchie Martori
    notifications@github.comwrote:

    /to @bajtos https://github.com/bajtos @raymondfenghttps://github.com/raymondfeng
    @seanbrookes https://github.com/seanbrookes

    From a previous discussionhttps://github.com//issues/3#issuecomment-44499682

    There should be a convention for adding new things by adding a new file,
    without modifying any existing code.

    I really like this approach. I think we should adopt it for models as
    well. This would mean all model definitions would exist as models/*.json.

    For this app it would look like:

    /models
    /todo.json
    /todo.js
    /user.json
    ...

    todo.json would be the typical models.json definition just in a separate
    file.

    todo.js would be the extensions to the todo definition. One
    simplification we might want to do is the todo.js file could define the
    model by requiring the todo.json file.

    // todo.jsvar Todo = module.exports = loopback.PersistedModel.extend('Todo', require('./todo.json'));

    This would close the gap between hand written models and generated models.
    There isn't a lot of boilerplate that we would have to generate either. We
    would have to generate the additional files. This should be simple to add
    to the workspace.

    .extend() could support the definition format of todo.json without much
    modification.

    —
    Reply to this email directly or view it on GitHubhttps://github.com//issues/13
    .

    StrongLoop http://strongloop.com/_\* makes it easy to develop APIs
    http://strongloop.com/mobile-application-development/loopback/
    in Node,
    plus get DevOps capabilities
    http://strongloop.com/node-js-performance/strongops/ like monitoring,
    debugging and clustering*
    ._

  2. bajtos commented on May 29, 2014

    @bajtos
    Member

    all model definitions would exist as models/*.json.

    👍

    One simplification we might want to do is the todo.js file could define the model by requiring the todo.json file.

    I have mixed feelings about this. On one hand, we are forcing the user code to include boilerplate for defining models. On the other hand, with the existing solution, users have to write var Todo = loopback.getModel('Todo') before they can extend, which is just a different kind of boilerplate.

    Besides that, this code is not enough:

    loopback.PersistedModel.extend('Todo', require('./todo.json'));
    
    1. createModel(name, properties, options) - you have to process todo.json and build properties and options objects.
    2. The model is not attached to a datasource which was configured in todo.json, neither it is registered with the application so that it can be exposed via REST API.

    I want to look at this in more detail tomorrow and come up with a proposal that would solve those issues.


    Here is an important discussion related to his thread: comment in #7.

    It touches two important subjects:

    • the global registry of models, which is used by the example presented here, but which we would like to replace with dependency injection in the long run.
    • possibility to package a set of Models into a standalone reusable npm package.
  3. self-assigned this
    on May 30, 2014
  4. added this to the 1.0 milestone on May 30, 2014
  5. bajtos commented on May 30, 2014

    @bajtos
    Member

    IMO the setup of models has two parts:

    1. Definition of Model classes.
    2. Configuration of app models.

    To define a Model, one has to specify:

    • The base class (Model or PersistedModel).
    • Model properties.
    • Options like strict and hidden.
    • Potentionally some of the relations.

    In many cases, the model class needs to be extended with extra methods and optionally modified via javascript code, since some settings are not exposed via the declarative API (e.g. computed default values, hooks).

    To add a Model to an application, the following is needed:

    • Attach the model to a given datasource.
    • Add more relations and/or override the targets of existing relations. Think of AccessToken belongsTo User provided by LoopBack core, that is overrided by the application to AccessToken belongsTo Customer, where Customer is extending User.
    • Override model options like public.
    • Override model-method metadata like public.

    The solution for both parts should use JSON declarations as much as possible, so that loopback-workspace, yo generators and Studio can easily read and modify the settings.

    Definition of model classes

    Filesystem layout

    Model definitions should be kept in a directory organized in a module-like structure. The directory should provide an index.js file exposing all models.

    The model files should live in a lib subdirectory: it is the convention used for node modules, it makes it easier to automatically list all Model files, and it prevents the situation where package.json can be interpreted as both model definition and npm module description.

    models/
      lib/*
      index.js
      package.json # optional
    

    Each model file should define a single model and register it with loopback (that's the default behaviour of Model.extend or loopback.defineModel now).

    Accesing the loopback singleton

    When the models are distributed as a standalone-package, it's important to ensure we register the models with the correct instance of the loopback module. This means we cannot access the loopback object via require('loopback'), but the object has to be injected from outside.

    See the discussion in loopback#275:

    Best fix is for the model registry to be provided to a connector/juggler when
    it is instantiated, so there is no need to mis-use peer deps to attempt to
    guarantee singleton-ness of the juggler (this doesn't work under a number of
    conditions).

    // models/index.js
    module.exports = function(loopback) {
      require('./lib/todo.js')(loopback);
    };
    
    // models/lib/todo.js
    module.exports = function(loopback) {
      loopback.createModelFromConfig('Todo', require('./todo.json'));
    };
    

    Defining the model

    The function createModelFromConfig presented in the code above should extract properties and options from the config object and pass them to loopback.createModel.

    Experienced developers can define the models entirely in code by building the config structure by hand or directly calling loopback.createModel.

    Studio and yo generators will:

    • scaffold todo.js and todo.json when a new model todo is created
    • modify todo.js to alter the configuration of an existing todo model

    Adding a model to an app

    A loopback app typically uses models from multiple sources:

    • built-in loopback models
    • models from shared modules
    • application-specific models

    E.g. the full-stack example client uses User from loopback, Todo from models shared between the server and the client, and LocalTodo which is specific for the client supporting offline access.

    I am proposing to define application-specific models using the same mechanism as for shared models.

    html5/
      app.html5.js
      models/
        index.js
        lib/
          local-todo.json
          local-todo.js
    models/
      index.js
      lib/
        todo.js
        todo.json
    

    Once we have all models defined (registered with loopback), we need to tweak their configuration for the application as mentioned in the beginning:

    • Attach the model to a given datasource.
    • Add more relations and/or override the targets of existing relations. Think of AccessToken belongsTo User provided by LoopBack core, that is overridden by the application to AccessToken belongsTo Customer (Customer is extending User).
    • Override model options like public.
    • Override model-method metadata like public.

    All of that should be configured in a JSON file (or more files). Since most models will have only one setting - datasource name, it should be fine to keep this configuration in a single file.

    {
      "Todo": {
        "datasource": "db",
      },
      "AccessToken": {
        "datasource": "db",
        "public": false
        "relations": {
          "user": {
            "model": "Customer"
          }
        },
      },
      "Customer": {
        "datasource": "db"
    
        "relations": {
          // relation hasMany AccessToken
          // was defined in model definition
          // no need to repeat it here
        },
    
        "remote-methods": {
          "create": {
            // modify metadata of Customer.create
            // and hide the method from the REST API
            "public": false
          }
        }
      },
    }

    Note that all entries in this file are referencing models that must already exist (i.e. can be resolved by loopback.getModel). Settings like properties and options cannot be used here.

    To prevent confusion, the file must use a different name from models.json,
    e.g. app-models.json.

    Work items

    1. Implement loopback.createModelFromConfig()
    2. Modify loopback-workspace to support the new models/*.json layout: read model definitions from multiple files, write the changed model definition to the correct file. Update app-models.json when a new model is added or an existing model was attached to a different datasource.
    3. Implement migration script for upgrading apps based on models.json to the new structure.
    4. Implement new loopback methods for applying model modifications supported by app-models.json. Modify loopback-boot to read app-models.json and apply the config by call appropriate loopback methods.

    Notes

    N1

    The currently implemented way of loading models has a significant flaw.

    Consider a models.json defining two models Todo and LocalTodo, where LocalTodo is based on Todo. There is also models/todo.js that adds Todo.stats = function(cb).

    Becase LocalTodo is defined as Todo extension before models/todo.js is loaded, the method stats is not inherited.

    The proposal presented above solves this problem, since the developer controls the order in which the models are require().

    N2

    The proposal will simplify the build process for browserified applications, as there are no dynamic requires needed. The boot process can be described like this:

    var app = loopback();
    // shared models
    require('../models')(loopback);
    // client models
    require('./models')(loopback);
    configureAppModels(app, require('./app-models.json'));
  6. bajtos commented on May 30, 2014

    @bajtos
    Member

    @ritch @seanbrookes what's your opinion on my proposal in the previous comment?

  7. ritch commented on Jun 3, 2014

    @ritch
    MemberAuthor

    All of that should be configured in a JSON file (or more files). Since most models will have only one setting - datasource name, it should be fine to keep this configuration in a single file.

    I think most models will also have relations. Independent models are rare. In this app there should be a relation: User.hasMany(Todo) and Todo.belongsTo(User).

  8. ritch commented on Jun 3, 2014

    @ritch
    MemberAuthor
    // models/index.js
    module.exports = function(loopback) {
      require('./lib/todo.js')(loopback);
    };
    
    // models/lib/todo.js
    module.exports = function(loopback) {
      loopback.createModelFromConfig('Todo', require('./todo.json'));
    };

    I think this is essentially half baked dependency injection. We need a clean / clear way to provide the loopback instance globally.

    I'm not sure what advantage the above gives us over using what we have available right now? We can implement a single modelBuilder instance per app as well using the following.

    var app = require('../app');
    app.model('Todo', require('./todo.json'));

    The only issue I see is that loopback core and extensions make extensive use of

    loopback.getModel();

    This isn't enough information to get the right modelBuilder instance. Instead we can replace this with the following.

    // basic api
    app.getModel('MyModel'); // delegates to app.modelBuilder.getModel()
    
    // this would require all models to be attached to apps (not much of a downside here)
    SomeModel.getApp().getModel('MyModelName');
    
    // shortcut
    SomeModel.getModel('MyModel');

    If a reference to an app is not available, one will have to be provided. This will be a non trivial change, but not a major one either. A benefit to this approach is that refactoring the loopback tests to use this would be straightforward.

  9. bajtos commented on Jun 4, 2014

    @bajtos
    Member

    I'm not sure what advantage the above gives us over using what we have available right now?

    You can bundle your modules in an npm package and share them via npm registry.

    When the models are in their own package, there is no safe way how to access global singletons. You have to inject something into the module, to let the code find objects like app, loopback or SomeModel.

    I think this is essentially half baked dependency injection.

    Why half baked, what are you missing?

    We need a clean / clear way to provide the loopback instance globally.

    Globals are evil, it's be better to avoid them.

    var app = require('../app');
    app.model('Todo', require('./todo.json'));
    

    This does not work in isomorphic projects, as there are multiple apps you are going to attach the model (API server, browser client).


    IMO there are two issues in the current design of loopback:

    1. The model registry is a global singleton and there is no (easy) way how to create models locally without affecting the global registry.
    2. Existing loopback extensions use require('loopback') to access this global singleton. To make that work, projects must ensure there is only a single instance of loopback module, which may not be always possible.

    My proposal is addressing point 2 by introducing a lightweight dependency-injections pattern.

    The point 1 is IMO out of scope of this pull request. As long as the solution provides an object that has the same API as the current loopback singleton, the solution presented in my proposal will continue to work.

  10. bajtos commented on Jun 4, 2014

    @bajtos
    Member

    Here is an alternative way: what if we get rid of the singleton model registry, let each each module build (and export) its own model registry, and provide a way how to merge model definitions from multiple registries.

    loopback-models

    • provides base classes Model and PersistedModel
    • adds pre-built models User, AccessToken, etc.

    example-full-stack-models

    • depends on loopback-models
    • provides Todo model extending PersistedModel
    • adds relation User hasMany Todos

    loopback-passport

    • depends on loopback-models (possible a different instance than in example-full-stack-models)
    • provides UserIdentity model extending PersistedModel (and other models)
    • adds relation User hasMany UserIdentities

    example-full-stack

    • depends on example-full-stack-models (Todo), loopback-passports (UserIdentity,...) and loopback-models (User, AccessToken, ACL)
    • the app uses cherry-picked subset of models defined by the three modules above

    Caveats: How to resolve relations when there are multiple instances of base models? E.g. both Todo and UserIdentity has a relation to the User model.

    @ritch @raymondfeng It will be best to discuss this online, I'll try to catch you in next few days.

  11. jbcpollak commented on Feb 15, 2016

    @jbcpollak

    Sorry to revive an old ticket: I'm looking for best practices or an example of how to share models between multiple server-side projects. The externalized projects @bajtos mentioned in his last post, like example-full-stack-models doesn't exist. I found strong-mesh-models, but it seems to actually contain a full loopback server and export that, instead of just exporting the models.

    Are external models possible?

  12. superkhau commented on Feb 15, 2016

    @superkhau
    Contributor

    @jbcpollak Please create your own issue instead of reviving this old one. In addition, please post questions at https://groups.google.com/forum/#!forum/loopbackjs. See https://github.com/strongloop/loopback/wiki/Questions for more details.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions