Skip to content
This repository was archived by the owner on Apr 18, 2020. It is now read-only.
This repository was archived by the owner on Apr 18, 2020. It is now read-only.

Iteration 2 - make the infrastructure more generic #3

Closed

Description

@bajtos

List of things we need to improve in this example to make it a basis for loopback-workspace 3.0.

  • 1. Multiple datasources
  • 2. Dynamic configuration of datasources
  • 3. Reduce boilerplate, auto-load components
    • a. Extract most of api/app.api.js to loopback core.
    • b. Figure out how to simplify html5/app.html5.js
    • c. Refactor custom Grunt tasks into a module
    • d. There should be a convention for adding new things by adding a new file, without modifying any existing code.
  • 4. Automated tests
    • Integration tests for the models and the api server
    • Unit-tests for the html5 client which do not communicate with the backend
    • End-to-end tests - html5 client, api server, models, datasources
  • 5. Mix and match components

1. Multiple datasources

The current design of api/app.api.js seems to supports a single db datasource. This is a step back from what we already have in loopback-workspace, where datasources.json can define multiple different datasources.

Here is a list of datasources used by a fully-featured application:

  • db for the main database
  • push for push notifications
  • email for sending e-mails
  • rest or soap for getting data from legacy systems
  • storage for managing images via loopback-storage-service

On the client side, most models will be attached to the same Remoting datasource. The most notable exception is synchronization, it requires several datasources (local storage, remoting).

(SLA-1023)

2. Dynamic configuration of datasources

The current design makes it easy to change the db configuration depending on environment variables. While this is useful for demo purposes, our current direction regarding deployment & operations favours different approach - see the gist.

Configuration should not be present in a node application archive. The more configuration is bound in, the less choice there will be over deployment environment. Even deploying the same app to staging and then production could become impossible.

We need to keep in mind the use case of loopback-workspace, which should be able to load a list of available datasource without the need to build the project and load it to a Node.js process via require('app.js').

(SLA-1023)

3. Reduce boilerplate, auto-load components

The current version contains too much of boilerplate code and makes it impossible to add new features/models/datasources via loopback-workspace.

Here are few generators that cannot be implemented now:

  • add a new model
  • add a new datasource
  • enable explorer (swagger)
  • enable authentication

What is needed:

a. Extract most of api/app.api.js to loopback core. Ideally the existing method app.boot can be modified or extended to support the new layout.

b. Figure out how to simplify html5/app.html5.js. The code to require() all models should be generated by gulpfile, same applies to the code calling require() for all angular controller files.

c. (SLA-938) Refactor most of gulpfile into a module gulp-loopback, so that the application gulpfile contains only calls to gulp-loopback and application-specific options.

d. There should be a convention for adding new things by adding a new file, without modifying any existing code. All components (api, html5, web) should support these conventions, either directly in code (think of app.boot) or via a custom build step. (See also b.)

4. Automated tests

The pull request #1 touched the subject lightly, but no solution was implemented.

I would like to see at least these three categories of tests covered by the example project:

a. (SLA-841) Integration tests for the models and the api server, the tests are permitted to use the database (probably the real one, possibly an in-memory replacement). These tests should run on Node.js.

b. (SLA-1019) Unit-tests for the html5 client which do not communice with the backend. These tests should run in a browser via Karma.

c. (SLA-1020) End-to-end tests - html5 client, api server, models, datasources. There are two flavours: in the first one, developers write unit-test-like code, but the communication with backend is not mocked out. The second flavour tests the application via selenium or a similar framework (e.g. angular's protractor). Both flavours require to start the server before running the tests in the browser.

5. Mix and match components

To keep the loopback-workspace code simple, we should come up with a single structure flexible enough to support different kinds of projects:

  • full-stack server + client as presented here: api, models, html5, web.
  • a pure API server: api, models
  • a website with REST API: api, models, web
  • (optional) a website without REST API: models, web

The current design is pretty close, but not there yet.

Activity

  1. changed the title [-]Iteration 2[/-] [+]Iteration 2 - make the infrastructure more generic[/+] on May 20, 2014
  2. ritch commented on May 20, 2014

    @ritch
    Member

    On the client side, all models should be attached to a single Remoting datasource.

    I'm afraid it won't always be this simple. Maybe as a default? Take a look at the replication branch for an example of the client attaching models to various data sources.

  3. ritch commented on May 20, 2014

    @ritch
    Member

    We need to keep in mind the use case of loopback-workspace,

    Yes.

    which should be able to load a list of available datasource without the need to build the project and load it to a Node.js process via require('app.js').

    I guess it is a lot trickier to map the metadata back to a file if we are loading it from somewhere else. But we have to take into account models / relations / etc defined only at runtime - or design the workspace / ui in a way that makes it clear these aren't taken into account.

  4. ritch commented on May 20, 2014

    @ritch
    Member

    3 Reduce boilerplate, auto-load components

    Good ideas here.

    4 Automated tests

    Agreed. Help here would be appreciated.

  5. rmg commented on May 20, 2014

    @rmg
    Member

    The runtime configuration described in the gist that advocates an external configuration server is more about hostnames, ports to listen on, and sensitive credentials. The kind of details that should not necessarily be committed to the application's code base.

  6. bajtos commented on May 21, 2014

    @bajtos
    MemberAuthor

    On the client side, all models should be attached to a single Remoting datasource.

    I'm afraid it won't always be this simple. Maybe as a default? Take a look at the replication branch for an example of the client attaching models to various data sources.

    I have updated the issue description.

    Yesterday in our chat, we agreed the best solution is to keep the same list of datasources, but allow datasources to have different configuration on the server and on the client.

  7. bajtos commented on May 29, 2014

    @bajtos
    MemberAuthor

    Items 1 & 2 were implemented in #7.

  8. added this to the 1.0 milestone on May 30, 2014
  9. bajtos commented on Jun 2, 2014

    @bajtos
    MemberAuthor

    3.a. Extract most of api/app.api.js to loopback core.

    Done in #7, #16.

  10. removed this from the 1.0 milestone on Jun 27, 2014
  11. bajtos commented on Jun 27, 2014

    @bajtos
    MemberAuthor

    3.b. Figure out how to simplify html5/app.html5.js

    Done in #37.

  12. bajtos commented on Jun 27, 2014

    @bajtos
    MemberAuthor

    3.c. Refactor most of gulpfile into a module

    The build uses Grunt now. There are two non-trivial custom tasks (build-config and run) that are probably worth extracting to a new module.

  13. bajtos commented on Jun 27, 2014

    @bajtos
    MemberAuthor

    3.d. There should be a convention for adding new things by adding a new file, without modifying any existing code.

    This is done as of #33 and #37.

  14. bajtos commented on Oct 29, 2014

    @bajtos
    MemberAuthor

    Closing the issues as we don't plan to work on the integration/end-to-end tests now. We will create a new issue for that when the time comes.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions