Repository navigation
Iteration 2 - make the infrastructure more generic #3
Description
List of things we need to improve in this example to make it a basis for loopback-workspace 3.0.
- 1. Multiple datasources
- 2. Dynamic configuration of datasources
- 3. Reduce boilerplate, auto-load components
- a. Extract most of
api/app.api.jsto loopback core. - b. Figure out how to simplify
html5/app.html5.js - c. Refactor custom Grunt tasks into a module
- d. There should be a convention for adding new things by adding a new file, without modifying any existing code.
- a. Extract most of
- 4. Automated tests
- Integration tests for the models and the api server
- Unit-tests for the html5 client which do not communicate with the backend
- End-to-end tests - html5 client, api server, models, datasources
- 5. Mix and match components
1. Multiple datasources
The current design of api/app.api.js seems to supports a single db datasource. This is a step back from what we already have in loopback-workspace, where datasources.json can define multiple different datasources.
Here is a list of datasources used by a fully-featured application:
dbfor the main databasepushfor push notificationsemailfor sending e-mailsrestorsoapfor getting data from legacy systemsstoragefor managing images via loopback-storage-service
On the client side, most models will be attached to the same Remoting datasource. The most notable exception is synchronization, it requires several datasources (local storage, remoting).
(SLA-1023)
2. Dynamic configuration of datasources
The current design makes it easy to change the db configuration depending on environment variables. While this is useful for demo purposes, our current direction regarding deployment & operations favours different approach - see the gist.
Configuration should not be present in a node application archive. The more configuration is bound in, the less choice there will be over deployment environment. Even deploying the same app to staging and then production could become impossible.
We need to keep in mind the use case of loopback-workspace, which should be able to load a list of available datasource without the need to build the project and load it to a Node.js process via require('app.js').
(SLA-1023)
3. Reduce boilerplate, auto-load components
The current version contains too much of boilerplate code and makes it impossible to add new features/models/datasources via loopback-workspace.
Here are few generators that cannot be implemented now:
- add a new model
- add a new datasource
- enable explorer (swagger)
- enable authentication
What is needed:
a. Extract most of api/app.api.js to loopback core. Ideally the existing method app.boot can be modified or extended to support the new layout.
b. Figure out how to simplify html5/app.html5.js. The code to require() all models should be generated by gulpfile, same applies to the code calling require() for all angular controller files.
c. (SLA-938) Refactor most of gulpfile into a module gulp-loopback, so that the application gulpfile contains only calls to gulp-loopback and application-specific options.
d. There should be a convention for adding new things by adding a new file, without modifying any existing code. All components (api, html5, web) should support these conventions, either directly in code (think of app.boot) or via a custom build step. (See also b.)
4. Automated tests
The pull request #1 touched the subject lightly, but no solution was implemented.
I would like to see at least these three categories of tests covered by the example project:
a. (SLA-841) Integration tests for the models and the api server, the tests are permitted to use the database (probably the real one, possibly an in-memory replacement). These tests should run on Node.js.
b. (SLA-1019) Unit-tests for the html5 client which do not communice with the backend. These tests should run in a browser via Karma.
c. (SLA-1020) End-to-end tests - html5 client, api server, models, datasources. There are two flavours: in the first one, developers write unit-test-like code, but the communication with backend is not mocked out. The second flavour tests the application via selenium or a similar framework (e.g. angular's protractor). Both flavours require to start the server before running the tests in the browser.
5. Mix and match components
To keep the loopback-workspace code simple, we should come up with a single structure flexible enough to support different kinds of projects:
- full-stack server + client as presented here:
api,models,html5,web. - a pure API server:
api,models - a website with REST API:
api,models,web - (optional) a website without REST API:
models,web
The current design is pretty close, but not there yet.
Activity
[-]Iteration 2[/-][+]Iteration 2 - make the infrastructure more generic[/+]ritch commentedon May 20, 2014
I'm afraid it won't always be this simple. Maybe as a default? Take a look at the replication branch for an example of the client attaching models to various data sources.
ritch commentedon May 20, 2014
Yes.
I guess it is a lot trickier to map the metadata back to a file if we are loading it from somewhere else. But we have to take into account models / relations / etc defined only at runtime - or design the workspace / ui in a way that makes it clear these aren't taken into account.
ritch commentedon May 20, 2014
Good ideas here.
Agreed. Help here would be appreciated.
rmg commentedon May 20, 2014
The runtime configuration described in the gist that advocates an external configuration server is more about hostnames, ports to listen on, and sensitive credentials. The kind of details that should not necessarily be committed to the application's code base.
bajtos commentedon May 21, 2014
I have updated the issue description.
Yesterday in our chat, we agreed the best solution is to keep the same list of datasources, but allow datasources to have different configuration on the server and on the client.
bajtos commentedon May 29, 2014
Items 1 & 2 were implemented in #7.
bajtos commentedon Jun 2, 2014
Done in #7, #16.
bajtos commentedon Jun 27, 2014
Done in #37.
bajtos commentedon Jun 27, 2014
The build uses Grunt now. There are two non-trivial custom tasks (
build-configandrun) that are probably worth extracting to a new module.bajtos commentedon Jun 27, 2014
This is done as of #33 and #37.
bajtos commentedon Oct 29, 2014
Closing the issues as we don't plan to work on the integration/end-to-end tests now. We will create a new issue for that when the time comes.
Merge connector update from PR strongloop#3