You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
At the moment, we do not support ObjectId in model definitions. I am creating this issue to gather some opinions on how to move forward for a bigger fix than the one at loopbackio/loopback-datasource-juggler#778.
The aforementioned fix depends on the native mongodb driver's ObjectId implementation (simply delegates to it for the time being). The following are some interesting items:
Dependency on the the native mongodb impl
@bajtos@ritch and @superkhau do not like this solution for the long term. My suggestion is to probably use our own class impl that uses the strategy pattern to allow swappable ObjectId impl's. If none is specified, it will fall back to our own default impl, which is probably going to be a simple subset of Mongo's impl.
@ritch suggested that we have a registry for datatypes
Basically, we should have an app level registry for all datatypes that are currently valid in the system. It should be updateable during runtime. Something along the lines of:
With regards to impl, I'm not sure how to approach this yet. Maybe we can have LoopBack boot register dataTypes based on the ones read in from model definitions at startup. Ultimately though, I think Juggler should be the one with the main list of data types and the app level is just a link to the one maintained in Juggler.
How to persist the data consistently across SQL and NoSQL databases
This is an interesting question because we need to support persistence in both types of databases. For NoSQL dbs that support ObjectIds, the solution is simple. For SQL databases, do we persist them as String PKs when marshalling/unmarshalling or do we use another format?
This is interesting and probably dependent on how we choose to serialize/deserialize the ObjectId type. We could do deep comparison or simple string comparison via the hex string, etc. Not sure what is the best route here. In addition, do we support multi-level relations, etc.
One more thing to consider: how to support models with ObjectId properties when running in a browser, where we don't have any mongodb connector?
We should not include the full mongodb module in the browser bundle, this is something we should probably fix in the current short-term fix ASAP.
Since there is no mongodb module in the browser that could contribute the ObjectId type, we don't have any other choice than to implement our own ObjectId implementation that would be mapped to MongoDB's ObjectId by the MongoDB connector.
That way it should be reasonable easy to implement custom mapping for SQL and foreign keys too.
We should not include the full mongodb module in the browser bundle, this is something we should probably fix in the current short-term fix ASAP.
How do we fix this? Come up with our own impl now?
Since there is no mongodb module in the browser that could contribute the ObjectId type, we don't have any other choice than to implement our own ObjectId implementation that would be mapped to MongoDB's ObjectId by the MongoDB connector.
Not sure how to do this in the browser. It sounds like if the browser is a requirement, we need to impl our own ObjectId type that is not dependent on the mongodb module. The short term-fix doesn't take this use case into consideration at all since it just uses the mongodb module directly. Is the browser requirement a must have at this point in time? Or can it be patched in a future update?
Not sure how to do this in the browser. It sounds like if the browser is a requirement, we need to impl our own ObjectId type that is not dependent on the mongodb module. The short term-fix doesn't take this use case into consideration at all since it just uses the mongodb module directly. Is the browser requirement a must have at this point in time? Or can it be patched in a future update?
We must allow users to write universal/isomorphic LoopBack apps that can run in browser (backed by localStorage/indexedDB) and on the server (backed e.g. by MongoDB).
I don't know what are the implications of using GUID-like string ids in models stored in MongoDB. If using ObjectID instead of string would be more efficient, or perhaps if using ObjectId instead of string in browser models would make things easier, then I think we should create a new user story to investigate these options.
It makes me really wonder: what do we actually need from ObjectId, besides the type information that it's a string in a specific format? Why can't we simply implement our ObjectId as a thin wrapper around string with a bit of extra validation?
I realized that Loopback use toString to "hide" objects, which makes it very complicated to debug/understand what is happening. He are some examples:
console.log(accessToken.uid) gonna display MyUserId. One might think that accessToken.uid is a string when it actually is an object using toString() to return the actual value, which makes the following fail: accessToken.uid === "MyUserId"
The behavior was introduced by loopbackio/loopback-datasource-juggler#508. It fixes a security hole that potentially allows non-owner to mess up with the id. You can remove the id before it's sent to the backend.
The id property cannot be set in the data argument. We also allow the id if it is the same as the instance id. In the case of MongoDB, the id type is ObjectId, which doesn't always satisfy ===.
Came across this same issue and have noticed that's plaguing the mongodb connector issues ATM.
I was able to solve 99% of my querys and nested querys by adding the objectid datatype especially while trying to communicate between servers using the remote-connector.
I made a module to add the datatype but should probably be provided with the mongo connector
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
This issue has been closed due to continued inactivity. Thank you for your understanding. If you believe this to be in error, please contact one of the code owners, listed in the CODEOWNERS file at the top-level of this repository.
Activity
superkhau commentedon Dec 10, 2015
The aforementioned fix depends on the native mongodb driver's ObjectId implementation (simply delegates to it for the time being). The following are some interesting items:
@bajtos @ritch and @superkhau do not like this solution for the long term. My suggestion is to probably use our own class impl that uses the strategy pattern to allow swappable ObjectId impl's. If none is specified, it will fall back to our own default impl, which is probably going to be a simple subset of Mongo's impl.
See https://github.com/strongloop-internal/scrum-loopback/issues/534.
Basically, we should have an app level registry for all datatypes that are currently valid in the system. It should be updateable during runtime. Something along the lines of:
With regards to impl, I'm not sure how to approach this yet. Maybe we can have LoopBack boot register dataTypes based on the ones read in from model definitions at startup. Ultimately though, I think Juggler should be the one with the main list of data types and the app level is just a link to the one maintained in Juggler.
superkhau commentedon Dec 10, 2015
This is an interesting question because we need to support persistence in both types of databases. For NoSQL dbs that support ObjectIds, the solution is simple. For SQL databases, do we persist them as String PKs when marshalling/unmarshalling or do we use another format?
superkhau commentedon Dec 10, 2015
@bajtos @ritch @raymondfeng Please add your insights (especially @raymondfeng since you know database architecture/design pretty well).
superkhau commentedon Dec 10, 2015
This is pretty simple in that we can just do something like:
Note we should also alias
IdandIDdefensively (saw this in mongo's native impl).superkhau commentedon Dec 10, 2015
This is interesting and probably dependent on how we choose to serialize/deserialize the ObjectId type. We could do deep comparison or simple string comparison via the hex string, etc. Not sure what is the best route here. In addition, do we support multi-level relations, etc.
superkhau commentedon Dec 10, 2015
Ultimately, I think the long term proposal would involve this major items:
appobject which delegates to the registry in Jugglerbajtos commentedon Dec 14, 2015
One more thing to consider: how to support models with
ObjectIdproperties when running in a browser, where we don't have any mongodb connector?We should not include the full
mongodbmodule in the browser bundle, this is something we should probably fix in the current short-term fix ASAP.Since there is no
mongodbmodule in the browser that could contribute theObjectIdtype, we don't have any other choice than to implement our own ObjectId implementation that would be mapped to MongoDB's ObjectId by the MongoDB connector.That way it should be reasonable easy to implement custom mapping for SQL and foreign keys too.
Thoughts?
superkhau commentedon Dec 14, 2015
How do we fix this? Come up with our own impl now?
Not sure how to do this in the browser. It sounds like if the browser is a requirement, we need to impl our own ObjectId type that is not dependent on the
mongodbmodule. The short term-fix doesn't take this use case into consideration at all since it just uses themongodbmodule directly. Is the browser requirement a must have at this point in time? Or can it be patched in a future update?bajtos commentedon Jan 4, 2016
We must allow users to write universal/isomorphic LoopBack apps that can run in browser (backed by localStorage/indexedDB) and on the server (backed e.g. by MongoDB).
At the moment, we require sync-ed models to use
id: { type: 'string', defaultFn: 'guid' }and it seems to work reasonably well. So for short-term, all we need is to excludemongodbfrom the browser bundle, e.g. by providing a browser-specific implementation of ObjectID that can e.g. throw an exception in the constructor. We already do something similar fordepd, see https://github.com/strongloop/loopback-datasource-juggler/blob/5cf37aae5b217a92ed213c89884156dd42c1c0df/package.json#L22-L24I don't know what are the implications of using GUID-like string ids in models stored in MongoDB. If using
ObjectIDinstead ofstringwould be more efficient, or perhaps if usingObjectIdinstead ofstringin browser models would make things easier, then I think we should create a new user story to investigate these options.It makes me really wonder: what do we actually need from
ObjectId, besides the type information that it's a string in a specific format? Why can't we simply implement ourObjectIdas a thin wrapper aroundstringwith a bit of extra validation?bajtos commentedon Apr 6, 2016
Possibly related: #1994
bajtos commentedon May 13, 2016
Possibly related: strongloop/loopback-sdk-angular#134
BoLaMN commentedon May 27, 2017
Came across this same issue and have noticed that's plaguing the mongodb connector issues ATM.
I was able to solve 99% of my querys and nested querys by adding the objectid datatype especially while trying to communicate between servers using the remote-connector.
I made a module to add the datatype but should probably be provided with the mongo connector
https://github.com/BoLaMN/loopback-datatype-objectid
I've been using https://github.com/BoLaMN/loopback-angular-sdk-module/blob/master/src/index.coffee#L13 to generate objectids client side
may help some people!
thought id reference some more issues to link in
loopbackio/loopback-connector-mongodb#357 loopbackio/loopback-connector-mongodb#338 loopbackio/loopback-connector-mongodb#316 loopbackio/loopback-connector-mongodb#186 loopbackio/loopback-datasource-juggler#1386 #274 #3394
semi related
loopbackio/loopback-connector-mongodb#302
stale commentedon Aug 23, 2017
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
stale commentedon Sep 6, 2017
This issue has been closed due to continued inactivity. Thank you for your understanding. If you believe this to be in error, please contact one of the code owners, listed in the
CODEOWNERSfile at the top-level of this repository.