Support nested JSON objects

E.g.:

{
  name: {
    first: 'Devin',
    last: 'Torres'
  }
}

Thoughts:

name: DS.attr('object')
firstName: DS.attr('string', { key: 'name.first' })

Justification:

Nested JSON objects are too prevalent to require transforming them every time they're used. Models should model the data, not some transformed representation you morph in adapter, and hopefully not through hundreds of transformers.

@devinus

Even if I was arguing with @devinus on the subject I am actually +1 on this :)
If the idea is approved I could submit a patch

I'm +1 on this as well. This would be especially nice for people using non-relational databases like couch.

Or ElasticSearch in my case.

Another idea:

name: DS.attr({
  first: DS.attr('string'),
  last: DS.attr('string')
})

Hmm, the problem with this future is potential overhead on write. Read
is juste getPath, but on write, we would have to walk throu the whole
data hash, creating missing nodes...

On 19 janv. 2012, at 20:02, Devin Torres
reply@reply.github.com
wrote:

Another idea:

name: DS.attr({
first: DS.attr('string'),
last: DS.attr('string')
})

Reply to this email directly or view it on GitHub:
#53 (comment)

@tomdale

Could I have a quick comment on this from either @tomdale or @wycats so I could possibly move forward with a patch?

Is this for read-only cases or do you expect to write them back to the server as well? How do you detect when the top-level object is dirtied?

You should be able to create multiple embedded models. Given the JSON data:

{
  name: "Steve",

  address: {
    street: "1 Infinite Loop",
    country: {
      short_name: "USA",
      long_name: "United States"
    }
  }
}

You should be able to model this with the following models:

Country = DS.Model.extend({
  shortName: DS.attr('string', { key: 'short_name' }),
  longName: DS.attr('string', { key: 'long_name' })
});

Address = DS.Model.extend({
  street: DS.attr('string'),
  country: DS.hasOne(Country, { embedded: true })
});

Person = DS.Model.extend({
  name: DS.attr('string'),
  address: DS.hasOne(Address, { embedded: true })
});

Are you arguing that this is too much ceremony? It seems like you actually do want each of these conceptually to be separate models, so you can track dirty states separately.

I'm arguing that it can end up being too much ceremony, yes. For example, sometimes I may get back nested JSON that doesn't map well to hasOne attributes. For example:

{
  name: {
    first: 'Devin',
    middle: 'Alexander',
    last: 'Torres'
  }
}

Am I supposed to create a Name model for something like this? These things also aren't coming back with unique id's by the way, how does hasOne handle that?

Since you are dealing with crazy, non-conventional JSON anyhow, couldn't you normalize the data in your adapter before you send it to the store?

This kind of nested JSON is all over the place: CouchDB, ElasticSearch, Neo4j... I'd argue that there's nothing crazy or unconventional about it. I'd also argue normalizing data in an adapter isn't what an adapter should be for either. I've had thoughts of maybe introducing some type of Transformer, but really--shouldn't the model model your data and not the other way around? Your thoughts?

@devinus

+1 ... agree with @devinus . sometimes nested json properties are actually part of the "top-level object"... so then if any property inside of them changes, it would "dirty" the whole thing. and yes, for our app some of these nested-json sections are indeed being serialized and sent back to server

currently, can't use hasOne because nested models dont have "primary keys". could fake it in adapter, as mentioned... but that's extra work to make two unrelated concepts fit together (these aren't separate "database objects", which is really what hasOne/hasMany is doing for us.... so I'd be working around the framework)

will find another workaround for the timebeing...

Are you willing to write out the entire object when it changes, as opposed to being able to modify the object directly? Something like:

Person = DS.Model.extend({
  attrs: DS.attr('object')
});

var person = Person.find(1);

var attrs = person.modify('attrs', function(attrs) {
  attrs.age++;
});

// this would be illegal
person.get('attrs').age++;

The reason for this is that if you modify a conceptually whole attribute by mutating it directly, we don't know that the attribute has changed. We don't want to observe changes to the hash directly, because that may mutate the object in ways that harm its ability to be serialized back (for example, observers add on ES5 accessors to warn users who try to access them directly that they should be going through set/get; even without that, the object needs to be annotated with some extra observability information, which changes it from being a raw, easily serializable object to an extended object).

With that said, would you be comfortable with a .modify API (or something like it; open for suggestions here) for object-like attributes? If so, I think we could arrange it. Unfortunately, because we don't want to directly observe the object, it won't be easy for us to detect mistakes where people try to directly modify it :/ (we wouldn't want to make geting illegal, of course). One thing we could possibly do would be to use ES5 seal where available to try to trap mistakes, but the error messages won't necessarily be easy to figure out (because they'd come from the browser, not Ember).

Suboptimal all around.

@tomdale

Another idea is to return a proxy when you request the object (with [set]UnknownProperty), which would return the underlying value (or another proxy if it was nested data), and also mark the record as dirty when anything was set on it.

This would have the benefit of being mostly transparent, but @tomdale is still skeptical. I think it can work ;)

@wycats

@wycats You're assuming we're only working with JSON that's nested one level deep.

I understand what you're saying... and agree that it is suboptimal. I think hasOne() actually works pretty closely enough... it would give me the control to build a Model for my nested JSON, which allows me to determine what sub-properties are being observed and how... if I want to do that. but currently that pattern:

requires IDs for each model (I get errors if I dont fake the IDs... even with {embedded:true})
I dont see a way for a nested-model to notify its parent that it has changed... no way to link the parent's "isDirty" to the child's "isDirty". I think maybe DS.belongsTo() might have done provided something like this??... but it doesn't seem to be in the 0.9.5 code

feels like there would be a way to change the association code a bit (hasOne, hasMany)... so then you're not inventing a DS.attr('object') where you dont exactly know what the developer wants you to do with it. we can make an "object" transform now... and do the work there on full set/get... but it feels like the tools are already here for the developer to be explicit with the model schema using DS.Model and build up a nested schema as complicated as they want

Hello guys, I followed this discussion because I have the same use case (nested documents).
After some thinking, I decided the solve the problem by creating a server-side component which flattens the data so that it can be used with ember-data.
My input to this discussion: Probably you do not want embedded records in the first place. Think about the complexity you introduce the your ember app. It is probably better handled on the server.

Having said that, I vote against having model-fragments in ember-data.
👎

@boehlke

@boehlke You assume that any data can be flattened, but that's absolutely not the case.

@lolmaus

@lolmaus Sorry, I cannot think of an example JSON object which cannot be flattened. Can you provide one? Probably that's easier than for me to prove that every JSON can be flattened. Thanks!

This thread is turning into masturbation. Embedded objects are parts of JSON documents. If ember won't support it, fuck it. This framework is too "idiomatic" to be usable.

@boehlke

@boehlke Imagine a hash that has arrays which contain hashes which contain arrays of hashes.

You can of course undestructively flatten it by producing really elaborate key names. But working with such a flat structure from Ember will be a nightmare. You won't be able iterate over specific sub-arrays, etc. That's just not a working solution.

The reality is, depending on the scenario people want both flattened and nested structures are ideal.
As with a DBMS, if one wants to keep data consistent, normalized structures are ideal. Many times though, this constraint isn't important and more natural graphs become easier to deal with.

Compromises exists, but as ED's goal was consistency, that explains its current state (not its final form).

If ember won't support it, fuck it.

Ember works fine with deeply nested data-structures out of the box.

Ember-data also works with deeply nested documents.

I believe the pain people are noticing. Is they want there nested blobs to behave like ember-data models, in both a read/write/serializer/deserialize fashion. Ember-data today, is largely the following units (on top of ember), and models have nice patterns, such as:

an identity map (to ensure you are always dealing with a single canonical entity)
a set of adapters (to normalize foreign endpoints/api)
a set of serializers (meant to contain all logic related to transformation of data to/from the server)
a mechanism of relationship syncing
a mechanism to detect changes, and the ability for a developer to commit said changes back a foreign data-store

Expanding these patterns to the nested id'less entities, is something we are interested in exploring. the model-fragments project is showing promise, and is a very intriguing interpretation of the problem. Unfortunately, it relies on private API and monkey patching to accomplish its goal. One of our goals moving forward is, to ensure a stable public API for model-fragments. This will prevent the upgrade turmoil currently introduced.

Now as for model-fragments becoming part of ember-data itself, it's too soon to know for sure. It is great to push this logic into user-space, as it will come with some caveats (and the problem space needs some more fleshing out) The important part today is for both model-fragments and ember-data to work better in tandem. This will allow the flexibility for authors to experiment, but also the stability to ship real projects with it.

@wildchild

@wildchild I would advise you to read the community guidelines. I do not mean to belittle your frustration but this kind of behavior is not acceptable in the ember community. http://emberjs.com/guidelines/

@stefanpenner

@stefanpenner

Ember works fine with deeply nested data-structures out of the box.

I just ran into this last night:

{{upload-field inputId="desktop-hero" accept="image/*" signURL=signURL uploadedURL=model.templateSettings.context.hero.src}}

Because model.templateSettings = {} and model.templateSettings.context.hero doesn't exist yet, the value is never updated.

I am forced to do this as a workaround in afterModel:

    if (isNone(get(model, 'templateSettings.context.hero'))) {
      set(model, 'templateSettings.context.hero', {});
    }

@devinus

@devinus a better place would be to add that to your serialization code.

Also, those questions are better asked on SO.

@stefanpenner

@stefanpenner What question?

(Also, this is before serialization even happens. I was just providing a counter-example to your claim that "Ember works fine with deeply nested data-structures out of the box." In my experience, it does not.)

@lolmaus

@lolmaus I'm having trouble figuring out how to nicely flatten a JSON.
I have a similar problem to this SO question http://stackoverflow.com/questions/25220442/json-api-and-translations-with-ember-data and I really like the first answer with nesting everything under the translations field.
Sure, I could flatten everything to title_en, title_fr, but this way on a change of locale, I would have to manually change each field.
Any nice solution to this case?

@aivanov93

@aivanov93 ember-data-model-fragments seem to fit nicely into your use case.

But I would attempt a completely different solution. I would split the model into two parts, e. g. post and post-translation. A single post would look like this then:

{
  id: '1'
  type: 'post',
  relationships: {
    translations: [
      {id: '1', type: 'post-translation'},
      {id: '2', type: 'post-translation'}
    ]
  }
}

{
  id: '1'
  type: 'post-translation',
  relationships: {
    post: {id: '1', type: 'post'}
  },
  attributes: {
    lang: 'en',
    title: 'Hello world!',
    body: "Wonderful weather we have today, don't we?"
  }
}

{
  id: '2'
  type: 'post-translation',
  relationships: {
    post: {id: '1', type: 'post'}
  },
  attributes: {
    lang: 'ru',
    title: 'Здравствуй, мир!',
    body: 'Чудесная погодка сегодня, не так ли?'
  }
}

This approach will also let you collect language-specific comments to posts.

# This is a combination of 64 commits.

Navigation Menu

Repository navigation

Support nested JSON objects #53

92 remaining items

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Support nested JSON objects #53

Description

Activity

tchak commented on Jan 18, 2012

ghempton commented on Jan 19, 2012

devinus commented on Jan 19, 2012

devinus commented on Jan 19, 2012

tchak commented on Jan 19, 2012

devinus commented on Jan 26, 2012

tomdale commented on Jan 26, 2012

devinus commented on Jan 26, 2012

tomdale commented on Jan 26, 2012

devinus commented on Jan 27, 2012

NickFranceschina commented on Feb 21, 2012

wycats commented on Feb 21, 2012

wycats commented on Feb 21, 2012

devinus commented on Feb 21, 2012

NickFranceschina commented on Feb 21, 2012

92 remaining items

boehlke commented on Sep 10, 2015

lolmaus commented on Sep 10, 2015

boehlke commented on Sep 10, 2015

wildchild commented on Sep 10, 2015

lolmaus commented on Sep 10, 2015

stefanpenner commented on Sep 10, 2015

fivetanley commented on Sep 10, 2015

devinus commented on Sep 10, 2015

stefanpenner commented on Sep 10, 2015

devinus commented on Sep 10, 2015

aivanov93 commented on Mar 1, 2016

lolmaus commented on Mar 8, 2016

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions