Implement Sideways Data Loading

@jhusain

This is a first-class API for sideways data loading of stateless (although potentially memoized) data from a global store/network/resource, potentially using props/state as input.

type RecordOfObservables = { [key:string]: Observable<mixed> };

class Foo {

  observe(): RecordOfObservables {
    return {
      myContent: xhr(this.props.url)
    };
  }

  render() {
    var myContent : ?string = this.data.myContent;
    return <div>{myContent}</div>;
  }

}

observe() executes after componentWillMount/componentWillUpdate but before render.

For each key/value in the record. Subscribe to the Observable in the value.

subscription = observable.subscribe({ onNext: handleNext });

We allow onNext to be synchronously invoked from subscribe. If it is, we set:

this.data[key] = nextValue;

Otherwise we leave it as undefined for the initial render. (Maybe we set it to null?)

Then render proceeds as usual.

Every time onNext gets invoked, we schedule a new "this.data[key]" which effectively triggers a forcedUpdate on this component. If this is the only change, then observe is not reexecuted (componentWillUpdate -> render -> componentDidUpdate).

If props / state changed (i.e. an update from recieveProps or setState), then observe() is reexecuted (during reconciliation).

At this point we loop over the new record, and subscribe to all the new Observables.

After that, unsubscribe to the previous Observables.

subscription.dispose();

This ordering is important since it allows the provider of data to do reference counting of their cache. I.e. I can cache data for as long as nobody listens to it. If I unsubscribed immediately, then the reference count would go down to zero before I subscribe to the same data again.

When a component is unmounted, we automatically unsubscribe from all the active subscriptions.

If the new subscription didn't immediately call onNext, then we will keep using the previous value.

So if my this.props.url from my example changes, and I'm subscribing to a new URL, myContent will keep showing the content of the previous url until the next url has fully loaded.

This has the same semantics as the <img /> tag. We've seen that, while this can be confusing and lead to inconsistencies it is a fairly sane default, and it is easier to make it show a spinner than it would be to have the opposite default.

Best practice might be to immediately send a "null" value if you don't have the data cached. Another alternative is for an Observable to provide both the URL (or ID) and the content in the result.

class Foo {

  observe() {
    return {
      user: loadUser(this.props.userID)
    };
  }

  render() {
    if (this.data.user.id !== this.props.userID) {
      // Ensure that we never show inconsistent userID / user.name combinations.
      return <Spinner />;
    }
    return <div>Hello, {this.data.user.name} [{this.props.userID}]!</div>;
  }

}

We should use the RxJS contract of Observable since that is more in common use and allows synchronous execution, but once @jhusain's proposal is in more common use, we'll switch to that contract instead.

var subscription = observable.subscribe({ onNext, onError, onCompleted });
subscription.dispose();

We can add more life-cycle hooks that respond to these events if necessary.

Note: This concept allows sideways data to behave like "behaviors" - just like props. This means that we don't have to overload the notion state for these things. It allows for optimizations such as throwing away the data only to resubscribe later. It is restorable.

undefined is probably the safest value to assign to data until the observable provides its first value via onNext. For example in Relay we assign different meanings to null (data does not exist) and undefined (not yet fetched), so our ideal default data value would be undefined. The alternative is to provide a new method, eg getInitialData, but I suspect this is unnecessary/overkill.

This is pretty interesting however from the point of view of static typing I'm not so happy of key/value system, their type is pretty much impossible to express.
Why not having observe return a single observable and set/merge the value resolved to this.data :

class Foo {

  observe() {
    return (
      loadUser(this.props.userID)
        .map(user => { user })
  }

  render() {
    if (this.data.user.id !== this.props.userID) {
      // Ensure that we never show inconsistent userID / user.name combinations.
      return <Spinner />;
    }
    return <div>Hello, {this.data.user.name} [{this.props.userID}]!</div>;
  }

}

And for the case of multiple fetch something like :

class Foo {

  observe() {
    return (
     combineLatest(
      loadUser(this.props.userID),
      loadSomethingElse(this.props.somethingElseId),
      (user, somethingElse) => ({ user, somethingElse})
     )
  }

  render() {
    ..
  }

}

This is perhaps a bit more verbose, but it allows to have nice static type :

interface Comp<T> {
  observe(): Observable<T>;
  data: T;
}

Also instead of re executing observe when props/state change we can have access to 'props' 'state' as an observable :

class Foo {

  observe(propsStream) {
    return (
      propsStream
        .flatMap(({ userID }) => loadUser(userId))
        .map(user => { user })
    );
  }

  render() {
    if (this.data.user.id !== this.props.userID) {
      // Ensure that we never show inconsistent userID / user.name combinations.
      return <Spinner />;
    }
    return <div>Hello, {this.data.user.name} [{this.props.userID}]!</div>;
  }
}

The reason is because we don't want to require the use of combinators and understanding RxJS to be able to subscribe to (multiple) Observables. Combining two Observables in this way is quite confusing. In fact, at least for our data sources, we'll probably implement the subscription API but not even include the combinators on the Observables' prototype. That's not a requirement, but you're free to use combinators if you need to.

However, to subscribe to a simple Flux store, you shouldn't need to.

I think that Flow will probably be able to handle this static type using constraints, but I will check with those guys to make sure. I think that it'll be enough to type the data property and then the observe type can be implied.

class Foo extends React.Component {
  data : { user : User, content : string };
  observe() /* implied as { user : Observable<User>, content : Observable<string> } */ {
  }
}

This change is not about going all in on Observables as a way to describe application state. That can be implemented on top of this, like you've done before. This is explicitly not about application state since this method is idempotent. The framework is free to unsubscribe and resubscribe as needed.

@ericvicenti

cc @ericvicenti

At least in the case of typescript, there would be no way to constraint the return type of observe based on the type of data, at least until something like microsoft/TypeScript#1295 is implemented.

I'd love to use this for the next version of React DnD, but obviously this requires waiting for React 0.14.
I wonder if I can “polyfill” this for the time being with a higher-order component that sets this.data on a ref instance.. Might be too crazy though.

Would it be possible to observe Promises? Then one could use a Promises tree to resolve data for the whole component tree before the first render! This would be very useful for server-side React.

What are the benefits of making this a first-class API? It could essentially be accomplished using a "higher-order component."

What are the benefits of making this a first-class API? It could essentially be accomplished using a "higher-order component."

Wrapping in 5 HOCs to get 5 subscriptions is a bit unwieldy and harder to understand for beginners. Understanding componentWillReceiveProps is also non-trivial. This fixes both.

I, for one, welcome our new observable overlords.

I wonder if this can help bring https://github.com/chenglou/react-state-stream closer to React's vanilla API

Wouldn't it just take one HOC? In the example in your Medium post, you iterate over stores and subscribe to each.

stores.forEach(store =>
  store.addChangeListener(this.handleStoresChanged)
);

@aaronshaf

@aaronshaf Depends on use case, for sure. Sometimes it's different kinds of state sources, not just “several stores”. But I can't say on behalf of React team, let's hear what @sebmarkbage says.

Would love some sort of polyfill to play with this now. I didn't get the idea completely, yet. What is the mechanism involved with dealing with future updates. I'll spend some more time understanding it. I think it should be doable with a simple mixin.

@vjeux

(@vjeux told me I should chime in! so here I am.)

I don't mean to promote my own work, but I think this hook is very similar to the getNexusBindings hook in React Nexus. You declare data deps at the component level via a lifecycle hook (which can depend on props).

The API looks like:

class UserDetails {
  getNexusBindings(props) {
    return {
      // binding to data in the datacenter
      posts: [this.getNexus().remote, `users/${this.props.userId}/posts`],
      // binding to data in the local flux
      mySession: [this.getNexus().local, `session`],
    }
  }
}

The binding is applied/updated during componentDidMount and componentWillReceiveProps. In the latter case, the next bindings are diffed with the previous bindings; removed bindings are unsubscribed, added bindings are subscribed. The underlying fetching/updated mechanism is described in the implementation of Nexus Flux. Basically with the same API you can either subscribe to local data (traditional local stores) or remote data (fetch using GET and receive patches via Websockets/polyfill). You could actually subscribe to data from another window (using postWindow) or a WebWorker/ServiceWorker but I still haven't found a truly useful use case for this.

Long story short, you synchronously describe data deps at the component level using a Flux abstraction and the hooks make sure your dependencies are automatically subscribed, injected on updates, and unsubscribed.

But it also comes with a nice feature: the exact same lifecycle functions are leveraged to perform data prefetching at server-side rendering time. Basically, starting from the root and recusrively from there, React Nexus pre-fetches the bindings, renders the component, and continues with the descendants until all the components are rendered.

@sebmarkbage

@sebmarkbage @jimfb I've been following this thread and the alt thread (#3858) for a few months now, and am curious if the core team has reached any consensus about this problem, or at least a general direction.

@oztune

@oztune No updates; we've been focused on other priorities. We will post to one of the threads when there is an update on this topic.

Guys, I have a created a universal API to compose containers. Check react-komposer. Which that, we could create containers with a higher order function.

import { compose } from `react-komposer`;

// Create a component to display Time
const Time = ({time}) => (<div>{time}</div>);

// Create the composer function and tell how to fetch data
const composerFunction = (props, onData) => {
    const handler = setInterval(() => {
    const time = new Date().toString();
    onData(null, {time});
  }, 1000);

  const cleanup = () => clearInterval(handler);
  return cleanup;
};

// Compose the container
const Clock = compose(composerFunction)(Time);

// Render the container
ReactDOM.render(<Clock />, document.getElementById('react-root'));

Here's the live version: https://jsfiddle.net/arunoda/jxse2yw8/

We've also has some easy ways to compose containers with Promises, Rx.js Observables and With Meteor's Tracker.

Also check my article on this: Let’s Compose Some React Containers

@arunoda

@arunoda We wound up doing something very similar. One thing I'm wondering is how do you prevent composerFunction from being called on every prop change?

@oztune

@oztune Actually now it'll run again. We use this Lokka and Meteor. Both of those have local caches and does not hit the server even when we call the composerFunction multiple times.

But I think we could do something like this:

const options =  {propsToWatch: ["postId"]};
const Clock = compose(composerFunction, options)(Time);

Any ideas?

@arunoda

@arunoda That's what we tried too, but it creates a bit of a disconnect between the action and its dependencies. We now do something similar to react-async, where, instead of immediately performing the given action, composerFunction would return a function and a key. If the key is different from the previous key returned by composerFunction, the new function will be performed. I'm not sure if this is a tangent from this github thread, so I'd be glad to continue on Twitter (same username).

@oztune

@oztune I created a new GH issue and let's continue our chat there. Much better than twitter I guess.

Why not just let JSX to understand and render Observables directly so that I can pass Observable in props, e.g. this.props.todo$ and embed them in JSX. Then I wouldn't need any API, the rest is managed outside of React and HoC is used to fill in observables. It shouldn't matter if props contain plain data or an observable hence no special this.data is needed.

{this.props.todo$

Additionally React render could be able to render Oservable[JSX] which would allow design described in the links without additional library.

https://medium.com/@milankinen/containers-are-dead-long-live-observable-combinators-2cb0c1f06c96#.yxns1dqin

https://github.com/milankinen/react-combinators

Hello,
As for now we could use stateless components with rxjs streams easily.
I dont understand the need for another API.
I wrote an example - a board that you can move mouse over it and when it gets to 26 it changes to restart.
I would love to hear what you think about this way.
Here is the code:
https://jsfiddle.net/a6ehwonv/74/

@giltig

@giltig: I'm learning this way lately as well and like it. Together with Cycle.js.

I would appreciate to be able to listen to event handlers defined on components somehow easily without having to pass in Subjects for bridging. If I understand correctly it is pretty much the other way around than what is suggested here. Alternatively, if we could observe React vdom for its synthetic events, maybe "refs" could be used for observing to avoid having CSS tags just to observe.

Further RxJs could support object in combineLatest so that we could use React functional components directly to create larger components.

const myFancyReactComponent = ({surface, number, gameover}) => (
        <div> 
          {gameover ? gameover : surface}
          {number}
        </div>
)

const LiveApp = Rx.Observable.combineLatest(
    LiveSurface, DynamicNumberView, DynamicGameOver,
    myFancyReactComponent
)

Hi,
The reason we don't use Cycle or any other framework is that I prefer libraries over frameworks (more control to the developer) and also I want to enjoy the community power that React has.
So many render engines are now developed for React and it's a pity not to use it (ReactNative, ReactDom, ReactThree etc..). It's pretty simple to use just Rxjs and react as the example I've shown above.

It would have made thinks easier if react components could accept pojos as long as observables as props. As for now it's not possible so what I've shown above is the way we chose.

BTW what you did with MyFancyReactComponent is possible and we actually did that too in some cases though you can also write the jsx straightforward.

Regarding subjects - I think it's a valid way because eventually in React component I'm just using a handler function which could be anything. I choose to implement it with subject inside that receives events but someone else can choose otherwise - its flexible.

It would have made thinks easier if react components could accept pojos as long as observables as props. As for now it's not possible so what I've shown above is the way we chose.

observable props have a no sense in the long run. It has no sense at all in that context, actually..

It sounds to me like we ended up with a different model with Suspense (cache + context). The cache itself might get support for subscriptions. You can track remaining work for Suspense in #13206.

We also provide a subscription package for more isolated cases.

Navigation Menu

Repository navigation

Implement Sideways Data Loading #3398

109 remaining items

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Implement Sideways Data Loading #3398

Description

Activity

josephsavona commented on Mar 13, 2015

fdecampredon commented on Mar 13, 2015

fdecampredon commented on Mar 13, 2015

sebmarkbage commented on Mar 13, 2015

vjeux commented on Mar 13, 2015

fdecampredon commented on Mar 13, 2015

gaearon commented on Mar 13, 2015

RickWong commented on Mar 14, 2015

aaronshaf commented on Mar 14, 2015

gaearon commented on Mar 14, 2015

gaearon commented on Mar 14, 2015

aaronshaf commented on Mar 14, 2015

gaearon commented on Mar 14, 2015

nmn commented on Mar 14, 2015

elierotenberg commented on Mar 14, 2015

109 remaining items

oztune commented on Nov 16, 2015

jimfb commented on Nov 16, 2015

arunoda commented on Jan 3, 2016

oztune commented on Jan 3, 2016

arunoda commented on Jan 3, 2016

oztune commented on Jan 3, 2016

arunoda commented on Jan 4, 2016

vladap commented on Jan 17, 2016

giltig commented on Mar 1, 2016

vladap commented on Mar 2, 2016

giltig commented on Mar 2, 2016

lexfrl commented on Nov 20, 2016

gaearon commented on Aug 15, 2018

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions