Add API to allows handling of paginated (infinite scroll) queries inside the Query itself

[FrancescoCioria]

requirements

the fetch function of a Query is called with a single argument: an object containing all the needed params + other queries to perform the async call to the API.

This is not enough for paginated APIs as, for those, we would need to receive as input of fetch the previous result as well.

This is the use case I'm imagining:

1. API to get a feed of posts
2. Fe client that shows the feed and, when you scroll to the bottom, loads more

Right now, we could:

1. reload every rendered post + load more post every time the user scrolls (bad perfs)
2. store the previous value in a component + load only the new posts + concat the new data to the previous one inside the component (probably in componentWillReceiveProps) and render the posts using the component's this.state
3. add a global state param "prevValue" + keep it in sync in the component (probably in componentWillReceiveProps) + move the concat logic in the query

What I want:

1. Move the concat logic inside the query
2. no state in the UI Component
3. no prevValue variable in the global state

The container should be rendered with the correct values and the query should ask the API only for the posts that haven't been previously downloaded

specs

An option could be to pass the value stored in the cache (previous value) as second argument of fetch.

One problem I see with this is that the previous value is only useful for paginated API+UI.

Maybe we could make it "more difficult" for the user to use the previous value by adding another class called PaginatedQuery that looks the same as Query but passes a second argument to fetch (this is only needed to discourage the user to use the cached value when it's not needed, or worse, risky)

misc

{optional: other useful info}

[giogonzo]

Hi @FrancescoCioria , infinite scroll is definitely an interesting case (do we have an internal use-case for this already?)

Just to be clear, simple pagination instead works OK as is, as you typically show a single "page" at a time, and have a page parameter the query can accept and pass to the API.

About the possible solutions as of now, this is what I had in mind:

• the query remains stateless and returns something in the form { page: [list of items], index: number }
• we use a new reduce query prop function to handle pagination: it will concat different pages in order, based on the index, and cache all previous pages in its accumulator similar to what cacheQueryValues already does (in this case, a map index -> page could work as accumulator I think)

About providing an ad-hoc API in avenger core: sounds reasonable, considering even GraphQL (Apollo/Relay) have ad hoc apis for paginated queries (usually based on the concept of a "pagination cursor" though)

I tend to see this as a special case of a more general one: an infinite scroll query would like to have itself as a dependency, but pass different params (in this case, the param to change would be the page, e.g.: for a query list(page=3), we'd need to recursively depend on list(page=2) and list(page=1) (this last one would be the base case for the recursion).
I think this can be easily extended to a from-to version (from=4,to=10) to start not at the beginning of a list.
In other words, an infinite scroll query can be seen as a paginated query depending on itself. Note that this means you can freely move in the pagination index even if you are missing some pages here and there (you don't depend on a "previous state", you can remain "stateless")

Currently, a query cannot specify param values to pass to a dependency - they are implicit and gathered from the outside world. To do this, we'd need a new api for specifying dependencies, one that allows a query to threat a dependency as lazy, e.g. instead of obtaining a resolved value as part of fetch arguments (e.g. an array posts) it obtains a function (query) that expects to receive all the parameters (e.g. posts() returning a Promise<Post[]>)

This I think would answer to all the desiderata above:

What I want:

1. Move the concat logic inside the query
2. no state in the UI Component
3. no prevValue variable in the global state

More in general if we need/want to provide first-class support for infinite scroll in the framework, I would consider an approach that is not based on pages, but on something more "reliable" (no missing/duplicated elements while scrolling) like cursors. This would require some full-stack thinking