Create a method to get changes until hasMoreChanges is false

[alagane]

As a library user (eg: OpenPaas)
I want to get all the Mailbox changes using only one method
So that I do not have to worry about the hasMoreChanges mechanic which will be handled by the library

When I get all the changes
And the response property hasMoreChanges is false
Then the behaviour is the same than mailbox_changes

When I get all the changes
And the response property hasMoreChanges is true
Then more requests are made and merged until hasMoreChanges is false
And the client should get a response like if it was done in one request (but it will take more time)

Merging responses would be:

• union of created (minus those in destroyed) -> if a Mailbox was created and destroyed, no need to fetch it
• union of updated (minus those on created and those on destroyed) -> if a mailbox was created and updated, it will be fetched because in created; if a mailbox was updated and destroyed, no need to fetch it
• union of destroyed (minus those on created) -> if a Mailbox was created and destroyed, no need to remove it

[oubchid]

We need to make multiple requests until hasMoreChanges is false.
So IMO we need only the result of the last request that contains all mailboxes ids of all previous requests ?
Or should get response for every requests and merge them

[chibenwa]

So IMO we need only the result of the last request that contains all mailboxes ids of all previous requests ?

If each request holds 100 results but there is 1000 changes would it be acceptable to return only the last?

[oubchid]

I'm asking a question cause I'm waiting for more clarification or an answer.
I had thought that we pass the last state we will have all the changes

[Arsnael]

See it as batches. Let's retake Benoit example and detail a bit more. Let's say user did not use his working laptop for a while, and then he comes back to it and refresh it.

The webclient will ask for the changes. Let's say your state locally is 0.Let's say a 1000 changes occured, so the state on the server is 1000. But we limit the number of changes per request to 100 per call (supplied by the client or limited by the server).

When you call /changes with sinceState 0, and maybe server limited to 100 the number of max changes returned... You will get the first 100 changes, with a hasMoreChanges to true and a new state to 100. Then you call /changes again, with since state 100, you will get next batch of changes, new state 200, hasMoreChanges still true... Then you keep calling until hasMoreChanges is false and you reach the server state 1000 which is equal to the latest state on the server.

I had thought that we pass the last state we will have all the changes

Once again you are a client, you do not know if the server puts a limit or not to max number of changes, thus the parameter hasMoreChanges in the response. Allows to make batches. Too many changes can make a response too heavy maybe, or subject to losses over the network... I'm not sure but there must be good reasons like this. Thus the batching strategy. Is it clearer @oubchid ? :)

[chibenwa]

Maybe let's retake the basics.

https://jmap.io/index.html

Why is JMAP better than IMAP?

It provides flood control. The client can always restrict how much data the server should send. For example, 
a command might return a `tooManyChanges` error if it exceeds the client's limit, rather than returning a 
million `* 1 EXPUNGED` lines as can happen in IMAP. Sometimes it's just more efficient to throw away 
cached data and refetch, especially in the case of a mobile/webmail interface with only a partial cache 
of the server's data.

How flood control works

The way this flood control works is to have the client explicitly specifying the amount of data he wishes to receive (https://jmap.io/spec-core.html#changes)

maxChanges: UnsignedInt|null The maximum number of ids to return in the response. The server MAY
 choose to return fewer than this value but MUST NOT return more. If not given by the client, the server 
 may choose how many to return. If supplied by the client, the value MUST be a positive integer greater 
 than 0. If a value outside of this range is given, the server MUST reject the call with an invalidArguments error.

So here the question might be: Say we request 128 changes, if there's more changes maybe we can just go with a getAllMailboxes.

We can batch updates but IMO it should be reasonably low. (No more than 512?)

James BTW applies the the limit from the client without restriction.

Why would a client uses a lower value of maxChanges compared to the count of value it wants?

Because a server MIGHT return recent results first (and manage intermediate states) - not implemented in James.

Also smaller batch might give an impression of interactivity - but it is not doable if hidden in the jmap-client-ts.

What we should likely do for INBOX?

We likely should not care about getting changes until hasMoreChanges is false.

We should ask James upfront for the count of changes we want (say 512).

If exceeded we do a full resynch (getAllMailboxes)

[oubchid]

See it as batches. Let's retake Benoit example and detail a bit more. Let's say user did not use his working laptop for a while, and then he comes back to it and refresh it.

The webclient will ask for the changes. Let's say your state locally is 0.Let's say a 1000 changes occured, so the state on the server is 1000. But we limit the number of changes per request to 100 per call (supplied by the client or limited by the server).

When you call /changes with sinceState 0, and maybe server limited to 100 the number of max changes returned... You will get the first 100 changes, with a hasMoreChanges to true and a new state to 100. Then you call /changes again, with since state 100, you will get next batch of changes, new state 200, hasMoreChanges still true... Then you keep calling until hasMoreChanges is false and you reach the server state 1000 which is equal to the latest state on the server.

I had thought that we pass the last state we will have all the changes

Once again you are a client, you do not know if the server puts a limit or not to max number of changes, thus the parameter hasMoreChanges in the response. Allows to make batches. Too many changes can make a response too heavy maybe, or subject to losses over the network... I'm not sure but there must be good reasons like this. Thus the batching strategy. Is it clearer @oubchid ? :)

More clear now :)
and good to know that we have limit of changes in the server side

[chibenwa]

@oubchid I did edit my comment.