[M2] Improve acknowledgements.

[evanlinjin]

Depends on #36

Feature description

ACK frames serve as both: a method to ensure reliable delivery (just in case the remote client shuts down), and a way of basic flow control.

The way it is currently implemented can be improved to allow greater performance.

Is your feature request related to a problem? Please describe.

In the current state of the system, a (*dmsg.Transport).Write() operation awaits to receive an ACK frame from the remote Client. This is both time consuming and resource-intensive (as we have to use ioutil.AckWaiter to align sequences of both ends of the transport).

Describe the solution you'd like

• dmsg.Transport should have fields that keep track of the local and remote window sizes (the buffer size left in bytes).
• REQUEST and ACCEPT frames should contain window sizes. And each edge keeps track of the remote edge's window size and stops sending when it is maxed out (when this happens, the Write operation should block).
• Write operations no longer needs to wait for an ACK frame from remote.
• ACK frames are sent on every Read operation and contain the number of bytes read. The remote then increases the tracked window size by that amount.

Describe alternatives you've considered

We can also make the dmsg.Server be responsible for sending the ACK frames after the FWD is forwarded. However, this doesn't give us the ability to do flow control, and it is not as performant as the aforementioned solution.

Possible implementation

• Add window size fields to dmsg.Transport
• Add window sizes to REQUEST and ACCEPT frames.
• Remove awaiting for ACK after Write() operation.
• Send ACK after every Read() operation, which includes number of bytes read.
• Add logic to handle ACK frames, that adds to remaining window size.
• Write operation should block until remote window size frees up.