Wantlist Race B

[Stebalien]

Forked from: #99

Problem

1. Peer A connects to peer B.
2. Peer A sends its wantlist to peer B.
3. Peer A disconnects from peer B.
4. Peer B notices the disconnect, forgets peer A's wantlist.
5. Peer A immediately reconnects to peer B.
6. Peer A notices the disconnect, sees that it still has a connection, doesn't resend it's wantlist.

Solution

The only thing I can think of is the following: whenever the stream we're using to send wantlists closes, we (a) open a new one and (b) send the complete wantlist. Unfortunately, I really wanted to eventually move away from keeping a stream open entirely but, we'll, I can't think of a better solution.

[hannahhoward]

It seems to me, the more we get into these scenarios, the more we are working around the fact that we have a replication problem. We're sending want lists on initial connect based on the assumption that they are received, and then deltas on the assumption they are received. However, there are a million reasons this could fail. It seems to me, we need to treat this as an eventual consistency problem, and make sure the other peer always receives enough information to reconstruct the entire want list. I can't tell -- just looking at the bitswap message format -- if this can be achieved without protocol changes.

[Stebalien]

I agree. As-implemented, bitswap is actually an unreliable message based protocol. Given that, we should probably be assigning sequence and session numbers to messages. But yeah, that would require a protocol change.

[hannahhoward]

Is the protocol open to revision if it can be non-breaking (i.e. newer peers can support more reliable communication if supported, but their messages can still be used by older peers?)

[Stebalien]

I don't see why not. However, I still think we should try to negotiate a new protocol as that'll allow both sides to know that we're supporting this new feature. When we know our peer supports the new feature, we can close idle bitswap streams without affecting our wantlists.

[hannahhoward]

ipfs/specs#201

[hannahhoward]

I personally don't think there is a better solution than period rebroadcast of wantlists with the current protocol. Otherwise I feel like we're chasing our own tail with all the bad things that can happen with a network connection.

[Stebalien]

You're probably right.