There doesn't appear to be any mechanism to clear the buffer once all Shared streams have read an entry at a particular index. This will result in unbounded memory usage for infinite streams.
Perhaps a VecDeque could be used for the buffer, with values only being popped if all Shared streams have finished reading at a particular index. This would also need to take into account whether a Shared has been dropped.
Thoughts?