Handling a large queue of old messages

[aaron97neu]

While operating, ISOBlue has a tendency to spend extended periods of time without cell coverage. Due to this, ISOBlue ends up with an large amount of data queued to upload and cannot catch up as data comes in faster than it can upload it. This leads to the degradation in functionality of apps further down the data pipeline as they do not have access to the newest data. Additionally this can cause many users to believe that their board is non-functioning.

Possible solutions:

• Notify the user when this situation is detected
• Allow ISOBlue to continue to upload even when the machine is off
• A "Low resolution mode" when ISOBlue is falling just a little bit behind. IE - ISOBlue uploads 1 out of every N data points to catch up while still uploading some data
• Throw out old data and start fresh
  • At what point should this occur?
  • Should data continue to be stored on the SSD for later retrieval?
  • Should the user be asked before we delete their data?

This issues is mainly to discuss and prioritize the possible solutions to the problem. Each solution will likely become a child issue

[aaron97neu]

6/23/19 Discussion with @wang701: Uber ureplicator discussed as possible replacement for kafka. Will investigate if this is a feasible and desirable replacement