Add basic Mongo db maintenance functions

[jimsiff]

Creating this issue based on Gitter conversation with @bewest.

There are currently no user friendly tools to manage Mongo database data. It would be nice to add basic collection manipulation functions to the CGM Tool. Examples include

• delete individual or groups of documents with bad timestamps or data
• modify individual documents

Supporting some basic maintenance would help resolve issues like stuck battery status on Pebble and future data in websites and Pebble without resorting to hunting through the collection on Mongolab.com, RoboMongo, or deleting the entire collection dataset.

[altintx]

This is not advice, just my experience:

I'm pretty aggressive about what data I allow IN to app. Data can only be 24h into the future (to allow for traveling across TZ basically). I have facilities to both download entire Mongolab EGVs and re-publish them. That said, one easy way to "clean" Mongolab DB is to start with an empty Chrome Tools profile (Reset), Download from Mongolab (gets MOST data, ignoring invalid records), empty out Mongo table, and then re-publishing.

Not a particularly well exposed function, but there it is. It's also the exact opposite of how you're suggest (En mass vs individual).

How would you picture your proposed feature fitting into Chrome app? Some kind of Table Viewer hiding behind a button? A Fix It For Me button? I find people are generally unwilling to press (or send, anyway) the Something's Broke button, so Fix It For Me might go mostly unused.

[jimsiff]

Thanks for the feedback. I don't have a good sense on the UI layout or integration. This is more of an idea fishing expedition at this point.

I guess one function might be a search box that allows users to search collections based on document fields with operations like <,>,=,!=, include/exclude, etc. Example uses might be "find any EGV documents with the date > now" to find future data. I'm not sure how well that would work with devicestatus or treatments which use different date/time stamps.

I think @bewest has some ideas for Chrome functions but I'm not sure exactly what they are or how he'd envision integrating them with the CGM Tool.

From Gitter:

"I've been thinking about putting in some controls on the chrome app to allow selecting some data and deleting it since the chrome app runs on your PC, it can be trusted to do maintenance activities on your db"

[altintx]

Does NS's API allow find-all/filter-by? I wonder if it'd be better to preform DML through it than directly on Mongo. I haven't managed to keep perfect parity and because deletes are so damned dangerous...

[jimsiff]

I'm not sure that the API currently supports that... but I think @jasoncalabrese agrees with you. He suggested extending the API to support data manipulation in Gitter last night.

"It should not use direct mongo, we should have utility API methods that check for data issues, and with the API-secret can clean them. We'll probably need some type migration when we change the date formats"