Skip to content

Conversation

@jeanleonov
Copy link
Contributor

@jeanleonov jeanleonov commented Sep 30, 2019

This branch was inspired by ELK and how processes stats can be analyzed there (and other types of stats actually). It doesn't modify existing functionality, but extends it with a new v2 API which currently supports only processes stats.

I'd highlight two aspects of changes:

  1. Processes stats v2 structure provides more easy-to-use properties comparing to v1 (which is not removed btw).
  2. New API v2 concept.

1. Processes stats v2 vs v1

Instead of 3 categorization fields monit_name, unified_service_name and application_id, in version two there are two repeated fields own_tags and all_tags, which suppose to get values like [appscale, solr] or [appscale, searchservice] or [<rabbitmq-childprocess-name>, appscale, rabbitmq] etc.
own_tags holds only explicitly assigned tags like [appscale, datastore] or a single [<process_name>]. all_tags contains own + ancestors' tags.
Such tagging allows flexible and powerful processes filtering in ELK.

Additionally to cumulative CPU and IO counters, v2 brings *_1h_diff fields which provides estimated hourly diff since previous measurement. So it suppose that a single client will request processes stats regularly. Otherwise estimated diffs might be inaccurate.

2. API v2 concept

Objects provided by API v2 suppose to be flat which matches analytic needs (like ELK) much better.
It doesn't support include lists. It should simplify caching on proxy side and simplify collecting stats from multiple nodes without need to parse and serialize JSON (just joining responses).

@jeanleonov

This comment has been minimized.

@jeanleonov
Copy link
Contributor Author

Recommended to go after #3165 and it may have a conflict with it in solr management command related to sudo removal.

@jeanleonov jeanleonov marked this pull request as ready for review October 4, 2019 15:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant