API to give the backlog for a databus consumer

[sreedishps]

This is to come up with an API which will give the pending amount of data to be consumed by a databus consumer.

[raju-bairishetti]

@sreedishps Do you mean how much time is lagging behind current time or how much data(in size) is pending? I am assuming that this is for providing the lagging time instead of lagging size.

[sreedishps]

@raju-bairishetti this is for getting the current pending 'size' (collector + local) for a databus consumer

[raju-bairishetti]

We have a metric for the last read minute directory for local, merge & mirror streams for databus consumer.

We have metrics for the lagging time as well for all stream types. Seems this metric is sufficient.

References:

https://github.com/InMobi/pintail/search?utf8=%E2%9C%93&q=getLatestDirectoryLagTime

https://github.com/InMobi/pintail/blob/b9cbe17e5042ad08ec51b6cf3987c77a4dc4d592/messaging-client-databus/src/main/java/com/inmobi/messaging/metrics/PartitionReaderStatsExposer.java