Goal: Reduce Network Usage

[MicBun]

detected our network usage was incredibly high (TB/day), and it was increasing proportionally with amount of streams
requests seemed to be coming from KGW since its outgoing data was low while incoming was high (eth0 interface)
Determined that the source is KGW polling all the schemas
We'll temporarily make this poll interval to 5 minutes. Consequence: we'll probably get dataset/schema/procedure not found errors right after creating streams. It only affects deployment. If desired, we can target nodes directly to avoid this error while deploying

Making the network usage lower, will reduce our cost on infrastructure

Problem:

• Problem: High network usage in instances #738

[zolotokrylin]

what's the benefit here? can you quantify it please?

[MicBun]

what's the benefit here? can you quantify it please?

I edited the Goal description, the benefit is lower network usage and lowering cost of our infrastructure