CPU percentage exceeds 100%

[janheinrichmerker]

I noticed when tracking the CPU_USED_PROCESS_PERCENT, that the resulting tracked values can exceed 100%, e.g.:

{max: 244, min: 0, avg: 0, timeseries: {timestamps: [0ms,100ms,204ms,304ms,404ms,504ms,605ms,705ms,805ms,906ms,1006ms,1107ms,1207ms,1307ms,1408ms,1509ms,1609ms,1711ms,1812ms,1912ms,2012ms,2112ms,2212ms,2313ms,2413ms,2513ms,2613ms,2714ms,2814ms,2914ms,3014ms,3114ms,3215ms,3315ms,3415ms,3515ms,3616ms,3716ms,3816ms,3916ms,4016ms,4117ms,4217ms,4317ms,4417ms,4518ms,4618ms,4718ms,4818ms,4918ms,5019ms,5119ms,5220ms,5321ms,5421ms,5521ms,5621ms,5721ms,5822ms,5922ms,6022ms,6122ms,6223ms,6323ms,6423ms,6523ms,6624ms,6724ms,6824ms,6924ms,7025ms,7125ms,7225ms,7325ms,7425ms,7526ms,7626ms,7726ms,7826ms,7927ms,8027ms,8127ms,8227ms,8327ms,8428ms,8528ms,8628ms,8728ms,8829ms,8929ms,9029ms,9129ms,9229ms,9330ms,9430ms,9530ms,9630ms,9731ms,9831ms,9931ms,10031ms,10132ms,10232ms,10332ms,10432ms,10533ms,10633ms,10733ms,10833ms,10934ms,11034ms,11134ms,11234ms,11335ms,11435ms,11535ms,11635ms,11736ms,11836ms,11936ms,12036ms,12136ms,12237ms,12337ms,12437ms,12537ms,12638ms,12738ms,12838ms,12938ms,13038ms,13139ms,13239ms,13339ms,13440ms,13540ms,13640ms,13740ms,13840ms,13941ms,14041ms,14141ms,14241ms,14342ms,14442ms,14542ms,14642ms,14743ms,14843ms,14943ms,15043ms,15143ms,15244ms,15344ms,15444ms,15544ms,15645ms,15745ms,15845ms,15945ms,16045ms,16146ms,16246ms,16346ms,16447ms,16547ms,16647ms,16748ms,16848ms,16948ms,17048ms,17149ms,17249ms,17349ms,17449ms,17550ms,17650ms,17750ms,17850ms,17950ms,18051ms,18151ms,18251ms,18351ms], values: [0,34,190,164,174,164,174,174,244,214,204,184,194,194,194,189,164,148,120,170,130,130,170,150,150,140,170,120,240,160,140,140,160,140,140,160,140,140,160,140,140,4,150,160,140,150,150,170,140,150,140,170,138,150,130,140,170,130,150,150,130,190,130,140,130,140,180,130,130,150,130,180,140,130,140,150,190,140,130,140,130,160,120,140,130,140,130,180,140,140,140,140,150,170,140,130,130,120,140,150,180,130,140,140,130,140,190,140,140,140,140,130,140,160,140,140,130,150,170,120,140,110,140,140,130,170,140,130,140,130,130,190,130,140,140,130,170,140,140,130,140,140,170,150,130,130,140,130,190,140,130,140,140,180,140,130,140,150,140,110,200,240,130,100,100,100,240,150,150,14,84,94,160,100,100,100,100,110,100,100,130,190,130,180]}}

If this behavior is intended, then what do 100% mean in the context of the measure?

For context, the CPU_USED_SYSTEM_PERCENT measure seems to work as expected. For the same tracking run as above, it produces percentages <= 100%:

{max: 100, min: 0, avg: 0, timeseries: {timestamps: [0ms,100ms,204ms,304ms,404ms,504ms,605ms,705ms,805ms,906ms,1006ms,1107ms,1207ms,1307ms,1408ms,1509ms,1609ms,1711ms,1812ms,1912ms,2012ms,2112ms,2212ms,2313ms,2413ms,2513ms,2613ms,2714ms,2814ms,2914ms,3014ms,3114ms,3215ms,3315ms,3415ms,3515ms,3616ms,3716ms,3816ms,3916ms,4016ms,4117ms,4217ms,4317ms,4417ms,4518ms,4618ms,4718ms,4818ms,4918ms,5019ms,5119ms,5220ms,5321ms,5421ms,5521ms,5621ms,5721ms,5822ms,5922ms,6022ms,6122ms,6223ms,6323ms,6423ms,6523ms,6624ms,6724ms,6824ms,6924ms,7025ms,7125ms,7225ms,7325ms,7425ms,7526ms,7626ms,7726ms,7826ms,7927ms,8027ms,8127ms,8227ms,8327ms,8428ms,8528ms,8628ms,8728ms,8829ms,8929ms,9029ms,9129ms,9229ms,9330ms,9430ms,9530ms,9630ms,9731ms,9831ms,9931ms,10031ms,10132ms,10232ms,10332ms,10432ms,10533ms,10633ms,10733ms,10833ms,10934ms,11034ms,11134ms,11234ms,11335ms,11435ms,11535ms,11635ms,11736ms,11836ms,11936ms,12036ms,12136ms,12237ms,12337ms,12437ms,12537ms,12638ms,12738ms,12838ms,12938ms,13038ms,13139ms,13239ms,13339ms,13440ms,13540ms,13640ms,13740ms,13840ms,13941ms,14041ms,14141ms,14241ms,14342ms,14442ms,14542ms,14642ms,14743ms,14843ms,14943ms,15043ms,15143ms,15244ms,15344ms,15444ms,15544ms,15645ms,15745ms,15845ms,15945ms,16045ms,16146ms,16246ms,16346ms,16447ms,16547ms,16647ms,16748ms,16848ms,16948ms,17048ms,17149ms,17249ms,17349ms,17449ms,17550ms,17650ms,17750ms,17850ms,17950ms,18051ms,18151ms,18251ms,18351ms], values: [100,78,93,83,68,74,79,65,85,84,80,80,80,80,80,81,77,82,98,87,52,27,32,36,32,21,27,28,38,27,27,25,31,30,26,28,26,30,37,28,25,40,35,41,32,25,30,30,27,25,25,29,36,79,33,27,28,31,35,29,35,37,26,30,30,26,30,29,32,26,29,30,26,25,28,26,29,28,28,25,25,30,26,34,33,24,24,30,25,25,24,25,24,31,27,25,25,80,33,26,30,27,34,30,33,25,29,27,29,27,24,28,37,33,24,25,25,26,32,27,25,26,34,28,30,28,24,29,31,27,25,32,25,25,26,24,31,31,34,25,25,24,36,33,30,27,25,27,33,25,25,25,32,40,30,26,25,30,38,29,42,47,39,42,36,44,53,44,52,67,80,76,44,30,35,27,23,27,23,21,25,37,32,42]}}

[janheinrichmerker]

I've taken a quick look into the C code yesterday, but could not spot any obvious miscalculation there. So maybe it is "deeper" in some third-party call?

[TheMrSheldon]

The utilization maybe has to be divided by the number of cores. There are different ways to measure CPU usage on a multicore system:

1. In the range [0%, N*100%]
2. In the range [0%, 100%]

Maybe we are currently using the first option for process CPU usage and the second for system utilization. I will investigate this soon as well. If that is indeed the problem, I could look and see if I can make this configurable in some way. What would be a good default? I personally like the first option more since it can be compare more easily between systems. E.g. the single-threaded process used 100% on both systems. It can also give a more intuitive value for the same programm on systems with varying levels of paralelism. E.g., if it uses 200% on a 2 CPU system, 400% on a 4 CPU system and 400% on a 6 CPU system, we can easily see that it can at most do 4 things in parallel.