Extending Profile Ranges

[kharvey2]

At its current state, the ProfileRange wrapper is applied to all cuPyNumeric operations and it can identify whether a cuPyNumeric operation executed in deferred or eager mode, as well as whether it fell back (and why). For more information, this is the design doc.

I’d like to open a discussion about how we should extend it further. Some questions for consideration:

• Should the profile pop-up boxes for cuPyNumeric operations also display what hardware is being used (CPU/CPU OMP/GPU)?
• Should the profile pop-up boxes include what tasks are being called by the cuPyNumeric operation?
• Right now this tool is primarily helpful for outside users, since the _fixup_co_name decorator in coverage.py hides internal code. Should we consider creating a version of profile ranges that is more useful for internal developers?
• When a user calls a cuPyNumeric function, the functions used to construct the called function also appear in the profiling output. Should we hide these helper functions to reduce noise?

Would love your thoughts on this, @bryevdv @Jacobfaib

[Jacobfaib]

Should the profile pop-up boxes for cuPyNumeric operations also display what hardware is being used (CPU/CPU OMP/GPU)?

That would be really useful yeah. This would allow people at-a-glance to see where stuff is being offloaded to.

When a user calls a cuPyNumeric function, the functions used to construct the called function also appear in the profiling output. Should we hide these helper functions to reduce noise?

Yes, let's make it only start showing the backtrace from where the original function is "defined". Otherwise it'll just be more noise to ignore, as you say.

[lightsighter]

Should the profile pop-up boxes include what tasks are being called by the cuPyNumeric operation?

This relationship actually already exists in the profiles in the opposite direction (i.e. with the creator relationship). The challenge with inverting it is how to render the case where a box creates many tasks. If someone can articulate how exactly they would want that rendered we can start a discussions about if/how that is possible.

[kharvey2]

Are there other things we should add?

[kharvey2]

PRs for the extensions:

• Tracking which variant is used (C++/OMP/CUDA): https://github.com/nv-legate/cupynumeric.internal/pull/993
• Flag to show/hide cuPyNumeric operations that were used to create the top-level cuPyNumeric operation to the profiling output: https://github.com/nv-legate/cupynumeric.internal/pull/1003