[FEA] Support profiling with NVTX #137
Description
Is your feature request related to a problem? Please describe.
It would be nice to have profiling support so that we can analyze the performance easily (visualizing the bottleneck through NVIDIA Nsight Systems).
Describe the solution you'd like
- Make use of NVTX.
- Provide a build flag to enable/disable the use of NVTX.
- Check the overhead of NVTX and enable it by default if the overhead is not much
- Make runtime-configurable through an environment variable or API call.