Automatic mixed precision in PyTorch using AMD GPUs — ROCm Blogs

[giscus[bot]]

Automatic mixed precision in PyTorch using AMD GPUs — ROCm Blogs

In this blog, we will discuss the basics of AMP, how it works, and how it can improve training efficiency on AMD GPUs. As models increase in size, the time and memory needed to train them--and consequently, the cost--also increases. Therefore, any measures we take to reduce training time and memory usage can be highly beneficial. This is where Automatic Mixed Precision (AMP) comes in.

https://rocm.blogs.amd.com/artificial-intelligence/automatic-mixed-precision/README.html

[cccat6]

Not sure but seems nobody cares ROCm, and thats why nobody figured out the code in this article is based on CUDA but not ROCm. Who wrote this article? Are you doing the copy paste and even not looking at the content?

[ic]

There is indeed little love for ROCm. But the article is "correct" on using torch.cuda, assuming the code uses the ROCm version of PyTorch (e.g. from https://download.pytorch.org/whl/nightly/rocm6.4/). I find it lazy and annoying, but it does work. A pity they have not yet entered the torch.accelerator list in PyTorch's documentation.

[alexjude4480-creator]

Not sure but seems nobody cares ROCm, and thats why nobody figured out the code in this article is based on CUDA but not ROCm. FHSEOHUB Who wrote this article? Are you doing the copy paste and even not looking at the content?

[williejoe1889-dev]

Not sure but seems nobody cares ROCm, and thats why nobody figured out the code in this article is based on CUDA but not ROCm. Free Who wrote this article? Are you doing the copy paste and even not looking at the content?

[larryjack2998-eng]

Not sure but seems nobody cares ROCm, and thats why nobody figured out the code in this article is based on CUDA but not ROCm. APK Who wrote this article? Are you doing the copy paste and even not looking at the content?