Problems with torch 1.4

[vatai]

Jens reported this, so didn't check myself what's exactly happening. I suspect the following:

• old (e.g. 1.4) pytorch didn't have mkldnn backend, only had mkldnn tensor layout. and then newer pytorch had both.
• so old torch can only do mkldnn tensor layout (with .to_mkldnn() method), while newer version can also doo backend=mkldnn separately...
• what benchmarker messes up in your case with pytorch 1.4 is that it forces backend=mkldnn when you just want the .to_mkldnn() (i.e. --tensor_layout).

Some error messages:

jens@kiev0:~/fake_a64fx_cuda/dep/benchmarker|master! 
⇒  python3 -m benchmarker --mode=training --framework=pytorch --problem=resnet50 --problem_size=32 --batch_size=4 --backend=DNNL --tensor_layout=DNNL
Traceback (most recent call last):
  File "/home/jens/spack/opt/spack/linux-centos7-broadwell/gcc-7.4.0/python-3.7.6-l6ym2ggx7veduyeembjgckv5r4a4fuui/lib/python3.7/runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "/home/jens/spack/opt/spack/linux-centos7-broadwell/gcc-7.4.0/python-3.7.6-l6ym2ggx7veduyeembjgckv5r4a4fuui/lib/python3.7/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/home/jens/fake_a64fx_cuda/dep/benchmarker/benchmarker/__main__.py", line 114, in <module>
    main()
  File "/home/jens/fake_a64fx_cuda/dep/benchmarker/benchmarker/__main__.py", line 70, in main
    result = benchmarker.benchmarker.run(unknown_args)
  File "/home/jens/fake_a64fx_cuda/dep/benchmarker/benchmarker/benchmarker.py", line 120, in run
    benchmark = framework_mod.Benchmark(params, unknown_args)
  File "/home/jens/fake_a64fx_cuda/dep/benchmarker/benchmarker/frameworks/do_pytorch.py", line 70, in __init__
    self.setup_data_and_model()
  File "/home/jens/fake_a64fx_cuda/dep/benchmarker/benchmarker/frameworks/do_pytorch.py", line 96, in setup_data_and_model
    torch.backends.mkldnn.enabled = True
AttributeError: module 'torch.backends' has no attribute 'mkldnn'

jens@kiev0:~/fake_a64fx_cuda/dep/benchmarker|master!
⇒  python3 -m benchmarker --mode=training --framework=pytorch --problem=resnet50 --problem_size=32 --batch_size=4 --tensor_layout=DNNL
Traceback (most recent call last):
  File "/home/jens/spack/opt/spack/linux-centos7-broadwell/gcc-7.4.0/python-3.7.6-l6ym2ggx7veduyeembjgckv5r4a4fuui/lib/python3.7/runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "/home/jens/spack/opt/spack/linux-centos7-broadwell/gcc-7.4.0/python-3.7.6-l6ym2ggx7veduyeembjgckv5r4a4fuui/lib/python3.7/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/home/jens/fake_a64fx_cuda/dep/benchmarker/benchmarker/__main__.py", line 114, in <module>
    main()
  File "/home/jens/fake_a64fx_cuda/dep/benchmarker/benchmarker/__main__.py", line 70, in main
    result = benchmarker.benchmarker.run(unknown_args)
  File "/home/jens/fake_a64fx_cuda/dep/benchmarker/benchmarker/benchmarker.py", line 120, in run
    benchmark = framework_mod.Benchmark(params, unknown_args)
  File "/home/jens/fake_a64fx_cuda/dep/benchmarker/benchmarker/frameworks/do_pytorch.py", line 70, in __init__
    self.setup_data_and_model()
  File "/home/jens/fake_a64fx_cuda/dep/benchmarker/benchmarker/frameworks/do_pytorch.py", line 104, in setup_data_and_model
    torch.backends.mkldnn.enabled = False
AttributeError: module 'torch.backends' has no attribute 'mkldnn'

[undertherain]

this really should be the other way around
backend first, layout came alter
I'd assume at 1.4. there were none, we can surely do some try catch there

[vatai]

Thanks, I really wanted your input on this. It felt it should be the other way around... can it be that in old torch, the way to set mkldnn was .to_mkldnn instead of setting backend? But maybe I overlooked something

The two commits Jens compared:
e20f59d9c2326367eb7ce329b5a251e2ce61d1eb works
5e67fe4a7111d2b40a6b9314290ca1ba809543e7 broken

[vatai]

Ok. It is something funky indeed. Both commits have backend.mkldnn.enable = True etc. but for some reason the version that worked for Jens was not calling the backend.mkldnn.enable = True code, despite having --backend=DNNL set 😢

[undertherain]

why torch 1.4 btw? it's more ancient than dinosaurs

[vatai]

Yepp!!! 😁