Incorrect results when using MPS backend (MacOS)

[JAEarly]

TS2Vec uses the GELU activation function. Unfortunately, there's currently a bug in PyTorch for the GELU activation function on MPS devices (since April 2023).

This won't throw an error, just leads to rubbish results in both training and inference.

Hopefully this will be patched by PyTorch, but in the meantime a work around is to change F.gelu(x) to F.gelu(x.contiguous()) on lines 34 and 36 of models/dilated_conv.py . Results are then as expected.

I won't bother with a PR as I assume this will be patched by PyTorch in the (near) future.

More info:
pytorch/pytorch#98212
huggingface/transformers#22468

[ezpzbz]

I have not tried it yet but it seems fixed here