Training data loading error - "Can't pickle..."

[michaeljones1]

Hi,

Firstly, thanks for making all the code available!

I'm getting an error that I can't solve when trying to run python step_recognition/main.py --config step_recognition/configs/miniroad_assembly101-O.yaml

I get the error:

Epoch:1 Training: 0%| | 0/9815 [00:00<?, ?it/s, lr: 0.0001000] Traceback (most recent call last): File "C:\Users\DotaPC\PREGO\step_recognition\main.py", line 91, in <module> epoch_loss = train_one_epoch( ^^^^^^^^^^^^^^^^ File "C:\Users\DotaPC\PREGO\step_recognition\trainer\train.py", line 10, in train_one_epoch for it, (rgb_input, flow_input, target, vid, start, end) in enumerate(tqdm(trainloader, desc=f'Epoch:{epoch} Training', postfix=f'lr: {optimizer.param_groups[0]["lr"]:.7f}')): File "C:\Users\DotaPC\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.11_qbz5n2kfra8p0\LocalCache\local-packages\Python311\site-packages\tqdm\std.py", line 1181, in __iter__ for obj in iterable: File "C:\Users\DotaPC\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.11_qbz5n2kfra8p0\LocalCache\local-packages\Python311\site-packages\torch\utils\data\dataloader.py", line 493, in __iter__ return self._get_iterator() ^^^^^^^^^^^^^^^^^^^^ File "C:\Users\DotaPC\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.11_qbz5n2kfra8p0\LocalCache\local-packages\Python311\site-packages\torch\utils\data\dataloader.py", line 424, in _get_iterator return _MultiProcessingDataLoaderIter(self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\DotaPC\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.11_qbz5n2kfra8p0\LocalCache\local-packages\Python311\site-packages\torch\utils\data\dataloader.py", line 1170, in __init__ w.start() File "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.11_3.11.2544.0_x64__qbz5n2kfra8p0\Lib\multiprocessing\process.py", line 121, in start self._popen = self._Popen(self) ^^^^^^^^^^^^^^^^^ File "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.11_3.11.2544.0_x64__qbz5n2kfra8p0\Lib\multiprocessing\context.py", line 224, in _Popen return _default_context.get_context().Process._Popen(process_obj) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.11_3.11.2544.0_x64__qbz5n2kfra8p0\Lib\multiprocessing\context.py", line 336, in _Popen return Popen(process_obj) ^^^^^^^^^^^^^^^^^^ File "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.11_3.11.2544.0_x64__qbz5n2kfra8p0\Lib\multiprocessing\popen_spawn_win32.py", line 95, in __init__ reduction.dump(process_obj, to_child) File "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.11_3.11.2544.0_x64__qbz5n2kfra8p0\Lib\multiprocessing\reduction.py", line 60, in dump ForkingPickler(file, protocol).dump(obj) _pickle.PicklingError: Can't pickle <class 'datasets.dataset.THUMOSDataset'>: it's not the same object as datasets.dataset.THUMOSDataset PS C:\Users\DotaPC\PREGO> Traceback (most recent call last): File "<string>", line 1, in <module> File "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.11_3.11.2544.0_x64__qbz5n2kfra8p0\Lib\multiprocessing\spawn.py", line 122, in spawn_main exitcode = _main(fd, parent_sentinel) ^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.11_3.11.2544.0_x64__qbz5n2kfra8p0\Lib\multiprocessing\spawn.py", line 132, in _main self = reduction.pickle.load(from_parent) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ EOFError: Ran out of input

I think the error is to do with the dataset not being the same format when loaded back in from when it was saved (see: https://stackoverflow.com/questions/1412787/picklingerror-cant-pickle-class-decimal-decimal-its-not-the-same-object#:~:text=Pickle%20requires%20you%20to%20have,and%20before%20you%20unpickle%20it.&text=Try%20fiddling%20with%20your%20imports%2C%20it%20might%20correct%20the%20problem.)

Have you had this error in the past and do you have a fix? Would really appreciate it if you have time!