Sharing Vocab and config?

[Stefano-retinize]

Hey,
I wanted to make the h3m model work, I followed all the steps to get the dataset and run the 3 steps to then build dataset from this repository. But I ended with two problems:

1. All the config.yaml files are different than the model_h3D_mask_TRUE_TF0.7_D10 provided, so I've been trying to guess what are the hidden_dim, embbeded_dim and others.
2. The Vocab Size is different than what this model_h3D_mask_TRUE_TF0.7_D10 was trained on. I ended with around 5800 vocab size. Which is different than that model.

Is there any chance you could provide both vocab object and the config for that model?

Thanks in advance for your help!

[rd20karim]

Hey @Stefano-retinize,

The dataset building process is not only for constructing the vocabulary but also for ensuring that the download and processing of HumanML3D have been completed correctly to generate the corrected sentences for the h3D dataset. Otherwise, additional errors may occur when simply loading the vocabulary object. In any case, I will share the vocabulary object as soon as possible.

The h3D model is exactly the one available here one drive link.
Its config is given here: path_config_h3D

Regarding the vocabulary size mismatch, could you share the relevant output you get when running the build_data.py?

[Stefano-retinize]

Hey @rd20karim . Thanks for your reply. And thanks for sharing that vocab object.
I uploaded both all_humanML3D.npz and sentences_corrections_h3d.csv files generated to drive so you can take a look if you want.
But since my goal is to try to extrapolate your model to other dataset (always in SMPL character), I think all would be solved with that vocab object only.

Thanks for your help

[rd20karim]

Hey @Stefano-retinize,
You can download the language object from this one drive link.

[Stefano-retinize]

Hey @rd20karim ,

I coudl succesfully load the object into the script in this code Line58 I modified it to be:
`self.lang.build_vocabulary(min_freq=min_freq)

with open(r"C:\Users\jack\Desktop\IA\Edinburgh\M2T-Segmentation\self_lang_GRU_h3D_.pkl", "rb") as f:
self.lang = pickle.load(f)
self.corrected_sentences = self.lang.corrected_sentences`

assuming the build_vocabulary is not needed because the object is ready to use. But turns out that if I inspect self.lang.corrected_sentences is an empty list. I was expecting the object came with this part inside. And even if I load the object before the build_vocabulary the same happens when I inspect self.lang.corrected_sentences.

Please your help!!

[rd20karim]

Hey @Stefano-retinize,
As a quick workaround, you could try instantiating the object using the sentences you received earlier, self.lang.corrected_sentences= read(sentences_corrections_h3d.csv)

[Stefano-retinize]

hey @rd20karim, yeah I did, but I ended with this error:

 Traceback (most recent call last):
  File "C:\Users\jack\Desktop\IA\Edinburgh\M2T-Segmentation\visualizations\poses2concepts.py", line 213, in <module>
    loaded_model, train_data_loader, test_data_loader,val_data_loader = load_model_config(device=device,args=args)
  File "C:\Users\jack\Desktop\IA\Edinburgh\M2T-Segmentation\src\evaluate_m2L.py", line 83, in load_model_config
    loaded_model.load_state_dict(new_dict)
  File "C:\Users\jack\anaconda3\envs\M2T-Segmentation\lib\site-packages\torch\nn\modules\module.py", line 2041, in load_state_dict
    raise RuntimeError('Error(s) in loading state_dict for {}:\n\t{}'.format(
RuntimeError: Error(s) in loading state_dict for seq2seq:
        size mismatch for dec.embedding.weight: copying a param with shape torch.Size([3605, 128]) from checkpoint, the shape in current model is torch.Size([5793, 128]).
        size mismatch for dec.fc.weight: copying a param with shape torch.Size([3605, 640]) from checkpoint, the shape in current model is torch.Size([5793, 640]).
        size mismatch for dec.fc.bias: copying a param with shape torch.Size([3605]) from checkpoint, the shape in current model is torch.Size([5793]).

which I think has to do with my vocab build somehow is bigger than is suposed to be, it actually is printing before the error:
[INFO] VOCAB SIZE = 5793

Could you maybe provide your sentences_corrections_h3d.csv? This should solve everything, I think. Thanks for your help so far!

[Stefano-retinize]

@rd20karim,

do you think sharing the sentences_corrections_h3d.csv is something possible to do?

Thanks again

[rd20karim]

@Stefano-retinize
I don't think I have the rights to share the annotations for HumanML3D derived from the AMASS dataset. However, I believe the reason for the error might be in the data download step. Something could be missing or incorrectly configured, which would otherwise result in no errors.