Missing Data Files for Preprocessing Script

[juanitapuentes]

Hi!

First of all, great job on this project! I really appreciate the work you've put into it.

I am trying to run 0_preprocess_training_data.py, but I can’t find the following files:

• processed_30fps_no_skating/ (directory)
• egoalgo_no_skating_dataset.hdf5
• egoalgo_no_skating_dataset_files.txt

Could you let me know where to find them or how to generate them?

Thanks!

[brentyi]

Hi @juanitapuentes, thanks for asking and sorry for the delay in getting back to you here!

How we process the dataset is:

• Download AMASS
• Run the HuMoR preprocessing script; we put the output of this into processed_30fps_no_skating (the "no_skating" is because we run the cleanup_amass_data.py script which removes sequences with a lot of foot skating, like ice skating etc) (see below for link to updated script)
• Run 0_preprocess_training_data.py, which gives us the .hdf5 and .txt files

The training and eval code then both take the .hdf5 and .txt files as input.

Because of the AMASS license, we unfortunately can't put up a public link to the preprocessed dataset. #5 is also a related issue; sorry that this is a little bit of a hassle...

[Tengbo-Yu]

Hi!
Did you rename the folder after you finished running process_amass_data.py? Because there is no folder named neutral in the dataset I downloaded, but the command assert "neutral" in str(npz_path) exists in the 0_preprocess_training_data.py.

Or did I download the wrong dataset? I downloaded is the SMPL+ H G dataset in AMASS. Should I download SMPL+ H N? But there are only few datasets with SMPL+ H N.

[brentyi]

Hi @Tengbo-Yu, I made #15 in part in response to your questions/issues. Can you see if the changes there fix your problems?

[Tengbo-Yu]

Cool! Thanks for your update!!!

[Tengbo-Yu]

Oh! There seems to a mistake in your 0a_preprocess_training_data.py. In Humor process_amass_data.py line 600, they only process */*_poses.npz but not all .npz files (like shape.npz in BMLhandball dataset) as some of them don't have trans and mocap_framerate keys. Hope this is helpful to you!

[brentyi]

I just pushed a fix, thanks @Tengbo-Yu! I think we missed this because we were always running this script with multiple devices/multiprocessing, which resulted in silent failures for npz files with missing keys.