Skip to content

GLARING bug with the process data pipeline. #35

@MadcowD

Description

@MadcowD

Hey all! There is a glaring bug with the data_proccess pipeline.

Currently, for each clip, the video generator sorts the data by filename
ep_frame_paths = (glob(os.path.join(ep_dir, '*'))). If your files are written as

filename1
filename2
...
filenameN

and N > 10 this will interlace earlier frames of the data into the clips used for training. With a large history window this adds some robustness to the model, but it really invalidates the modeling assumption!

Sortiing by the last integer fixes this (change line 80 to this)

ep_frame_paths = sorted(glob(os.path.join(ep_dir, '*')), key=lambda x: int(x.split("frame")[-1].split(".png")[0]))

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions