Skip to content

File compression robustness #169

@epaillas

Description

@epaillas

I was thinking that compressing the model parameters with this function:

x = cls.compress_x(paths=paths, cosmos=cosmos, n_hod=n_hod, phase=phase, seed=seed)

Is not super robust against potential holes in the list of HODs for a given statististic. For instance, let's say somebody measures 100 HOD per cosmology in total, but by mistake forgets about processing a certain HOD index among the first 100 (maybe they forgot hod003), so the "first 100 HODs" in their list are now different than the first 100 HODs assumed in the compress_x function. That is, they still measured 100 HODs from the 500 ones that are available, but their indices are shifted because they missed one or more along the way.

Back them, I was passing this dict

hods[cosmo_idx] = [int(f.stem.split('hod')[-1]) for f in filenames]

to the compress_x function, so that the HOD list is always guaranteed to match, but it seems we are not using it anymore...

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions