Skip to content

added_filter_rules breaks with local_variables #2340

@achiefa

Description

@achiefa

When a filter rule containing local_variables is added to the added_filter_rules key in the runcard, validphys raises an error of this kind

  ...
  File "/opt/homebrew/Caskroom/miniconda/base/envs/nnpdf/lib/python3.12/site-packages/reportengine/configparser.py", line 491, in _resolve_key
    val = produce_func(**kwargs)
          ^^^^^^^^^^^^^^^^^^^^^^
  File "<string>", line 3, in __hash__
TypeError: unhashable type: 'dict'

The error is triggered by the functool.cache in produce_rules. The cache decorator will try to hash the function arguments to use them as hash keys. However, added_filter_rules is a tuple containing AddedFilterRule objects which have a dictionary within, namely the field local_variables. However, a dictionary is not hashable as it is mutable, and I think this error is generated by the dataclass' __hash__ method when it tries to hash all fields. Note that this does not happen with the default rules, because they do not enter as arguments of the function produce_rule, but are collected within the function with default_filter_rules_input, and thus they don't need to be hashed.

A possible solution that I've found is the following:

@dataclasses.dataclass(frozen=True)
class FilterRule:
    """
    Dataclass which carries the filter rule information.
    """

    dataset: str = None
    process_type: str = None
    rule: str = None
    reason: str = None
    local_variables: Mapping[str, Union[str, float]] = dataclasses.field(default=None, hash=False)
    PTO: str = None
    FNS: str = None
    IC: str = None

    def to_dict(self):
        rule_dict = dataclasses.asdict(self)
        filtered_dict = {k: v for k, v in rule_dict.items() if v is not None}
        return filtered_dict

In that way, we explicitly avoid hashing the dictionary. I tested this solution and it seems to work well.

cc @RoyStegeman @scarlehoff

Metadata

Metadata

Assignees

Labels

Type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions