Multiple bugs with Optional nested dataclasses

[gottalottarock]

Describe the bug

1. post_init runs with None

If a nested dataclass inside an optional field defines a post_init depending on a required field, it gets called even when the optional container is None.

2. Ignored nested args

When passing only a nested argument (e.g. --datamodule.data_none.feature.path), the parser silently ignores it and keeps the optional container as None. Unless another field of that container (like --datamodule.data_none.name) is also provided.

To Reproduce

from simple_parsing import ArgumentParser, ArgumentGenerationMode, NestedMode, field, subgroups
from dataclasses import dataclass
from pathlib import Path

@dataclass
class Feature:
    path: Path
    name: str = field(init=False)

    def __post_init__(self):
        self.name = self.path.name

@dataclass
class Data:
    feature: Feature
    name: str = 'data'

@dataclass
class Datamodule():
    name: str = 'datamodule'
    data_none: Data | None = None

@dataclass
class Config():
    datamodule: Datamodule
    
def main(config: Config):
    print(config)


if __name__ == "__main__":
    parser = ArgumentParser(
        argument_generation_mode=ArgumentGenerationMode.NESTED,
        nested_mode=NestedMode.WITHOUT_ROOT,
    )
    parser.add_arguments(Config, dest="config")
    args = parser.parse_args()
    main(args.config)

1. Unexpected exception
Expected behavior

$ python issue.py --datamodule.name data
Config(datamodule=Datamodule(name='data', data_none=None))

Actual behavior
post_init of Feature runs with path=None and crashes:

$ python issue.py --datamodule.name data
Traceback (most recent call last):
  ...
AttributeError: 'NoneType' object has no attribute 'name'

Full traceback:

$ python simple-parsing-error.py --datamodule.name "data"                                                                                                                      
Traceback (most recent call last):                                                                                                                                             
File "/home/data/skuznetsov/conform-rank/simple-parsing-error.py", line 47, in <module>
  args = parser.parse_args()                                                                                                                                                 
         ^^^^^^^^^^^^^^^^^^^                                                                                                                                                 
File "/home/skuznetsov/miniconda3/lib/python3.12/argparse.py", line 1904, in parse_args
  args, argv = self.parse_known_args(args, namespace)                                                                                                                        
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^                                                                                                                        
File "/home/data/skuznetsov/conform-rank/.venv/lib/python3.12/site-packages/simple_parsing/parsing.py", line 361, in parse_known_args
  parsed_args = self._postprocessing(parsed_args)                                                                                                                            
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^                                                                                                                            
File "/home/data/skuznetsov/conform-rank/.venv/lib/python3.12/site-packages/simple_parsing/parsing.py", line 593, in _postprocessing
  parsed_args = self._instantiate_dataclasses(                                                                                                                               
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^                                                                                                                               
File "/home/data/skuznetsov/conform-rank/.venv/lib/python3.12/site-packages/simple_parsing/parsing.py", line 862, in _instantiate_dataclasses
  value_for_dataclass_field = _create_dataclass_instance(                                                                                                                    
                              ^^^^^^^^^^^^^^^^^^^^^^^^^^^                                                                                                                    
File "/home/data/skuznetsov/conform-rank/.venv/lib/python3.12/site-packages/simple_parsing/parsing.py", line 1160, in _create_dataclass_instance
  return constructor(**constructor_args)                                                                                                                                     
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^                                                                                                                                     
File "<string>", line 4, in __init__                                                                                                                                         
File "/home/data/skuznetsov/conform-rank/simple-parsing-error.py", line 11, in __post_init__
  self.name = self.path.name                                                                                                                                                 
              ^^^^^^^^^^^^^^                                                         
AttributeError: 'NoneType' object has no attribute 'name'

2. Nested args inside an Optional field are ignored unless another field is set
Expected behavior:

$ python issue.py --datamodule.name data --datamodule.data_none.feature.path /home
Config(datamodule=Datamodule(
  name='data',
  data_none=Data(feature=Feature(path=PosixPath('/home'), name='home'), name=data)
))

Actual behavior:
The nested argument is ignored, and data_none stays None:

$ python issue.py --datamodule.name data --datamodule.data_none.feature.path /home
Config(datamodule=Datamodule(name='data', data_none=None))

Only when another field is provided does it work:

$ python issue.py --datamodule.name data \
                  --datamodule.data_none.name name \
                  --datamodule.data_none.feature.path /home
Config(datamodule=Datamodule(
  name='data',
  data_none=Data(feature=Feature(path=PosixPath('/home'), name='home'), name='name')
))

Desktop (please complete the following information):

• Version 0.1.7
• Python version: 3.12.11

[gottalottarock]

Oh, wow, found one more. Maybe more interesting example for second case:

$ python simple-parsing-error.py --datamodule.name data --datamodule.data_none.feature.path /home \
                                                       --datamodule.data_none.name "data"
Config(datamodule=Datamodule(name='data', data_none=None))
$ python simple-parsing-error.py --datamodule.name data --datamodule.data_none.feature.path /home \
                                                       --datamodule.data_none.name "not data"
Config(datamodule=Datamodule(name='data', data_none=Data(feature=Feature(path=PosixPath('/home'), name='home'), name='not data')))

It seems that if the parameter matches the default value, the nested class is also not initialized.

[lebrice]

Hello @gottalottarock , thanks for posting, sorry for the delay.

Hmm. There's a lot to unpack here. For the time being, would you mind giving me a bit more info about your use-case? There might be a simpler way to do this in a way that works, for example using subgroups, instead of optional nested dataclasses.

Otherwise, I'm unfortunately unable at the moment to commit to implementing a fix for this in the near future. I'd welcome a contribution though if you're so inclined, I could give you some pointers, let me know. :)