doctify mystery - it's adding keys to a dict

[patrickwolf]

The code mystifies me @rsgalloway

shotgrid/lib/shotgrid/base.py

Lines 70 to 75 in 963fa2c

	def _set_data(self, data: dict):
	"""Sets data.
	:param data: data dictionary
	"""
	self.data = dotdictify(data)

self.data == None
True
data
{'type': 'PublishedFile', 'code': 'DEV_1000_comp_PIPE_v001.mov'}
data2 = dotdictify(data)
data2
{'type': 'PublishedFile', 'code': 'DEV_1000_comp_PIPE_v001.mov'}
self.data = data
self.data
{'type': 'PublishedFile', 'code': 'DEV_1000_comp_PIPE_v001.mov'}
self.data = data2
self.data
{'type': 'PublishedFile', 'code': 'DEV_1000_comp_PIPE_v001.mov', 'id': {}, 'name': {}, 'content': {}}

do you have any idea of what is happening there?

[rsgalloway]

@patrickwolf the doctictify class is just a way to use dot-notation to access keys inside a data dict. if the key doesn't exist it will return en empty dict:

>>> life = {
...     'bigBang': {
...         'stars': {
...             'planets': {}
...         }
...     }
... }
>>> data = dotdictify(life)
>>> print(data.bigBang.stars.planets)
{}
>>> data.bigBang.stars.planets.earth = {'singleCellLife': 1}
>>> print(data.bigBang.stars.planets)
{'earth': {'singleCellLife': 1}}
>>> print(data.bigBang.stars.planets.earth.singleCellLife)
1
>>> print(data.bigBang.stars.planets.earth.foobar)
{}

[patrickwolf]

@rsgalloway okay solved the mystery

from pprint import pprint
from shotgrid.dotdictify import dotdictify
life = {
    'bigBang': {
        'stars': {
            'planets': {}
        }
    }
}
print("life before: ",life)
data = dotdictify(life)
print("data before: ", data)
a = data.bigBang.stars.planets.earth.singleCellLife
data.bigBang.sun = 'hot'
print("life after: ",life)
print("data after: ", data)

result

life before:  {'bigBang': {'stars': {'planets': {}}}}
data before:  {'bigBang': {'stars': {'planets': {}}}}
life after:  {'bigBang': {'stars': {'planets': {}}}}
data after:  {'bigBang': {'stars': {'planets': {'earth': {'singleCellLife': {}}}}, 'sun': 'hot'}}

after a dict is dotdictifyed

1. assigning a value changes only the new dict
2. accessing a property creates the key in the new dict

i can understand 1) even though i had assumed it would work on the original dict instead of on the copy

and with 2) i do consider it a bug
this line should not have side-effects
a = data.bigBang.stars.planets.earth.singleCellLife
but it generates the same keys as if i had written
data.bigBang.stars.planets.earth.singleCellLife = {}

at least now the mystery is solved what created those keys on assignment of self.data... i have properties that access id,code,content and the debugger would have read those properties which in turn magically created these keys i then had to remove before i could save the object again

[rsgalloway]

thanks for the breakdown, @patrickwolf . I probably should have been more thorough in my response. regarding 2, yes, it will create missing keys. this was by design, similar in spirit to collections.defaultdict:

    def __getitem__(self, key):
        found = self.get(key, dotdictify.marker)
        if found is dotdictify.marker:
            found = dotdictify()
            dict.__setitem__(self, key, found)
        return found

dotdictify was copied over from a very old gist I had laying around:

https://gist.github.com/rsgalloway/107efbef258a28470bdb

I will investigate either making this behavior more customizable, or switching to using defaultdict instead.