Please consider using composition over inheritance

[NeilGirdhar]

In my opinion, frozendict shouldn't inherit from dict and therefore MutableMapping. Ideally, you would inherit from Mapping, and use composition over inheritance. I realize that may be a significant change, but it would make the code more correct.

from collections.abc import Mapping, MutableMapping
print(issubclass(frozendict, Mapping))  # True :)
print(issubclass(frozendict, MutableMapping))  # True, unfortunately.

This means doing something like:

class frozendict(Mapping):
    def __init__(...) -> ...:
        self._dict = dict(...)
    # Delegate __getitem__, __iter__, and __len__ to self._dict, along with any methods that would benefit like __contains__.

[Marco-Sulla]

First of all, this is not true for the C Extension.

About the pure py implementation, originally it was designed this way. But accessing _dict is very simple, and this makes the immutability really easy to circumvent.

So I tried to use MappingProxyType instead of dict for _dict. It was really slow. This is why I decided in the end to use inheritance.

PS: usually I prefer composition

[NeilGirdhar]

But accessing _dict is very simple, and this makes the immutability really easy to circumvent.

I don't think it's a reasonable requirement for the immutabilty to be impossible to circumvent. No other Python library goes to such an extent of preventing users from circumventing guarantees. In Python, this is known as the principle of consenting adults.

However, it is a problem for frozendict to inherit from MutableMapping. It will make static type checking not detect problems when a frozendict is passed to a function that requires a MutableMapping, and it will make things like singledispatch to fail.

So I tried to use MappingProxyType instead of dict for _dict. It was really slow. This is why I decided in the end to use inheritance.

I figured that was the reason. I'm surprised that even with a C extension, it's still too slow to use composition.

[Marco-Sulla]

I don't think it's a reasonable requirement for the immutabilty to be impossible to circumvent.

I think it's more than resonable, is a must-have for an immutable type.

No other Python library goes to such an extent of preventing users from circumventing guarantees.

The other libraries does not implements immutable types.

However, it is a problem for frozendict to inherit from MutableMapping. It will make static type checking not detect problems when a frozendict is passed to a function that requires a MutableMapping, and it will make things like singledispatch to fail.

Do you have practical use cases?

I'm surprised that even with a C extension, it's still too slow to use composition.

The C Extension does not need composition, since.... well, is written in C.

All you said applies only to the pure py version. The C extension is not affected by this problem:

$ python
Python 3.10.0 (heads/3.10-dirty:f6e8b80d20, Nov 18 2021, 19:16:18) [GCC 10.1.1 20200718] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import frozendict
>>> frozendict.c_ext
True
>>> from frozendict import frozendict as fd
>>> from collections.abc import MutableMapping
>>> issubclass(fd, MutableMapping)
False
>>> fd.__mro__
(<class 'frozendict.frozendict'>, <class 'object'>)
>>> 

[NeilGirdhar]

The other libraries does not implements immutable types.

I believe you're mistaken here. For example, a frozen dataclass, which is part of the standard library can have immutability circumvented using object.__setattr__.

The C Extension does not need composition, since.... well, is written in C.

Then what's the cost of fixing the Python version so that it's a Mapping but not a MutableMapping? If the C version is fast, what does it matter how the Python version is implemented?

All you said applies only to the pure py version. The C extension is not affected by this problem:

Great! So, you should at least register frozendict with Mapping: collections.abc.Mapping.register(frozendict). It costs nothing and it will answer isinstance checks correctly.

[Marco-Sulla]

I believe you're mistaken here. For example, a frozen dataclass, which is part of the standard library can have immutability circumvented using object.__setattr__.

Indeed I use it in my py code, or I can't even set private attributes. But there's a huge difference in using object.__setattr__, that is known only by few people, and be able to change a simple _dict attribute.

Then what's the cost of fixing the Python version so that it's a Mapping but not a MutableMapping? If the C version is fast, what does it matter how the Python version is implemented?

The C extension does not build on Windows, for example.

Great! So, you should at least register frozendict with Mapping: collections.abc.Mapping.register(frozendict). It costs nothing and it will answer isinstance checks correctly.

Already done, but thank you.

[NeilGirdhar]

Indeed I use it in my py code, or I can't even set private attributes. But there's a huge difference in using object.setattr, that is known only by few people, and simply be able to change a _dict attribute.

I don't think there's a difference. If your goal is to build some kind of security by obfuscation, then you can call the private variable _klajsdflkjas.

The C extension does not build on Windows, for example.

But this means that you different behavior on Windows then? isinstance(frozendict, MutableMapping) gives a different value depending on the platform?

Already done, but thank you.

Awesome!

[Marco-Sulla]

I don't think there's a difference. If your goal is to build some kind of security by obfuscation, then you can call the private variable _klajsdflkjas.

I'm quite sure that this is not "security by obscurity". Since this is Python, this is the best I found to create a reasonably immutable dict without sacrificing speed. Consider that also inheriting from Mapping and using a _dict is much more slow than inheriting from dict.

You can always use MappingProxyType. It's really slow, but it does not have this problem.

But this means that you different behavior on Windows then? isinstance(frozendict, MutableMapping) gives a different value depending on the platform?

This means that the pure py version has this problem, and, as I already said, I don't see any real and practical reasons to change that.

[Marco-Sulla]

MutableMapping fixed, thanks to @d-maurer: ad449be

[NeilGirdhar]

It's better than nothing, but that's a very flimsy way of fixing the problem. If another library comes along and tries that approach, it will clobber your change.

But I understand your desire for speed.

[Marco-Sulla]

If another library comes along and tries that approach, it will clobber your change.

How?

[Marco-Sulla]

If another library comes along and tries that approach, it will clobber your change.

How?

Now that I have more time I understand what you intended.

Listen to me. There are some points that maybe are not clear:

1. The pure py version is only here for the unlucky case the C Extension is not available to your machine and you can't compile it. It's a toy. I created a refactor branch that will make it possible to have a frozendict in C from Python 3.6+ on Linux, Windows and MacOS. And it's faster than dict in some cases, while it has the same speed in many others. The pure py version is the past, the C extension is the future.
2. This is not the only frozendict implementation in the world. If you don't like it, you can consider another package, or
3. this project is FOSS. You can create your fork and do whatever you want.

Good holidays.

[Marco-Sulla]

It's better than nothing, but that's a very flimsy way of fixing the problem. If another library comes along and tries that approach, it will clobber your change.

But I understand your desire for speed.

It's a late reply, but I post this because I think I must.

First of all, I misunderstood the post. I thought "flimsy" was an insult. My English was very bad and it's not improved. I use Context Reverso and AIs now.

I'm sorry. Really.

A normal reply should had been:

---

Yes, it's true. The monkeypatch and the fact that, in pure py, frozendict inherits from dict is fragile. Not only that, it's horrible:

1. As you said, if someone makes a change to MutableMapping, the monkeypatch will break
2. to do this, I also have to access private members of MutableMapping
3. It's terrible dict is in the MRO of the pure py impl
4. If in the future a mutating method is added to dict, I have to update the library. The old versions anyway will have this mutating method available
5. the C extension and the pure py versions are not 100% aligned this way

Anyway, you have also to consider this:

1. it's quite improbable someone will do the same monkeypatch to MutableMapping
2. Even if they do it, if they do it the right way, they will break nothing. It's really simple: they should not only substitute the old function, but save it and call it if the class is not their class, as I do it now.
3. The fact dict is a superclass of frozendict in the pure py version has a lot of advantages:
       - the pure py version is "immutable" this way. About this, dear user, read below
       - dict subclasses have a fast-path for many dict methods, so the pure py version is faster this way. And this is more crucial than in the past, because the C extension is stuck to 3.10.
       - many libraries that expects a dict, like json, will work out-of-the-box
       - practicality beats purity

About immutability: it's undeniable that the pure py version is more immutable this way, and the primary goal of this library is... well, having an immutable dict.

You firstly said no one pushed this so far to make an object immutable. Lately, you quoted @dataclass(frozen=True), that uses exactly the same trick I use. If so, I think you should write to the authors of dataclasses they are making a terrible, terrible mistake ;-)

About object.__setattr__, let me write in addition of what I already said in past that you can also mutate a tuple using ctypes, or, in general programming, using reflection. Like in the real world. No one can stop a seasoned thief from stealing a car, if they wants. So I suppose you can leave your car unlocked at Tor Bella Monaca too ;-)

About MappingProxyType, it's slow no more. Probably Python devs were motivated to make it and dict itself faster because of the C version of frozendict... who knows? ;-)
If this is true, it's a real win not only for frozendict, but for all Python users. For a short period, MappingProxyType had also a hash!
But it doesn't change the fact that, even if I will use MappingProxyType as a self._dict, it's the same because you can do myfrozendict._dict = "mbuti!".

Let me also say that your implementation of a frozendict with an has-in _dict is broken. You use __init__, while you strongly want __new__ instead. If you use __init__, you can call it another time to change the frozendict. You can say there's no reason to worry about it, because you can easily change _dict anyway. But I wanted to be academic this time ;-)

Anyway, as I said, the pure py version is a little toy. It's only 254 lines, and I suspect that the 99% of people uses only less than one half of them, and in a wrong way :D

Anyway, as you can see, I left the discussion open and I'll continue to leave it this way, because I'm always open to change my mind. I'm against censorship or absolutisms -- even if my last reply can be seen as the opposite! :-P 

I'm still continuing to develop the C extension, even if my time is very short now, but I don't think it will ever be implemented in CPython. At least not before I died X-D I don't really understand why, but anyway, hey, that's life.

Furthermore, as a separate project, I'm free to add functional programming methods, like set and delete, that I copied from frozenmap, and setdefault. I would also like to add an update. These methods will never be implemented in a highly hypothetical Cpython frozendict. Maybe in frozenmap, that I will very like to see in the stdlib. But since people prefer having a frozendict, probably frozenmap will never be added too. And this is very sad, even if, I have to admit, I like frozendict much more. Not only because I consider it a mine creature.

Anyway, I prefer a billion times having frozenmap in Python, a real builtin immutable dict, instead of having to do the horrible trick of using a MappingProxyType and delete the underlying dict after I initialized it! Definitively unpythonic, unintuitive and not beautiful.

Furthermore, frozenmap is easier to write :D