Question/Feature request: Iterable usage in `require` and `snapshot` decorators

[claudio-ebel]

Question

How do I use Iterable in a @snapshot or in a @require decorator? I didn't find any hints in the documentation so I assume I either used them wrong or didn't understand a core concept. Otherwise, I would like to propose a feature request 😀

Examples for @ensure

If I try to take a snapshot of an Iterable, it is not possible since snapshot consumes it. Using the file iter_in_ensure.py

# ––– file iter_in_ensure.py
from collections.abc import Iterable
from icontract import ensure, snapshot
from more_itertools import always_iterable


@snapshot(lambda i: list(i))
@ensure(lambda result, OLD: set(OLD.i) == set(result))
def ensure_iter(i: Iterable[int]) -> list[int]:
    return list(i)


assert 1 == len(ensure_iter(always_iterable(1)))

in a python environment where icontract and more_itertools is included, the error message is:

$ python iter_in_ensure.py
Traceback (most recent call last):
  File "iter_in_ensure.py", line 13, in <module>
    assert 1 == len(ensure_iter(always_iterable(1)))
  File "/…/icontract/_checkers.py", line 649, in wrapper
    raise violation_error
icontract.errors.ViolationError: File iter_in_ensure.py, line 8 in <module>:
set(OLD.i) == set(result):
OLD was a bunch of OLD values
OLD.i was [1]
i was <tuple_iterator object at 0x7fe0fe9afb20>
result was []
set(OLD.i) was {1}
set(result) was set()

showing that the @snapshot decorator already consumed the iterator, leaving an empty result back. A possible workaround is to use itertools.tee inside the function:

# ––– file fixup_ensure.py
from collections.abc import Iterable
from icontract import ensure
from itertools import tee
from more_itertools import always_iterable


@ensure(lambda result: set(result[0]) == set(result[1]))
def ensure_iter(i: Iterable[int]) -> tuple[list[int], Iterable[int]]:
    tee0, tee1 = tee(i, 2)
    return list(tee0), tee1


assert 1 == len(ensure_iter(always_iterable(1))[0])

but that requires to change the functions's signature for usage in @ensure only which – at least in my opinion – contradicts icontract's philosophy.

Examples for @require

With the @require decorator, I even didn't find a workaround:

# ––– file iter_in_require.py
from collections.abc import Iterable
from icontract import require
from more_itertools import always_iterable


@require(lambda i: 1 == len(list(i)))
def require_iter(i: Iterable[int]) -> list[int]:
    return list(i)


length = len(require_iter(always_iterable(1)))
assert 1 == length, f"result length was {length}"

results in

$ python iter_in_require.py
Traceback (most recent call last):
  File "iter_in_require.py", line 13, in <module>
    assert 1 == length, f"result length was {length}"
AssertionError: result length was 0

showing that @require already consumed the iterator and the function require_iter has no chance to access it again.

Versions

• Python: Python 3.10.4
• more-itertools: 8.13.0
• icontract: 2.6.1

Feature pitch

If I didn't miss anything, there are features missing for the @snapshot and the @require function decorators. I suggest to introduce additional arguments to disable iterator consumption.

A possible example usage for @snapshot:

from collections.abc import Iterable
from icontract import ensure, snapshot
from more_itertools import always_iterable


@snapshot(lambda i: list(i), iter='i')  # <- new argument 'iter'
@ensure(lambda result, OLD: set(OLD.i) == set(result))
def ensure_iter(i: Iterable[int]) -> list[int]:
    return list(i)


assert 1 == len(ensure_iter(always_iterable(1)))

A possible example usage for @require:

from collections.abc import Iterable
from icontract import require
from more_itertools import always_iterable


@require(lambda i: 1 == len(list(i)), iter='i')  # <- new argument 'iter'
def require_iter(i: Iterable[int]) -> list[int]:
    return list(i)


length = len(require_iter(always_iterable(1)))
assert 1 == length, f"result length was {length}"

The new argument iter indicates to use an independent iterator. In @require's case, it forwards it to the decorated function.

I'm aware that the proposed solution is not applicable to all iterables, but I'm still convinced it would pose an improvement.

[claudio-ebel]

If there really are features missing, I gladly offer my help and contribution by implementing those features!
Maybe the name tee as parameter instead of iter shows more explicit what it does – simply splitting the original iterator. A usage of tee: str | Iterable[str] as parameter signature could be handy, the first type providing a single, the second multiple iterator names.

[mristin]

@claudio-ebel thanks for sharing your thoughts! It is indeed a missing feature.

Some two years ago I spent quite some time thinking about it. However, what seems to work on some cases turns out to be inappropriate for other cases. For example, what if there are two iterators in the input and an iterator as the output and all these three somehow correlate?

I came to the conclusion that iterators, which have side effects by their nature, are simply not a good fit for design-by-contract (similar to multi-threading). The complexity of many contracts in this context can not be captured by the simplistic logic of require/snapshot/ensure. Now enter behavioral sub-typing and inheritance and it becomes all very, very confusing :-).

This does not mean that this shouldn't be covered by icontract; but I am still fuzzy on how to really approach the problem.

Do you have a couple of practical examples where you wanted to impose contracts on iterators? Maybe let's start from those and see where that journey takes us.

[claudio-ebel]

@mristin Thank you very much for your openness and this kind offer. I eagerly accept it! Indeed, I have a few practical examples. I am on vacation right now and have afterwords a stressful week at my job, but afterwords (July) I'd like to look at the project code and get myself used with the coding style and conventions therein. You prefer that I create a branch for this feature/issue, right? I'd create a few examples (as you suggested) there and maybe even a corresponding solution. Then it's all nicely separated…

[mristin]

@claudio-ebel a separate branch would be great! Take your time! Let's first think about use cases and at the conceptual level -- coding is the easy part once that is all figured out :-).