Loss of type-hinting benefits

[edan-bainglass]

I have two methods in my Gradient class that I would like to overload as scalar_product:

def scalar_gradient_product(self, gradient: Gradient) -> Density:
    ...

and

def scalar_density_product(self, density: Density) -> ndarray:
    ...

I decorated both with multimethod as follows:

@multimethod
def scalar_product(self, gradient: Gradient) -> Density:
    ...

and

@multimethod
def scalar_product(self, density: Density) -> ndarray:
    ...

This works, i.e. calling scalar_product with Gradient or Density arguments yields the expected respective results. However, I seem to have lost my type-hinting benefits. My editor (VS code) now thinks scalar_product is a property with a return type of multimethod or MethodType.

Am I missing something? Is there some way of gaining the overloading functionality without losing type-hinting?

Thanks in advance 🙂

[coady]

I don't think it's possible. scalar_product is a single multimethod object. So even if it did some sort of code generation with annotations on the __call__ method, it's not clear what it they would be. E.g., would the return type be Union[Density, ndarray].

[edan-bainglass]

Not sure. I'm not familiar with how it works on your end. There are several NumPy methods/functions that can be overloaded. np.array and np.zero for example. Not sure how NumPy implements the overloading.

I'll dig into it if I have time. In any case, thanks for responding :)

[edan-bainglass]

Here's an example of np.array introspection...

Note how it doesn't provide a Union of returns, but rather multiple versions of the overloaded method/function. This is done through the use of the typing module's @overload decorator in the respective stub file.

I tried manually creating the overloads for the scalar_product methods in the corresponding stub file, but it is still not quite doing what I want it to do, i.e. the above behavior for np.array.

[uuirs]

Hi @edan-bainglass, I might be able to provide a tricky method for you to try.
Add a decorator to pass through the type, in my case, it just like:

@innocent(multimethod)
def to_chunks(
    data: Union[Dict, List], chunk_size: int = 1 << 16
) -> Iterable[Tuple[List, List]]:
    raise NotImplementedError("Unknown data type")


@innocent(to_chunks.register)
def _(data: list, chunk_size: int = 1 << 16) -> Iterable[Tuple[List, List]]:
    pass


@innocent(to_chunks.register)
def _(data: dict, chunk_size: int = 1 << 16) -> Iterable[Tuple[List, List]]:
    pass

which the decorator is:

P = ParamSpec("P")
R = TypeVar("R")


def innocent(wrapper: Callable) -> Callable:
    def _innocent(f: Callable[P, R]) -> Callable[P, R]:
        nonlocal wrapper

        @wrapper
        @functools.wraps(f)
        def _wrapper(*args: P.args, **kwargs: P.kwargs) -> R:
            return f(*args, **kwargs)

        return _wrapper

    return _innocent

now the type hint works, it should also work on overloading.

Here is overloading version(but this version will cause the mypy check to fail, maybe need some cast or bound):

@overload
def to_chunks(data: Dict, chunk_size: int) -> Iterable[Tuple[List, List]]:
    ...

@overload
def to_chunks(data: List, chunk_size: int) -> Iterable[Tuple[List, List]]:
    ...

#Here is the code above

[coady]

AFAICT, typing.overload doesn't do anything; editors must be statically checking for it. Tried the same example with functools.singledispatch and VSCode showed

instance _SingleDispatchCallable(*args: Any, **kwargs: Any) -> _T