Cannot yield Numpy arrays or Pandas data frames. Truth value is ambiguous

[thomas-maschler]

When i try to yield a Numpy array or Pandas data frame from a Stage I get an error.
It sais, the truth of Array or DataFrame is ambiguous. Use a.any() or a.all().

I can type cast results as normal list, and it works. Any chance there is a way to catch this error?

import numpy as np
from parallelpipe import stage

@stage()
def myfunc(prod):
    for i in prod:
        yield np.asarray(i)

prod = [[1,2],[2,3],[3,4]]
pipe = prod | myfunc()

for array in pipe.results():
    print(array)

[gtsystem]

Hi @thomas-maschler, I looked at your issue and it seems related to the "special" way numpy arrays behave when compared:

>>> import numpy as np
>>> a  = np.asarray([1,2])
>>> False == a
array([False, False])
>>> object() == a
array([False, False])

As you can see the == operator that should return a boolean returns an array of booleans and the built in iter function fails to verify that this object is equal or different from the sentinel.
As a workaround I would suggest you to "box" your array in a tuple and "unbox" the array when consuming the data:

@stage()
def myfunc(prod):
    for i in prod:
        yield (np.asarray(i),)

prod = [[1,2],[2,3],[3,4]]
pipe = prod | myfunc()

for (array,) in pipe.results():
    print(array)

[thomas-maschler]

Great, that works!
To evaluate numpy correctly you would need to do something like

a.all() == True

or

a.any() == True

depending on what you are looking for. I will go with the boxing.
Thanks again. Parallelpipe makes my life soo much easier!