function that returns PyObject* is not binded

[jwuttke]

A class member function that returns a PyObject* is not binded. If the return type is changed to void*, it is (but my Python application will segfault).

I found no single occurrence of "PyObject" in the Binder sources. So where does this exception come from?

Are there other ways to return a Numpy array?

[lyskov]

@jwuttke the PyObject should certainly be bound-able (i just double checked that in local test). Could it be that your functions have other arguments that might not be bound-able? To check if this is the case i would recommend to add simply dummy function like PyObject * test() and see if it got binded.

Are there other ways to return a Numpy array?

-- this might help: https://pybind11.readthedocs.io/en/stable/advanced/pycpp/numpy.html

[jwuttke]

class Datafield {
    ...
    void* test0() const;
    PyObject* test1() const;
};

result:

>>> d=b.Datafield.TestDF()
>>> d.test0()
>>> d.test1()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AttributeError: 'bornagainapi.Datafield' object has no attribute 'test1'. Did you mean: 'test0'?

[jwuttke]

@lyskov In your test, how did your forward declare PyObject? Or did you bind all of Python.h?

[jwuttke]

Here my own minimal example, which does not work:

src.h:

struct _object;
typedef _object PyObject;

class A {
public:
    A(int i_) : i(i_) {}
    void add(int j) { i +=j; }
    int operator()() const { return i; }
    PyObject* po() { return nullptr; }
private:
    int i;
};

Bindables.hpp:

#include <src.h>

CMakeLists.txt:

project(BindTest LANGUAGES CXX)

set(CMAKE_CXX_STANDARD 17)

set(BINDER binder)
set(BIND_DIR ${CMAKE_SOURCE_DIR})
set(AUTOBIND_DIR ${CMAKE_SOURCE_DIR}/auto)
set(lib bt)

if(NOT (BINDER STREQUAL "binder" OR BINDER STREQUAL "autobind"))
    message(FATAL_ERROR "Invalid BINDER=${BINDER}")
endif()

message(STATUS "Testing availablity of program '${BINDER}':")
execute_process(COMMAND ${BINDER} --version RESULT_VARIABLE ret)
if(ret)
    message(FATAL_ERROR "Program '${BINDER}' not found. Required with BINDER.")
endif()

file(REMOVE_RECURSE ${AUTOBIND_DIR})
file(MAKE_DIRECTORY ${AUTOBIND_DIR})

find_package(Python3 COMPONENTS Interpreter Development NumPy)

execute_process(COMMAND ${BINDER}
    --prefix ${AUTOBIND_DIR}
    --root-module ${lib}
    --bind ""
    --config ${BIND_DIR}/config.cfg
    -v
    ${BIND_DIR}/Bindables.hpp
    --
    -std=c++${CMAKE_CXX_STANDARD}
    -I${CMAKE_SOURCE_DIR} #-I${PYTHON3_INCLUDE_DIRS}
    COMMAND_ECHO STDOUT
    COMMAND_ERROR_IS_FATAL ANY
    )

if(NOT EXISTS ${AUTOBIND_DIR}/${lib}.cpp)
    message(FATAL_ERROR
        "File ${AUTOBIND_DIR}/${lib}.cpp not found. Reconfigure with BINDER.")
endif()


find_package(pybind11)

file(GLOB_RECURSE source_files ${AUTOBIND_DIR}/*.cpp)

pybind11_add_module(${lib} MODULE ${source_files})

target_link_libraries(${lib} PUBLIC ${PYTHON3_LIBRARIES})

target_include_directories(${lib} PUBLIC ${CMAKE_SOURCE_DIR})

Result:

>>> import bt
>>> a=bt.A(40)
>>> a.add(2)
>>> a()
42
>>> a.po()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AttributeError: 'bt.A' object has no attribute 'po'
>>> 

[lyskov]

re your example: this will indeed not work: in order to bind any non-primitive type Binder need its full definition. So in this case since no full definition was provided binding generation for any functions that take/return PyObject will be skipped.

re my example: - i just included #include <Python.h>, here is the outline:

#include <Python.h>
PyObject * py_object_test() { return new PyObject(); }

[jwuttke]

Would that work? Wouldn't the Python binding of PyObject behave differently than a PyObject?

[lyskov]

I am not sure what do you mean by "work" but if intention was to create Python types inside function and bind such function then no, - i do not think that will work. Unless i am mistaken instead the Pybind11 py::object should be used in such cases (see https://pybind11.readthedocs.io/en/stable/advanced/pycpp/object.html).

However, I would not recommend to bind such functions but instead provide them as custom-binders or add-on-binders for given classes or namespaces. Please see binder, add_on_binder and binder_for_namespace options here: https://cppbinder.readthedocs.io/en/latest/config.html#config-file-options . - that way project main C+ code could be kept free from Python related add-ons.