Issue with dynamic type information when returning derived class object from C++ to Python

[jeongseok-meta]

I'm having an issue with pybind11 where the dynamic type information of an object seems to be lost when the object is returned from C++ to Python. I have a base class NodeFactory and a derived class PythonNodeFactory in Python. The NodeFactory class has a pure virtual function load that is overridden in PythonNodeFactory.

Here's a simplified version of my code:

C++ code:

class Node
{
public:
  virtual ~Node() = default;
  virtual void foo() = 0;
};

class NodeTrampoline : public Node
{
public:
  using Node::Node;

  void foo() override
  {
    PYBIND11_OVERRIDE_PURE(void, Node, foo);
  }
};

class NodeFactory
{
public:
  using NodePtr = std::shared_ptr<Node>;
  virtual ~NodeFactory() = default;
  virtual NodePtr load(const std::string& name) = 0;
};

class NodeFactoryTrampoline : public NodeFactory
{
public:
  using NodeFactory::NodeFactory;

  NodePtr load(const std::string& name) override
  {
    PYBIND11_OVERRIDE_PURE(NodePtr, NodeFactory, load, name);
  }
};

PYBIND11_MODULE(example, m)
{
  py::class_<Node, NodeTrampoline, std::shared_ptr<Node>>(m, "Node")
    .def(py::init<>())
    .def("foo", &Node::foo);

  py::class_<NodeFactory, NodeFactoryTrampoline, std::shared_ptr<NodeFactory>>(m, "NodeFactory")
    .def(py::init<>())
    .def("load", &NodeFactory::load);

  m.def("create_node", [](NodeFactory& factory, const std::string& name) {
    return factory.load(name);  // maybe lost type information at here
  });
}

Python code:

class PythonNode(Node):
  def __init__(self):
    Node.__init__(self)

  def foo(self):
    pass

class PythonNodeFactory(NodeFactory):
  def __init__(self):
    NodeFactory.__init__(self)

  def load(self, name):
    return PythonNode()  # called

node = create_node(PythonNodeFactory(), "my_node")
print(type(node).__name__) # Prints "Node" instead of "PythonNode"

In this code, create_node calls the load method on a PythonNodeFactory object, which returns a PythonNode object. However, when I print the type of the returned object in Python, it prints Node instead of PythonNode.

I've confirmed that the correct load method is being called, so it seems like the issue is with how the return type is being handled.

Does anyone know how I can preserve the dynamic type information of the returned object so that it is correctly recognized as a PythonNodeFactory in Python?

[JunkyByte]

Hello I think I have a similar problem and I'm seeking guidance! In my case my setup is similar to yours but I have the problem that when I use

.def("__getitem__", [](const DataFrame &df, const KeyType &key) {
    return df(key);
})

The instance returned is always from the original c++ class instead of the class DataFrame(CppDataFrame): python class :(

For instance I use the python class directly like

class DataFrame(CppDataFrame):
 ...
 
# I use it as 
df = DataFrame(...)
# (...)
print(type(df), type(df['x']))
>>> <class 'statistics_extension.DataFrame'> <class 'statistics_bindings.CppDataFrame'>

as you can see I cannot preserve the statistics_extensions.DataFrame instance and it gets back to the c++ only counterpart...