[BUG]: Relinquished instance exception for still alive object

[WillFroom]

Problem description

I am hitting an issue where nanobind is raising an exception when I am trying to use a C++ object that has already been on a round-trip to python and back but is now wrapped by a outer class.

I have attached a MRE, I get the following error on the use_inner_class line: RuntimeWarning: nanobind: attempted to access a relinquished instance of type ....

I believe what I am doing is valid, but if there is a way around this please let me know.

I haven't tried reverting it but I think that this change: #591 may be being to strict?

Reproducible example code

Extension:

struct InnerClass {};

struct OuterClass {
  std::unique_ptr<InnerClass> inner_class;
};

NB_MODULE(_extension, module_) {
  namespace nb = nanobind;

  nb::class_<InnerClass>(module_, "InnerClass");

  nb::class_<OuterClass>(module_, "OuterClass")
      .def("__init__",
           [](OuterClass* self, std::unique_ptr<InnerClass> inner_class) {
             new (self) OuterClass();
             self->inner_class = std::move(inner_class);
           })
      .def(
          "get_inner_class",
          [](OuterClass* self) { return self->inner_class.get(); },
          nb::rv_policy::reference_internal);

  module_.def("make_inner_class",
              []() { return std::make_unique<InnerClass>(); });

  module_.def("use_inner_class", [](const InnerClass* inner_class) {});
}

Failing test:

  import _extension

  inner_class = _extension.make_inner_class()
  outer_class = _extension.OuterClass(inner_class)
  _extension.use_inner_class(outer_class.get_inner_class()) // RuntimeWarning: nanobind: attempted to access a relinquished instance of type InnerClass

[wjakob]

Hello @WillFroom. First, a quick check before I look deeper into this: did you read about the nb_deleter in the documentation, and does the use of that deleter fix the issue?

Using unique casters without this special deleter has some inherent limitations that are discussed in the documentation sections on ownership.

[WillFroom]

Hi @wjakob, I did read the documentation about nb::deleter and did try updating the __init__ method for OuterClass to consume a std::unique_ptr<InnerClass, nb::deleter<InnerClass>> (among other combinations) and I still hit the same issue when I try and access the wrapped inner class.

[wjakob]

Ah. I think I understand. What you are trying to do is currently not supported by the library.

What is important to keep in mind is that nanobind associates at most one Python instance with each C++ object.

_extension.make_inner_class() creates a Python wrapper around the existing C++ pointer and transfers ownership from the C++ unique pointer to this wrapper.

outer_class = _extension.OuterClass(inner_class) transfers ownership back to C++ and marks the Python wrapper as invalid so that it cannot be used anymore.

outer_class.get_inner_class() returns a C++ pointer. nanobind checks to see if an instance wrapping this pointer exists. And indeed, there is one. It simply returns that instance.

.. but that instance is marked as invalid, so the use in _extension.use_inner_class fails.

The behavior that is tested and documented is that a type is consistently accessed using unique ownership. For example, if get_inner_class returned a unique pointer, we would revert back ownership to Python, and then this would work.

No proposal on how to fix this yet, this is just to explain the cause.

[wjakob]

Is it possible that your experiment works after adding:

del inner_class

just before the last line? That would confirm the hypothesis.

[WillFroom]

Adding del inner_class before trying to use the wrapped inner class does indeed stop the error being raised.

[wjakob]

Hello @WillFroom,

I thought about this issue some more, and it's complicated. I think it will need a significant redesign of how unique pointers are handed in nanobind.

Currently, instances have a "relinquished" flag to indicate that ownership has been transferred away. However, the data structures of nanobind still map the C++ instance pointer to this relinquished object, which prevents return via rv_policy::reference or rv_policy::reference_internal. To support your use case, we'd need a new mode for nb_inst instances that keeps them from being listed in the instance map, so that they cannot be found. Subsequently returning via rv_policy::reference_internal can then return a second Python object. However, the details of this are gnarly, likely has tons of corner cases, and it's possible that the whole notion of the relinquished flag needs to be dispensed with. The requirement for any fix would also be that it doesn't impact runtime performance of the general case (code that doesn't use unique pointers) or add huge amounts of code to the headers.

This isn't related to anything I need for my own work, and it will take a lot of time to understand and fix. I think I prefer to close it as "out of of (my) support scope" with an express invitation for external contributions.