docs: add guide chapter detailing current way to share classes between modules

[davidhewitt]

I had some time recently to investigate #1444 and plan how this might be one day addressed inside PyO3.

I came up with a protoype which shows a current (working) solution to this, with a bunch of limitations and caveats.

This PR writes up documentation about that prototype and proposes future work we could pursue to make this pattern easier in future. (It's hard, not supported well by rustc, cargo, or us, so lots of surface for possible improvement.)

[davidhewitt]

Guide deploy is at https://69d0b3d06941c83c959718a6--pyo3.netlify.app/main/advanced/sharing-types.html

[Tpt]

Thank you for the great writing.

I tend to feel using the cpython API is sometime a bit simpler than relying on the C-API.

For example if base-package expose a constant __version__ = (major, minor, patch, abi) and then classes like:

class Foo:
    @staticmethod
    def __from_capsule__(capsule: CapsuleType) -> Self: ...
    def __capsule__(self) -> CapsuleType: ...

then external crates can just do py.import("base_package").getattr("__version__") and py.import("base_package").getattr("Foo").call_method1("__from_capsule__", (my_capsule,)) (the actual repr(C) type for Foo being defined in a shared base-package-ffi crate or something similar). This is the approach used by Arrow and some other libraries.

To me, it feels slightly simple and less error prone than the C-ABI way but does not change much.

[Tpt]

Do we guarantee that Py is ffi-safe? If yes, it might be worth it to mention it in its documentation.

[davidhewitt]

Good idea, added some documentation for this.

[davidhewitt]

Thanks for the review!

Interesting idea to use __version__ constant at the Python level, I hadn't considered that. I think I prefer to keep it inside the capsule, because otherwise it's possible to have Python code reassign __version__ to defeat the safety checks. Agreed it's more fiddly, but this is all fairly fiddly FFI code :/

[codspeed-hq]

Merging this PR will improve performance by 13.8%

⚠️ Different runtime environments detected

Some benchmarks with significant performance changes were compared across different runtime environments,
which may affect the accuracy of the results.

Open the report in CodSpeed to investigate

⚡ 1 improved benchmark
✅ 104 untouched benchmarks
⏩ 1 skipped benchmark¹

Performance Changes

	Benchmark	BASE	HEAD	Efficiency
⚡	bench_pyclass_create	4.6 µs	4 µs	+13.8%

---

_{Comparing davidhewitt:pyclass-sharing-doc (8773874) with main (bd3dab0)}

Footnotes

1. 1 benchmark was skipped, so the baseline result was used instead. If it was deleted from the codebase, click here and archive it to remove it from the performance reports. ↩