Hello Pythonistas and Rustaceans,
TL;DR: Only PyO3 v0.19.2 (and later) will ever properly support Python 3.12. Port your Python projects to v0.19 **NOW**.
Older versions of PyO3 (especially pyo3 v0.15, v0.16, v0.17, and v0.18) are *not* compatible with Python 3.12 due to some ABI changes in unicode strings and behavioural changes wrt/ "immortal" objects. This also affects all current versions of the "cpython" Rust bindings, with no timeline for Python 3.12 support.
As far as I can tell, extensions that use pyo3 < v0.19 or the "cpython" bindings can (and likely will) not work as expected on Python 3.12 if they use the affected APIs (either by producing garbage data in strings that are passed over the FFI boundary, or by crashing).
There are applications in Fedora that still rely on *ancient* versions of PyO3, potentially affected by this:
- cpython: mercurial - pyo3 v0.15: fapolicy-analyzer, python-bcrypt, python-cryptography - pyo3 v0.16: python-y-py - pyo3 v0.17: unused compat packages, will be retired - pyo3 v0.18: matrix-synapse
I *stongly* recommend to move all of these packages to pyo3 v0.19 in Rawhide as soon as possible. I will try to submit pull requests with the required changes for affected packages (except mercurial, since there's no version of the "cpython" crate that supports Python 3.12 in sight).
There's already a few packages that depend on pyo3 v0.19, which I will rebuild in rawhide for pyo3 v0.19.2, which has much better support for Python 3.12 than v0.19.0 and v0.19.1 (breezy, python-rpds-py, orjson) unless there are any objections.
As soon as no packages depend on the compat packages for old versions of pyo3 any longer, I will retire them from Rawhide (and F39, depending on the timing), since they will never work with Python 3.12 and nothing should use them.
I've added <package>-maintainers@fedoraproject.org for all these packages to the CC of this message.
Fabio Rust SIG / PyO3 maintainer in Fedora
On Tue, Aug 1, 2023 at 5:43 PM Fabio Valentini decathorpe@gmail.com wrote:
There are applications in Fedora that still rely on *ancient* versions of PyO3, potentially affected by this:
- cpython: mercurial
- pyo3 v0.15: fapolicy-analyzer, python-bcrypt, python-cryptography
- pyo3 v0.16: python-y-py
- pyo3 v0.17: unused compat packages, will be retired
- pyo3 v0.18: matrix-synapse
Hello again.
It is now three months later, and three packages have moved to the latest available version of pyo3:
- python-cryptography - python-y-py - matrix-synapse
That leaves two packages that are stuck on pyo3 v0.15:
- fapolicy-analyzer - python-bcrypt
I have now filed bugs against both packages that they need to move to pyo3 v0.19.2+ ASAP on both Fedora 39 and Rawhide. Due to ABI changes in Python 3.12, they are not guaranteed to even work correctly on Python 3.12.
- https://bugzilla.redhat.com/show_bug.cgi?id=2249378 - https://bugzilla.redhat.com/show_bug.cgi?id=2249381
================================================================================
The packages for v0.16 and v0.18 of pyo3 are now unused in Fedora Rawhide - I will retire them later today.
The packages for pyo3 v0.15 will be retired in about *ONE WEEK*, but no earlier than Monday, Nov 20. They are known to be problematic and broken with Python 3.12, so no packages should use them.
================================================================================
That leaves the cpython crate and its only dependent package - mercurial. The upstream project for the cpython crate has been marked as no longer actively maintained, and recommends users to switch to pyo3 instead: https://github.com/dgrunwald/rust-cpython/commit/e815555
I've now also filed a bug against mercurial: https://bugzilla.redhat.com/show_bug.cgi?id=2249383
Fabio
On Sun, Nov 12, 2023 at 12:35 PM Fabio Valentini decathorpe@gmail.com wrote:
On Tue, Aug 1, 2023 at 5:43 PM Fabio Valentini decathorpe@gmail.com wrote:
There are applications in Fedora that still rely on *ancient* versions of PyO3, potentially affected by this:
- cpython: mercurial
- pyo3 v0.15: fapolicy-analyzer, python-bcrypt, python-cryptography
- pyo3 v0.16: python-y-py
- pyo3 v0.17: unused compat packages, will be retired
- pyo3 v0.18: matrix-synapse
Hello again.
It is now three months later, and three packages have moved to the latest available version of pyo3:
- python-cryptography
- python-y-py
- matrix-synapse
That leaves two packages that are stuck on pyo3 v0.15:
- fapolicy-analyzer
- python-bcrypt
I have now filed bugs against both packages that they need to move to pyo3 v0.19.2+ ASAP on both Fedora 39 and Rawhide. Due to ABI changes in Python 3.12, they are not guaranteed to even work correctly on Python 3.12.
The mainline git version of python-bcrypt has already been adapted for python 3.12, they just haven't made a release yet, it seems?
https://github.com/pyca/bcrypt
That said, it doesn't look like it needed code changes to make that work, so it should be possible to just bump things?
python-devel@lists.fedoraproject.org