python-devel June 2019

python-devel@lists.fedoraproject.org

7 participants
16 discussions

Python 3.7's Deterministic pycs
by Petr Viktorin 15 Jul '19

15 Jul '19

Hello! The first beta for Python 3.7 is out. It will hopefully get into Fedora soon as python37. After it comes out of beta, we'll upgrade python3 to it. The What's New list is at: https://docs.python.org/3.7/whatsnew/3.7.html One thing that's interesting for packagers is PEP 552: Deterministic pycs: https://www.python.org/dev/peps/pep-0552/ Let me summarize in my own words. A new opt-in mode for byte-compilation makes .pyc (bytecode cache) files depend only on the contents of the corresponding source file. If we use this, it will slow down imports, because the whole source file would need to be read and hashed in order to verify if a .pyc file is valid. (Currently, metadata like the modification time and file size is used.) To speed things up, there's an option, UNCHECKED_HASH, which skips cache validation entirely. Using this would mean that if you modify a .py source file installed by RPM, the changes wouldn't take effect (the .py contents would only be shown in tracebacks). Modifying installed files in production is extremely bad practice, of course, but it's quite useful for debugging on throw-away systems. If we adopt UNCHECKED_HASH, anyone doing it will have to remember to remove the corresponding .pyc file. Honestly, I'm not sure we want to use this in Fedora. Is anyone here into reproducible builds, to make a better argument for this? -- Petr Viktorin

5 5

The state of Python 3.8 in Fedora 31
by Miro Hrončok 03 Jul '19

03 Jul '19

Hello, When I filed the Python 3.8 [change] for Fedora 31, we knew that the schedule would be tight. For that very reason, we have not yet started to build for Python 3.8 in a f31 side tag, but instead we've only been doing it in [copr] so far. The mass rebuild happens on 2019-07-24, according to the Fedora 31 [schedule]. That gives us about 1 month + 1 week to be able to merge the side tag back in case we decide to start building it now. There are several challenges: - there are ~200 build failures that block this, tracked on [bugzilla] - further there are about ~300 packages that are blocked by the above, possibly more - some of the packages are quite crucial to make this happen (tornado, pygobject3) - the 3.8.0 [releases] have been delayed so far, so continuing the trend, we could very well end up with the first RC just one day before the F31 Final Freeze or even after that (risking 3.8 beta in Fedora 31 GA) I've met with Petr Viktorin and Tomáš Orsava today and we are prepared to deffer this change to Fedora 32, unless there is a large push-back against that. However we don't want this to be an internal decision behind closed doors, so we are sharing it with you and we are happy to reconsider, in case there is something that we haven't anticipated. What would that mean: - we would continue to build the packages in [copr] as new Python 3.8 beta versions are released - we would continue to report build failures and to provide pointers to affected maintainers - right after the F32 branching (2019-08-13 according to the [schedule]), we would start with the side tag builds - the Koji builds would start ~2 months later Benefits: - we would not be stressed by the immediate mass rebuild deadline - we would not need to care about ABI incompatibilities between beta releases, because the last beta should be out when we start Negatives: - the users would get 3.8 as the main python3 about 6 months later, but they already have Python 3.8 interpreter in Fedora to develop on We could of course just start building now and than decide not to merge the side tag, but we are worried that it would leave a big mess in git repos and RPM changelogs. If you think this is not a wise decision and would prefer to have Python 3.8 in Fedora 31 (as the main python3), please discuss quickly. The F31 mass rebuild is approaching fast and there's a lot to be done, so every day counts now; in other words, the later you present your argument, the stronger it must be ;) [change] https://fedoraproject.org/wiki/Changes/Python3.8 [copr] https://copr.fedorainfracloud.org/coprs/g/python/python3.8/ [schedule] https://fedoraproject.org/wiki/Releases/31/Schedule [bugzilla] https://bugzilla.redhat.com/showdependencytree.cgi?id=PYTHON38&hide_resolve… [releases] https://www.python.org/dev/peps/pep-0569/#schedule Miro Hrončok, on behalf of the Fedora's Python SIG and Red Hat's Python Maintenance team -- Phone: +420777974800 IRC: mhroncok

1 1

[HEADS UP] Trailing zeros are stripped from Python provides/requires
by Miro Hrončok 30 Jun '19

30 Jun '19

Hey, pip and Python packages generally deem 5.4.0 == 5.4. In order to allow processing != dependencies in Python generators, we now strip the trailing zero(s) for automatic Python provides and requires. E.g. python3-josepy-1.1.0-9.fc31 provides python3.7dist(josepy) = 1.1 What I wrongfully have not anticipated at first is that now some dependent packages can break if they are no both rebuilt. Packages rebuilt prior to this change might require stuff like "python3.7dist(josepy) >= 1.1.0". I'm sorry for that breakage, it should not be very common. Rebuilding the package with such broken dependencies should fix it and evrything should be fine after the mass rebuild. There is a very little change when a package cannot be rebuilt to be fixed, because some other package (needed to build this one) suffers from the same problem. Let me know if you need my help to untagle any possible mess in this. See https://src.fedoraproject.org/rpms/python-rpm-generators/c/ff085a044d19dd40… https://github.com/rpm-software-management/rpm/pull/757 https://github.com/rpm-software-management/rpm/issues/639 Sorry about the breakage. -- Miro Hrončok -- Phone: +420777974800 IRC: mhroncok

1 0

Re: [Python-Dev] Re: [RELEASE] Python 3.7.4rc1 and 3.6.9rc1 are now available
by Miro Hrončok 27 Jun '19

27 Jun '19

On 26. 06. 19 23:59, Ned Deily wrote: > On Jun 25, 2019, at 18:10, Miro Hrončok <mhroncok(a)redhat.com> wrote: >> I'm working on getting 3.7.4rc1 to Fedora rawhide (the "master branch" of Fedora). If nothing goes wrong with the build, it should land within a day. There are checks in Fedora that should uncover any suspicious test failures of our Python packages, if that happens, we'll report back hopefully before 2019-06-28 AOE. > > Thanks for the heads-up, Miro! I'm hoping to be able to start the manufacture of 3.7.4 final a bit early so we can release on 06-28, it would be very helpful if you could pass along any potential showstopper problems as soon as you can and before approximately 06-28 1600Z. There are couple Fedora packages that started to fail around the time when we updated: https://apps.fedoraproject.org/koschei/affected-by/python3?epoch1=0&version… (However, they can be, and mostly are, unrelated to Python 3.4.7 update.) Fox example pipenv is a false positive, pyqt5 fails on Python 2 for unrelated reason etc. One package that seems weird is python-dask, where there are some memory errors and threading errors: https://kojipkgs.fedoraproject.org/work/tasks/8942/35858942/build.log Not sure if this is random or related to 3.7.4, running a couple more builds now. -- Miro Hrončok -- Phone: +420777974800 IRC: mhroncok

1 1

[HEADS UP] Sphinx updated to 2.1
by Miro Hrončok 26 Jun '19

26 Jun '19

I've just pushed and built python-sphinx-2.1.2-1.fc31 for rawhide. I don't anticipate any breakage. In case unexpected breakage happens, let me know. https://src.fedoraproject.org/rpms/python-sphinx/pull-request/12 https://bugzilla.redhat.com/show_bug.cgi?id=1716158 Thanks, -- Miro Hrončok -- Phone: +420777974800 IRC: mhroncok

1 0

Re: [EPEL-devel] Re: RHEL 7.7 Beta
by Miro Hrončok 24 Jun '19

24 Jun '19

On 19. 06. 19 23:58, Miro Hrončok wrote: > On 19. 06. 19 19:22, Kevin Fenzi wrote: >> On 6/12/19 1:48 AM, Miro Hrončok wrote: >>> On 12. 06. 19 8:14, Thomas Stephen Lee wrote: >>>> Hi, >>>> >>>> Just for your information. >>>> >>>> https://www.redhat.com/en/blog/red-hat-enterprise-linux-77-beta-now-availab… >>>> >>>> >>>> Python 3.6 is there in 7.7 >>>> >>>> https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/7-be… >>>> >>>> >>>> >>>> thanks >>> >>> In that case, let's proceed with: >>> >>> https://src.fedoraproject.org/rpms/python-rpm-macros/pull-request/19 >>> https://src.fedoraproject.org/rpms/python-rpm-macros/pull-request/20 >>> https://src.fedoraproject.org/rpms/python34/pull-request/35 >>> >>> Starting here: https://bugzilla.redhat.com/show_bug.cgi?id=1719633 >> >> But we don't want to actually land that until 7.7 is out right? > > We do. It is designed to work, after 7.7 GA python-rpm-macros package gets > retired on EPEL. Everything ready as an EPEL7 update, with buildroot overrides: https://bodhi.fedoraproject.org/updates/FEDORA-EPEL-2019-f505f7d2fb If you get some weird failures, please report it there. -- Miro Hrončok -- Phone: +420777974800 IRC: mhroncok

1 0

Should python3-rpm-macros depend on python3?
by Miro Hrončok 19 Jun '19

19 Jun '19

We have an interesting request for python3-rpm-macros to depend on python3. See https://bugzilla.redhat.com/show_bug.cgi?id=1563789 Highlights: - users who build for Python 3 are told (in the guidelines) to BR python3-devel (that brings in both python3 and python3-rpm-macros) - the existence of python3-rpm-macros is left as an implementation detail for ^ - in theory, those macros can be used with other Pythons (such as pypy3 or python36, that is most likely not done in practice) Arguments: - to require: the macros are broken without a python3 interpreter - to not: the macros should work with any python3 interpreter Solutions? - declare direct BR of macros without a python3 interpreter unsupported - add dependency on python3. unused if used with another interpreter - add a common virtual provide for all python3 interpreters or require (python3 or pypy3 or python36...) - very tedious -- Miro Hrončok -- Phone: +420777974800 IRC: mhroncok

2 4

HEADS UP: python-tornado updated to 6
by Miro Hrončok 10 Jun '19

10 Jun '19

Hey, I've just built updated tornado in rawhide. It is big update (5 to 6). See https://www.tornadoweb.org/en/stable/releases/v6.0.0.html Highlights: * no more Python 2 * deprecated APIs removed Report any problems to https://bugzilla.redhat.com/show_bug.cgi?id=1600318 -- Miro Hrončok -- Phone: +420777974800 IRC: mhroncok

1 0

Re: Heads-up: rpm 4.15 alpha coming to rawhide
by Neal Gompa 10 Jun '19

10 Jun '19

On Mon, Jun 10, 2019 at 8:28 AM Panu Matilainen <pmatilai(a)redhat.com> wrote: > > > > A pile of language-specific macros and scripts have been eliminated from > rpm upstream, notably %__python and %__perl and everything built around > them, such as %python_sitelib and %perl_sitelib and their relatives. > Python packages should be using the version specific macros already, > provided by respective pythonN-devel packages so this shouldn't be an > issue. On perl-side, those macros have been temporarily added to > redhat-rpm-config instead to avoid breaking things right now, later to > be moved to final destination in perl-macros or such. > Unversioned Python macros need to get added to python-rpm-macros, as there are a number of packages that do rely on their existence (as a means of supporting only a single version of Python depending on distro releases or legacy reasons). CC'd python-devel, Miro, and Igor about this. -- 真実はいつも一つ！/ Always, there's only one truth!

2 2

numpy+wx breaks pillow on ppc64le: cannot allocate memory in static TLS block
by Elliott Sales de Andrade 06 Jun '19

06 Jun '19

I'm not sure in which package the error lies here and thus where to file the bug report. This occurs on only ppc64le with a matplotlib build [1]. Does anyone have any ideas? This works: >>> import wx >>> from PIL import Image and this works: >>> import numpy >>> from PIL import Image but if you import both first, then it fails: >>> import numpy, wx # or wx, numpy >>> from PIL import Image Traceback (most recent call last): File "<stdin>", line 1, in <module> File "/usr/lib64/python3.7/site-packages/PIL/Image.py", line 93, in <module> from . import _imaging as core ImportError: /lib64/libgomp.so.1: cannot allocate memory in static TLS block Also, if you import pillow first, it works fine. Using the ppc64le test machine with: python3-numpy-1.16.3-1.fc31.ppc64le python3-pillow-6.0.0-2.fc31.ppc64le python3-wxpython4-4.0.6-1.fc31.ppc64le [1] https://koji.fedoraproject.org/koji/taskinfo?taskID=35329049 -- Elliott

1 0

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

python-devel June 2019