Python 3.7's Deterministic pycs
by Petr Viktorin
Hello!
The first beta for Python 3.7 is out. It will hopefully get into Fedora
soon as python37.
After it comes out of beta, we'll upgrade python3 to it.
The What's New list is at: https://docs.python.org/3.7/whatsnew/3.7.html
One thing that's interesting for packagers is PEP 552: Deterministic
pycs: https://www.python.org/dev/peps/pep-0552/
Let me summarize in my own words.
A new opt-in mode for byte-compilation makes .pyc (bytecode cache) files
depend only on the contents of the corresponding source file.
If we use this, it will slow down imports, because the whole source file
would need to be read and hashed in order to verify if a .pyc file is
valid. (Currently, metadata like the modification time and file size is
used.)
To speed things up, there's an option, UNCHECKED_HASH, which skips cache
validation entirely. Using this would mean that if you modify a .py
source file installed by RPM, the changes wouldn't take effect (the .py
contents would only be shown in tracebacks).
Modifying installed files in production is extremely bad practice, of
course, but it's quite useful for debugging on throw-away systems. If we
adopt UNCHECKED_HASH, anyone doing it will have to remember to remove
the corresponding .pyc file.
Honestly, I'm not sure we want to use this in Fedora. Is anyone here
into reproducible builds, to make a better argument for this?
--
Petr Viktorin
4 years, 2 months
Re: Python 3.8 failures to triage -- packages failing to build
by Miro Hrončok
On 29. 05. 19 0:59, Randy Barlow wrote:
> On Fri, 2019-05-03 at 15:56 +0200, Miro Hrončok wrote:
>> python-mongoengine bowlofeggs echevemaster yograterol
>
> I discovered that this was FTBFS even on 3.7 due to a missing BR[0]. I
> fixed that just now, so please try again in Copr.
It works. Thanks.
--
Miro Hrončok
--
Phone: +420777974800
IRC: mhroncok
4 years, 3 months
Python 3.8 failures to triage -- packages failing to build
by Miro Hrončok
Hey,
the following packages failed Python 3.8 rebuild but still needs triage.
(Involved package maintainers are bcc'ed.)
Me and Patrik (cc'ed) slowly go trough them and file bugzillas blocking PYTHON38:
https://bugzilla.redhat.com/showdependencytree.cgi?id=PYTHON38&hide_resol...
However the amount is quite high and the triage is tedious. I'd appreciate help
from the package maintainers or volunteers.
Those are the packages that very likely actually fail to build (i.e. they don't
just miss a dependency rebuild). This was determined by the build.log.gz
Content-Length header, so it is not 100% accurate, but (actually very well)
educated guess.
What to do?
1. Find the package in Copr:
https://copr.fedorainfracloud.org/coprs/g/python/python3.8/builds/
2. Examine the failure in the logs.
3. File bugzilla blocking the PYTHON38 bug. In case the failure is reproducible
in rawhide (still Python 3.7), also block the F31FTBFS bug. See exisitng
bugzillas for examples.
The copr has a mock config in its description. Use it for debugging if you want.
If you manage to fix something immediately instead of filing the bugzilla
report, please let us know so we can rebuild it in the copr.
Thanks for help.
Maintainers by package:
clufter jpokorny
dlib bizdelnick
elastic-curator piotrp
gdcm ankursinha ignatenkobrain mrceresa sebp
gitg decathorpe ignatenkobrain nacho pwalter rathann
graphite-web jamielinux jsteffan piotrp
greenwave dcallagh gnaponie lholecek mjia mprahl ralph
grpc avsej
hokuyoaist rmattes
hplip jridky tkorbar twaugh zdohnal
ibus-cangjie bochecha
kdevelop-python dvratil jgrulich minh
koschei mizdebsk msimacek
libcec pbrobinson
libforensic1394 fab
libpreludedb totol
libpst carllibpst
libstoragemgmt cathay4t grover tasleson
lttng-ust dmarlin greenscientist ktdreyer mjeanson suchakra
meld cwickert dmaphy gilboa lkundrak
mlpack rcurtin
openslide-python bgilbert
patool eclipseo
petsc4py sagitter
pybind11 jussilehtola
pykde4 jreznik rdieter than
pymol timfenn
pystatgrab fab heliocastro potty slankes ttorling
python-CacheControl bkabrda
python-Pympler rathann
python-adal melmorabity
python-alchimia vpopovic
python-aodhclient apevec
python-apsw cicku dfateyev maci
python-asynctest rathann
python-bash8 flepied
python-bashate apevec chandankumar
python-behave churchyard
python-binaryornot pingou
python-bitstruct besser82
python-blosc tnorth zbyszek
python-breathe daveisfera
python-castellan apevec hguemar
python-colorspacious rathann
python-dbfread jujens
python-deap zbyszek
python-django-appconf mrunge
python-django-countries bkabrda
python-epub athoscr
python-fastavro ankursinha
python-flake8-polyfill cottsay
python-fsleyes-widgets ankursinha
python-gattlib limb
python-gfm thm
python-grapefruit kumarpraveen pjp sundaram
python-into-dbus-python grover ignatenkobrain ilgrad
python-jeepney churchyard
python-jenkins-job-builder ignatenkobrain ktdreyer pabelanger
python-jep raphgro
python-jira ishcherb ralph stevetraylen
python-joblib besser82 ignatenkobrain sergiopr
python-journal-brief twaugh
python-junit_xml jhogarth
python-mapnik tomh
python-marshmallow-enum ignatenkobrain
python-mongoengine bowlofeggs echevemaster yograterol
python-mutagen cicku ignatenkobrain mbaldessari moezroy
python-neovim asn salimma
python-networkmanager jdulaney
python-nixio ankursinha
python-ntlm3 jhogarth
python-numcodecs qulogic
python-os-testr apevec chandankumar
python-pdir2 carlwgeorge
python-pelican churchyard firemanxbr jamielinux mrunge
python-pint mrunge
python-precis_i18n michich
python-pydicom ignatenkobrain
python-pykalman ankursinha
python-pymssql melmorabity
python-pyswip thofmann
python-pytest-spec lbalhar
python-pytest-testmon churchyard
python-qtconsole nonamedotc
python-rebulk jorti
python-recommonmark jujens
python-restauth-common jkaluza
python-ripozo churchyard
python-ruamel-yaml chandankumar jchaloup smilner
python-scales filabrazilska lkundrak
python-scrapy echevemaster
python-simpleeval sharkcz
python-snuggs qulogic
python-sphinxcontrib-programoutput itamarjp zbyszek
python-sphinxtesters jjames
python-structlog piotrp
python-suds jortel swt2c
python-svgwrite vtrefny
python-swiftclient apevec jruzicka zaitcev
python-terminaltables terjeros
python-tw2-core lmacken ralph
python-txaio jujens
python-units ignatenkobrain
python-vulture ykarel
python-wand barracks510
python3-exiv2 asn
python3-postgresql hhorak
vint fszymanski
waiverdb dcallagh gnaponie lholecek mprahl ralph
Packages by maintainer:
ankursinha gdcm python-fastavro python-fsleyes-widgets python-nixio python-pykalman
apevec python-aodhclient python-bashate python-castellan python-os-testr
python-swiftclient
asn python-neovim python3-exiv2
athoscr python-epub
avsej grpc
barracks510 python-wand
besser82 python-bitstruct python-joblib
bgilbert openslide-python
bizdelnick dlib
bkabrda python-CacheControl python-django-countries
bochecha ibus-cangjie
bowlofeggs python-mongoengine
carllibpst libpst
carlwgeorge python-pdir2
cathay4t libstoragemgmt
chandankumar python-bashate python-os-testr python-ruamel-yaml
churchyard python-behave python-jeepney python-pelican python-pytest-testmon
python-ripozo
cicku python-apsw python-mutagen
cottsay python-flake8-polyfill
cwickert meld
daveisfera python-breathe
dcallagh greenwave waiverdb
decathorpe gitg
dfateyev python-apsw
dmaphy meld
dmarlin lttng-ust
dvratil kdevelop-python
echevemaster python-mongoengine python-scrapy
eclipseo patool
fab libforensic1394 pystatgrab
filabrazilska python-scales
firemanxbr python-pelican
flepied python-bash8
fszymanski vint
gilboa meld
gnaponie greenwave waiverdb
greenscientist lttng-ust
grover libstoragemgmt python-into-dbus-python
heliocastro pystatgrab
hguemar python-castellan
hhorak python3-postgresql
ignatenkobrain gdcm gitg python-into-dbus-python python-jenkins-job-builder
python-joblib python-marshmallow-enum python-mutagen python-pydicom python-units
ilgrad python-into-dbus-python
ishcherb python-jira
itamarjp python-sphinxcontrib-programoutput
jamielinux graphite-web python-pelican
jchaloup python-ruamel-yaml
jdulaney python-networkmanager
jgrulich kdevelop-python
jhogarth python-junit_xml python-ntlm3
jjames python-sphinxtesters
jkaluza python-restauth-common
jortel python-suds
jorti python-rebulk
jpokorny clufter
jreznik pykde4
jridky hplip
jruzicka python-swiftclient
jsteffan graphite-web
jujens python-dbfread python-recommonmark python-txaio
jussilehtola pybind11
ktdreyer lttng-ust python-jenkins-job-builder
kumarpraveen python-grapefruit
lbalhar python-pytest-spec
lholecek greenwave waiverdb
limb python-gattlib
lkundrak meld python-scales
lmacken python-tw2-core
maci python-apsw
mbaldessari python-mutagen
melmorabity python-adal python-pymssql
michich python-precis_i18n
minh kdevelop-python
mizdebsk koschei
mjeanson lttng-ust
mjia greenwave
moezroy python-mutagen
mprahl greenwave waiverdb
mrceresa gdcm
mrunge python-django-appconf python-pelican python-pint
msimacek koschei
nacho gitg
nonamedotc python-qtconsole
pabelanger python-jenkins-job-builder
pbrobinson libcec
pingou python-binaryornot
piotrp elastic-curator graphite-web python-structlog
pjp python-grapefruit
potty pystatgrab
pwalter gitg
qulogic python-numcodecs python-snuggs
ralph greenwave python-jira python-tw2-core waiverdb
raphgro python-jep
rathann gitg python-Pympler python-asynctest python-colorspacious
rcurtin mlpack
rdieter pykde4
rmattes hokuyoaist
sagitter petsc4py
salimma python-neovim
sebp gdcm
sergiopr python-joblib
sharkcz python-simpleeval
slankes pystatgrab
smilner python-ruamel-yaml
stevetraylen python-jira
suchakra lttng-ust
sundaram python-grapefruit
swt2c python-suds
tasleson libstoragemgmt
terjeros python-terminaltables
than pykde4
thm python-gfm
thofmann python-pyswip
timfenn pymol
tkorbar hplip
tnorth python-blosc
tomh python-mapnik
totol libpreludedb
ttorling pystatgrab
twaugh hplip python-journal-brief
vpopovic python-alchimia
vtrefny python-svgwrite
ykarel python-vulture
yograterol python-mongoengine
zaitcev python-swiftclient
zbyszek python-blosc python-deap python-sphinxcontrib-programoutput
zdohnal hplip
--
Miro Hrončok
--
Phone: +420777974800
IRC: mhroncok
4 years, 4 months
python-more-itertools - dropped Python 2 support?
by Scott Talbert
Hi,
It seems that you just updated python-more-itertools and dropped Python 2
support. Unfortunately, this breaks quite a few packages - namely
python2-pytest and all of its users. Can you please restore Python 2
support for now?
Thanks,
Scott
4 years, 4 months
PySide2 Review Request needed
by Richard Shaw
I think I have the package in pretty good shape but could use not only a
review, but also some advice on the sub-package breakdown. I've tried to
mimic the PySide1 packaging scheme but with PySide2 all 3 parts (shiboken2,
pyside2, pyside2-tools) are included in one big archive.
The history of this crazy package is here:
https://bugzilla.redhat.com/show_bug.cgi?id=1634658
And the review request here:
https://bugzilla.redhat.com/show_bug.cgi?id=1701013
I'm up for a review swap but I'm not a Python expert, I just need this for
FreeCAD and no one else has stepped up to package this monstrosity.
Thanks,
Richard
4 years, 4 months
Action needed: Recythonize the sources of your packages
by Miro Hrončok
Hello packagers. During the rebuilds for Python 3.8, we have found out that your
package most likely has precythonized C sources.
There was an update to Cython and the C sources needs to be regenerated. In line
with Fedora packaging policies and guidelines, this should be done anyway.
What needs to be done:
1. BR python3-Cython
2. figure out what C sources come from Cython sources
look for: /* Generated by Cython <version> */ on the top of them
often, all .c files are generated from Cython sources, but not always
3. in %prep, delete those files
If you build for Python 2 and Python 3, I recommend to only BR python3-Cython
and not python2-Cython, run %py3_build before %py2_build (Cython produces the
same sources on both Python versions).
Some setup.py files have custom detection whether Cython needs to be run, if the
above doesn't work alone, examine setup.py and search for "cython".
When you do this, reply to this thread, so we know we should rebuild your
package for 3.8 in https://copr.fedorainfracloud.org/coprs/g/python/python3.8/
Thank you for you help.
Maintainers by package:
afflib kwizart rebus
mpi4py deji tomspur
portmidi bsjones mjg xavierb
python-PyMuPDF swt2c
python-compreffor athoscr
python-cytoolz orion
python-http-parser bkabrda ralph
python-indexed_gzip ankursinha
python-llfuse dfateyev maci
python-pandas kushal orion sergiopr wakko666
python-pybloomfiltermmap athmane
python-pykdtree qulogic
python-sep sergiopr
python-shapely jcp volter
python-ssh2-python ignatenkobrain
python-wsaccel jujens
scipy cstratak jspaleta orion tomspur ttomecek
Packages by maintainer:
ankursinha python-indexed_gzip
athmane python-pybloomfiltermmap
athoscr python-compreffor
bkabrda python-http-parser
bsjones portmidi
cstratak scipy
deji mpi4py
dfateyev python-llfuse
ignatenkobrain python-ssh2-python
jcp python-shapely
jspaleta scipy
jujens python-wsaccel
kushal python-pandas
kwizart afflib
maci python-llfuse
mjg portmidi
orion python-cytoolz python-pandas scipy
qulogic python-pykdtree
ralph python-http-parser
rebus afflib
sergiopr python-pandas python-sep
swt2c python-PyMuPDF
tomspur mpi4py scipy
ttomecek scipy
volter python-shapely
wakko666 python-pandas
xavierb portmidi
--
Miro Hrončok
--
Phone: +420777974800
IRC: mhroncok
4 years, 4 months
Let's update tornado to 6 and drop python2-torando
by Miro Hrončok
Hi.
Tornado 6 doesn't support Python 2. Let's update the python-torando package to
Python 3 only. There are several consumers of python2-torando and if their
maintainers are interested, they can package it separately.
$ dnf repoquery --repo=compose{,-source} --whatrequires python2-tornado
bup-0:0.29.2-3.fc30.src
bup-web-0:0.29.2-3.fc30.x86_64
python-httpretty-0:0.9.5-5.fc30.src
python-pika-0:1.0.1-1.fc31.src
python-urllib3-0:1.24.2-1.fc31.src
salt-0:2019.2.0-1.fc31.noarch
uwsgi-plugin-python2-tornado-0:2.0.17.1-10.fc31.x86_64
Note that tornado is often used to test things. We can (and should) just skip
such tests from Python 2 httpretty, pika and urllib3.
Is the plan OK? I'll talk to the dependent packages maintainers, but wanted to
check with torando co-maintainers first.
--
Miro Hrončok
--
Phone: +420777974800
IRC: mhroncok
4 years, 4 months
Intent to orphan python-pytest-testmon
by Miro Hrončok
Hey! I intent to orphan python-pytest-testmon.
I no longer have use for it and it is a leaf package.
Th package is usually low maintenance, except now it doesn't build with Python
3.8 yet.
https://github.com/tarpas/pytest-testmon/issues/139
So my plan is to orphan it and eventually also retire it if nobody picks it
until the 3.8 rebuild (in ~1 month).
Let me know if you want it, I'll orphan it early next week otherwise.
--
Miro Hrončok
--
Phone: +420777974800
IRC: mhroncok
4 years, 4 months
Help wanted to convert Archmage to Python 3
by Robert-André Mauchin
Hello,
patool FTBFS depends on Archmage, which is Python 2 only.
I'm trying to convert it to Python 3.
So far I have the dependency pychm converted:
https://copr.fedorainfracloud.org/coprs/eclipseo/pythontests/build/903162/
Here's my Py3 port with 2to3 and minor fixes:
https://github.com/eclipseo/archmage/tree/python3_port
The problem is it doesn't work as expected:
$ archmage -x t.chm
Traceback (most recent call last):
File "/usr/bin/archmage", line 11, in <module>
load_entry_point('archmage==0.3.1', 'console_scripts', 'archmage')()
File "/usr/lib/python3.7/site-packages/archmod/cli.py", line 161, in main
CHMFile(options.chmfile) or CHMDir(options.chmfile)
File "/usr/lib/python3.7/site-packages/archmod/Cached.py", line 34, in __new__
__instance = object.__new__(classtype, *args, **kwargs)
TypeError: object.__new__() takes exactly one argument (the type to instantiate)
I haven't touched archmod/Cached.py code, and as far as I know the object.__new__()
method has not changed from Python 2 to Python 3 so I don't get why it's not
working anymore.
Can anyone with more Python chops help me?
For testing, use the chm from patool available here:
https://eclipseo.fedorapeople.org/archmage_test/
Thanks
4 years, 4 months