reproducible builds and python
by bmorbach
Hi everyone!
I've been doing some work towards reproducible builds in Fedora (mostly
with various upstreams so far) and one of the elephants in the Room are
obviously Pythons .pyc and .pyo files.
As those contain the mtime of the original .py file, they might be
different for each rebuild of an srpm.
For many rpms this isn't a problem, because the files are not modified
and thus retain their timestamp from the archive. Quite a few rpms do
modify to .py files though and because of that, every build has a
different result.
I would like to propose to set the mtime of all .py files to a fixed
(for this specific srpm) time. This could be done
in /usr/lib/rpm/brp-python-bytecompile before doing the actual
byte-compilation. This would result in the same .py{c,o} files being
created for each rebuild.
The timestamp could be e.g. the mtime of the oldest file in the
buildroot (which would assume that not _all_ of the files are modified)
But if you are interested in the idea, I'd certainly be open to
suggestions.
To address the obvious question:
Why not special-case those files when comparing rpms?
It will certainly be impossible to achieve this for all packages in
Fedora, so for some files this might indeed be needed, but I think we
should avoid this where possible. The idea of reproducible builds
becomes meaningless if the amount of differences that you just ignore
gets to big.
What do you think of this proposal?
Greetings,
Benedikt
9 years
mock, "megadeps" and PyPI
by Nick Coghlan
Jeff Fearn reminded me today of "mock --megadeps", a patched version of
mock he created that supports recursively downloading and building
dependencies in a chroot, without incurring the overhead of setting up
and tearing down multiple mock build environments the way the chainbuild
command does.
The mock RFE is at https://bugzilla.redhat.com/show_bug.cgi?id=843745,
while Jeff is now maintaining the patched version at
https://github.com/jfearn/mock
I'm bringing this to folks attention mainly due to one of the neat
features it has embedded in it, which is integration with CPAN: it can
use cpanspec to create an SRPM on the fly. That's likely not going to be
of suitable quality for Fedora itself, but it should be good enough for
COPR and potentially even Playground.
My question is whether it would be practical to do something similar
with pyp2rpm. Bonus points if we can come up with a way to integrate it
nicely with COPR, or even tidy up the implementation to the point where
we can convince Clark to accept the feature as part of mock itself :)
Regards,
Nick.
--
Nick Coghlan
Red Hat Hosted & Shared Services
Software Engineering & Development, Brisbane
HSS Provisioning Architect
9 years, 1 month
Python 2.7 SSL upgrade patch available for testing
by Nick Coghlan
PEP 466 approved bring the core Python 2 network security infrastructure
up to speed with the modern internet.
Alex Gaynor has provided a draft patch of the most complex part of that
PEP, backporting the bulk of the Python 3.4 SSL module to Python 2.7:
http://bugs.python.org/issue21308#msg223895
This is also the part of the PEP most likely to break things, so
figuring out a way to test it in Fedora before it makes it into an
upstream CPython release would be a good idea...
Cheers,
Nick.
--
Nick Coghlan
Red Hat Hosted & Shared Services
Software Engineering & Development, Brisbane
HSS Provisioning Architect
9 years, 1 month
Introducing Python 3.5 nightly builds for Fedora
by Miro Hrončok
Hi everybody, I'd like to introduce a new project of Slavek Kabrda and mine:
Copr repository with nightly builds of development version of Python 3.
http://copr.fedoraproject.org/coprs/churchyard/python3-nightly/
How does it work?
=================
Each night, at 00:01 CET/CEST, dgroc [1] is run and checks out if any of
the following upstream projects has some new commits in master/default
branch:
* cpython
* setuptools
* pip
* wheel
If so, it creates SRPM with code from that specific new commit and
builds it in the copr repository. Is is being built as a software
collection, not to break system python3 package (that being in critpath
once Python 3 will be the default).
Benefits
========
This will let us know immediately when our patches of Python 3 package
are broken. We can react when that happens a we know exactly what commit
(or at lest what day) broke it. This will avoid a big "fix all the
patches" spree once Python 3.5 is stable and hits Fedora. The builds
also run the test suite, so we can see regressions when they happen and
we can communicate with Python upstream fast and flexible (again,
instead of a mass failure once Python 3.5 hits Fedora). It also let us
and you to test if your packages build and run with Python 3.5, so we
can avoid more confusion once Python 3.5... you see what I mean.
Other than the packaging reasons: Fedora might attract Python
enthusiasts and developers as it is most likely the only Linux distro
that has development version of Python packaged nightly. Developers and
powerusers can play with new features committed yesterday just by doing
`dnf update`.
How to use it
=============
Ad the repository (with dnf copr plugin, or manually), and install
python35 package. That installs the software collection and you can use
it as any other software collection:
$ scl enable python35 python
Or
$ scl enable python35 bash
You can also install packages from PyPI:
# scl enable python35 bash
# pip install ipython
How to (test)build my package against Python 3.5
================================================
If you want to build your package against Python 3.5, you can do it in
the following way:
1. Update the spec by adding SCL related macros, see [2][3]
2. In mock or Copr, add the repository and add the following packages
to the chroot: scl-utils-build python35-build [4][5]
3. Build and profit
[1] https://github.com/pypingou/dgroc
[2]
http://docs.fedoraproject.org/en-US/Fedora_Contributor_Documentation/1/ht...
[3] https://bitbucket.org/bkabrda/spec2scl/
[4]
https://lists.fedorahosted.org/pipermail/softwarecollections/2012-Novembe...
[5] http://mmaslano.livejournal.com/9668.html
--
Miro Hrončok
--
Phone: +420777974800
IRC: mhroncok
9 years, 1 month