On Sun, Feb 17, 2019 at 04:07:46PM +0100, Adrian Reber wrote:
The latest release of MirrorManager2 is still in updates-testing. I
not want to push this version to updates-released because this is the
first Python3 based released.
Could someone test the version from updates-testing? Update the
mirrorlist container and test it on one of the proxies?
I tested it locally and it does not work. For someone knowing Python
better than me it is probably obvious, but the mirrorlist server using
Python 3 cannot read the pickle from the MirrorManager backend running
To be able to use a Python 2 based backend and Python 3 based mirrorlist
I added an additional export format using Protobuf.
The good news about this is that the file containing all the information
for the mirrorlist server using protbuf is smaller than the pickle file.
The protbuf is about 2/3 of the pickle file.
The bad thing about this change is that the memory required by the
mirrorlist server is 5 times higher using the protbuf file than using
the pickle file. It is amazing how reading a 32MB file results in a
process requiring 2.5GB of memory.
From the memory consumption point of view this change seems like a bad
idea. Being independent of the pickle format seems to be a good idea as
it would be possible to have Python2 and Python3 parts in our
MirrorManager deployment. It would also open up the possibility to
rewrite the mirrorlist server in something else which does not require
to understand Python's pickle format.
This is the pull request with all the changes:
Any recommendations if this is useful or how to convince Python to use