On Sun, 25 Oct 2009, Ben Boeckel wrote:
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256
Mike McGrath wrote:
> On Sun, 25 Oct 2009, Ben Boeckel wrote:
>
>> -----BEGIN PGP SIGNED MESSAGE-----
>> Hash: SHA256
>>
>> Hi,
>>
>> I am working on a project designed to mirror large, changing archives of
>> software in a manner that ensures data integrity and atomic updates using a
>> peer-to-peer protocol.
>>
>> My team and I would like to design this software such that it provides a
>> suitable replacement for rsync. As Fedora is among the leading
>> distributions, I would like to solicit your opinions on the implementation.
>> We have designed much of the architecture, but have not implemented anything
>> yet. One issue we wish to address currently is that of interpreted vs.
>> compiled languages.
>>
>> My team and I would like to know whether the community would be accepting of
>> such a project (which includes a daemon) if it were written in Python rather
>> than C or C++. If so, it would greatly simplify the implementation and
>> allow it to be more robust. Python's built-in libraries and facilities
>> provide much of the path and network manipulation that the daemon requires.
>> Using the Python standard libraries allows us to rely upon a well-tested
>> base and focus on higher-level issues.
>>
>> What are your opinions, as system administrators, on using Python for
>> long-running daemon processes when the developers are explicitly mindful of
>> memory considerations?
>>
>
> I've generally had better luck with C/C++ based daemons from a systems
> admin point of view. But we've used plenty of python based daemons that
> worked just fine. I think the problem is the lower barrier to using
> python means it's easier for less experienced developers to create python
> daemons.
>
> Really though, if it's done right, python daemons can be just as good as C
> daemons. And extending Python is very easy as you mentioned with using
> pythons built in libraries. If you're concerned about performance though
> it shouldn't take much to do the basics of what you want and compare C to
> python. Sometimes they're identical, sometimes C wins but I don't think
> I've seen python win yet (with performance). But it is much easier to
> work with python :)
>
> -Mike
Thanks for your reply.
Seeing as it is a mirroring daemon, the network is the bottleneck. If it isn't
then either you're sitting next door, our implementation is bad, or the
hardware shouldn't be a mirror in the first place.
With all my babbling I forgot to mention we do already run python daemons
in Fedora Infrastructure. Func is one, TurboGears (though it's wrapped in
mod_wsgi) and one that we wrote ourselves[1] is our mirrorlist server.
It's the backend that powers:
http://mirrors.fedoraproject.org/mirrorlist?repo=fedora-11&arch=i386
-Mike
[1] By 'ourselves' I mean Matt Domsch who, quite honestly, has designed
the most robust, fault tolerant dynamic content system we run.