On Thu, 2013-08-01 at 09:41 +0200, Vratislav Podzimek wrote:
On Wed, 2013-07-31 at 21:46 +0200, Martin Kolman wrote:
> As the Koji RPM search takes quite some time, it is now threaded,
> one thread per package. The thread handles both the lookup and package
> download.
> This should provide the biggest possible speedup and basically
> cuts the whole operation down to the duration of the longest of the
> Koji lookups.
> As urlgrabber turned out to be thread-unsafe, it has been replaced by
> a urllib.urlretrieve() call and the live download progress has been
> replaced by simple per-file/thread "download done" notification
>
> Signed-off-by: Martin Kolman <mkolman(a)redhat.com>
> ---
> scripts/makeupdates | 165 ++++++++++++++++++++++++++++++++++------------------
> 1 file changed, 108 insertions(+), 57 deletions(-)
>
This looks and works good to me. I just believe you can get rid of the
print_lock, the GIL and probably some buffering below should do it. Or
have you seen anything weird without the lock?
Without the lock, the was usually at least one collision per run,
something like:
thread 1: foo
thread 3:barthread 2: baz
thread 4: spam
So the actual output strings didn't get mixed together, but the newlines
did, resulting in unreadable output.
With the lock, everything seems fine.