Replying to myself, to report the current status.
On Fri, Apr 09, 2010 at 12:18:48PM +0200, Robert Vogelgesang wrote:
On Thu, Apr 08, 2010 at 05:03:50PM -0700, Ian Ward Comfort wrote:
[snip]
> Hm, that's unusual. We run with "-c cache -s sha
--update -C" here,
> on RHEL5.5, and mirroring works properly -- so far as we can tell --
> with this and my other patches. Maybe you're seeing the problem fixed
> by my createrepo-always branch?
Your createrepo-always branch fixes issues where createrepo does not
get started at all. The main issue in cobbler trac issue #579 is
about wrong or unexpected results of createrepo's timestamp check.
I think the fix in Ian's createrepo-always branch should be pulled
into master, even though it is not relevant to trac issue #579.
There are now two other cobbler trac issues, #564 and #583, that
would be fixed by this patch.
>
> [1] $ touch file &&
> stat file | grep Change &&
> sleep 1 &&
> wget -q
http://www.google.com/ -O file &&
> stat file | grep Change
> Change: 2010-04-08 16:57:07.000000000 -0700
> Change: 2010-04-08 16:57:08.000000000 -0700
Well, this might be part of the problem. During "reposync", cobbler
runs wget to fetch repomd.xml, and a little later createrepo.
As far as I know (now ;-)), tools such as wget, rsync, and others,
cannot "preserve" the ctime; I'm not aware of any system call that
would allow to explicitely set this timestamp. The utime(2) call
only allows to set mtime and atime, but not ctime.
On the other hand, this means that - as far as tools used by cobbler
to download files are concerned - the ctime always refers to some
kind of local action, whereas the mtime represents some change on
the remote, "original" site. If cobbler would not clobber the
ctime by the "chown -R" and "chmod -R" actions, the ctime could be
interpreted as the time of the last download, which in fact would
make perfect sense as a criterion to decide if createrepo should
update the metadata files.
If your're right, then the mirrored repomd.xml would always have a
newer ctime than all the other files in the repo, at the time
"createrepo --update -C" gets run. Just after createrepo,
"cobbler reposync" runs "chown -R ...", which would update all ctime
timestamps, which is why a subsequent interactive run of the same
createrepo command would find different timestamps and would start
updating the meta data. But each run of "cobbler reposync" would
update the ctime of repomd.xml just before createrepo is started.
Maybe your fetch-repomd-safely branch could actually help in this
situation.
I've now hacked /usr/bin/createrepo to log the mtime and ctime
timestamps of all files in the repo, and after the next scheduled run
of "cobbler reposync" (on monday morning) I'll look at the results
and decide about the next steps to resolve this issue.
The logs created by my hacked version of /usr/bin/createrepo confirmed
all this, and also confirmed that the timestamp check issue reported
in #579 is fixed by Ian's fetch-repomd-safely branch.
So please pull that into cobbker master.
Robert
So far, thank you for your comments.
Robert
>
> --
> Ian Ward Comfort <icomfort(a)stanford.edu>
> Systems Team Lead, Academic Computing Services, Stanford University
>
> _______________________________________________
> cobbler-devel mailing list
> cobbler-devel(a)lists.fedorahosted.org
>
https://fedorahosted.org/mailman/listinfo/cobbler-devel
_______________________________________________
cobbler-devel mailing list
cobbler-devel(a)lists.fedorahosted.org
https://fedorahosted.org/mailman/listinfo/cobbler-devel