On Tue, May 11, 2010 at 02:21:38PM -0500, Dennis Gilmore wrote:
On Tuesday 11 May 2010 01:19:09 pm Mike McGrath wrote:
> On Tue, 11 May 2010, Toshio Kuratomi wrote:
> > mmcgrath noted that we have a race condition in our syncStatic.sh script
> > that is leading to proxies not having copies of the website. Here's a
> > patch to the syncStatic script to fix that:
> >
[ old patch snipped]
> > The patch does two things:
> >
> > 1) Removes the rm -rf step as that is causing the race (when proxies sync
> > data after the rm -rf and before hte rsync, they get an empty directory)
> >
> > 2) Removes the wildcard from the rsync. The wildcard was preventing the
> > --delete from functioning as expected. I believe (but ricky can confirm)
> > that the --delete not removing files that are no longer needed from the
> > final output directory is why we had the rm -rf in the first place.
> >
> > Could I get two +1's for this change?
> >
> > -Toshio
>
> +1 from me, this was causing problems.
+1
With feedback from mdomsch and skvidal, here's a new version of the patch
that does --delay-update and --delete-after. This will trade space (which
we currently seem to have enough of on bapp01) for less chance of races
while the changes are being put into place.
Two +1's to this version?
diff --git a/modules/fedora-web/files/syncStatic.sh
b/modules/fedora-web/files/syncStatic.sh
index 23f5181..8affac3 100644
--- a/modules/fedora-web/files/syncStatic.sh
+++ b/modules/fedora-web/files/syncStatic.sh
@@ -26,7 +26,7 @@ cd fedora-web
pushd
fedoraproject.org > /dev/null
make > /dev/null 2>&1
-rsync -qa --delete out/* /srv/web/fedoraproject.org/
+rsync -qa --delete-after --delay-update out/ /srv/web/fedoraproject.org/
popd > /dev/null
# Make sure everything else builds from master.
@@ -34,26 +34,22 @@ popd > /dev/null
/usr/bin/git checkout -q master || exit 1
pushd
spins.fedoraproject.org > /dev/null
-/bin/rm -rf /srv/web/spins.fedoraproject.org/*
make > /dev/null 2>&1
-rsync -qa --delete out/* /srv/web/spins.fedoraproject.org/
+rsync -qa --delete-after --delay-update out/ /srv/web/spins.fedoraproject.org/
popd > /dev/null
pushd
talk.fedoraproject.org > /dev/null
-/bin/rm -rf /srv/web/talk.fedoraproject.org/*
make > /dev/null 2>&1
-rsync -qa --delete out/* /srv/web/talk.fedoraproject.org/
+rsync -qa --delete-after --delay-update out/ /srv/web/talk.fedoraproject.org/
popd > /dev/null
pushd
boot.fedoraproject.org > /dev/null
-/bin/rm -rf /srv/web/boot.fedoraproject.org/*
make > /dev/null 2>&1
-rsync -qa --delete out/* /srv/web/boot.fedoraproject.org/
+rsync -qa --delete-after --delay-update out/ /srv/web/boot.fedoraproject.org/
popd > /dev/null
pushd
start.fedoraproject.org > /dev/null
-/bin/rm -rf /srv/web/start.fedoraproject.org/*
make > /dev/null 2>&1
-rsync -qa --delete out/* /srv/web/start.fedoraproject.org/
+rsync -qa --delete-after --delay-update out/ /srv/web/start.fedoraproject.org/
popd > /dev/null
-Toshio