On 01/23/2014 05:02 AM, Siddhesh Poyarekar wrote:
Hi Carlos,
I wrote up a script that automates some bits of the process of
importing tarballs from upstream master into rawhide. I've attached
the script for your review. Do you think this would be useful to have
in the fedora rawhide branch? With its presence, the workflow for
resyncing rawhide would simply be:
Great job!
We should absolutely put this into the Fedora rawhide branch.
We want all of our tools easily accessible on checkout, either
from that branch or some Fedora glibc-releng repository.
Honestly I think you should check this into master as an example
distribution script for RPM systems. That way others can look at
what you've done without needing to go into a distribution specific
branch.
It's glibc's responsibility to help downstream distributions use
the project effectively.
0. Check out the master branch on the local glibc repo
1. Check out the master branch on the fedora glibc repo
2. In fedora glibc, run:
./sync-upstream.sh <path-to-local-glibc-repo>
3.1. If the script succeeds:
fedpkg push
fedpkg build
3.2. If the script fails, analyze the reason for failure and fall back
to manual process. The most likely failure is that a patch
refuses to apply and one may have to adjust the patch before
committing the changes.
Let me know what you think about this.
Sounds good to me.
It's a step in the right direction.
Siddhesh
#!/bin/sh
# Given a git source repo, generate a tarball from master, modify the spec file
# and upload it to lookaside cache if the tarball unpacks and gets patched
# cleanly. This currently needs the source repo to be on the master branch.
# XXX In future, we could just do the git-archive ourselves instead of relying
s/XXX/TODO:/g
Prefer TODO over XXX only because it's clear this is an RFE.
I generally reserve XXX and FIXME for something that are wrong.
# on the makefile bits. That way we don't have to waste time
waiting for the
# bzip and xz archives that we don't use.
if [ $# -ne 1 ]; then
echo "Usage: $0 <path-to-upstream-repo>" 1>&2
exit 1
fi
upstream=$1
tmpdir=/tmp/tmp-rawhide-gzbuild
Can't run two on the same machine.
Use mktemp please.
srcdir=$(git --git-dir=$upstream/.git describe)
cursrc=$(sed -ne 's/^%define glibcsrcdir \(.*\)/\1/p' glibc.spec)
These are automatically escaped by the sub-shell invocation.
Assignment also preserves escaping e.g. cursrc2=$cursrc is OK.
if [ "$cursrc" = "$srcdir" ]; then
echo Already in sync with upstream
exit 0
fi
echo "+ Generating tarballs..."
mkdir $tmpdir
pushd $tmpdir
These are not and and need escaping now.
# We don't need any options here since we only want the makefile
to be
# generated so that we can call make dist
$upstream/configure --prefix=/usr
Needs escaping e.g. "$upstream/configure"
# HACK! We don't want to regenerate the INSTALL file, so
regenerate the file
# explicitly, undo its changes and then make dist. That way make thinks that
# INSTALL is up-to-date.
make INSTALL
pushd $upstream
git diff INSTALL | patch -p1 -R
popd
make dist
popd
rm -rf $tmpdir
Needs escaping e.g. "$tmpdir".
if ! [ -f $upstream/$srcdir.tar.gz ]; then
Needs escaping.
echo "Failed to create the tarball"
exit 2
fi
# Copy over the tarball.
echo "+ Copying over generated tarball..."
cp $upstream/$srcdir.tar.gz ./
Needs escaping.
# Our changelog header
cldate=$(date +'%a %b %d %Y')
clname=$(git config --get user.name)
clemail=$(git config --get user.email | sed 's/@/\\@/')
OK.
# Getting our version and release number from the spec file.
nv=$(perl -ne 's/^%define glibcversion (.+)/printf("%s-", $1)/e;' \
-e 's/^%define glibcrelease ([0-9]+).*/printf("%d\n", $1 + 1)/e;'
\
glibc.spec)
# Our changelog entry.
changelog="* $cldate $clname <$clemail> - $nv\n- Sync with upstream
master.\n"
# Change the glibcsrcdir variable, bump up the release number and add an extra
# entry to the changelog.
echo "+ Updating spec file..."
perl -pi \
-e "s/^(%define glibcsrcdir ).+/\$1$srcdir/;
s/^(%define glibcrelease )(\d+)/print(\$1); print(\$2 + 1);'';/e;
s/^(%changelog)$/\$1\n$changelog/" \
glibc.spec
echo "Testing if fedpkg prep works..."
if fedpkg prep; then
Run `fedpkg prep' as a distinct step, save the output, and
then check the return code.
echo "Source prep is clean, so we're good to go."
fedpkg new-sources $srcdir.tar.gz
Needs escaping.
git commit -a -m \"Sync with upstream master\"
# XXX I don't have the courage yet to do a fedpkg push && fedpkg build
# from the script.
Do it.
But do three things.
(1) Use `set -e' immediately after #!/bin/sh to ensure that any non-zero
exit from any command, builtin, or sub-shell exits the shell.
(2) Make sure you have no compound steps that could hide an exit code.
e.g.
[carlos@koi ~]$ cat test.sh
#!/bin/sh
# Oh, oh, typo! It should be carlos not carlso
dir=$(ls /home/carlso | sed -e "s,a,b,g")
echo $?
# We didn't notice though...
[carlos@koi ~]$ ./test.sh
ls: cannot access /home/carlso: No such file or directory
0
[carlos@koi ~]$
(3) Parse the fedpkg prep log output and verify that the expected
tarball name was used.
else
echo "Source prep failed."
echo "Check the output above and fix things before committing."
fi
echo "+ Done!"
Tada!
Cheers,
Carlos.