I will be out of the office starting 26/11/2007 and will not return until
If you have an urgent query please call the helpdesk on x4999 or email
This email has been scanned by the MessageLabs Email Security System.
For more information please visit http://www.messagelabs.com/email
warthog9 asked me to write him this mail regarding setup of permanent
mirror of static-repos.
Why a static-repos mirror?
It is helpful during the rawhide development cycle for hard-core rawhide
developers/testers to update their rawhide boxes directly from the
buildroot repos used by koji.
For example, during the freeze periods of F8 development I updated my
laptop from static-repos several times per day. Sometimes a new build
broke something in a very obvious fashion. Due to this advance warning,
I prevented a few utterly broken packages (like kernel) from hitting the
rawhide nightly tree.
Traditional rawhide compose + syncing to mirrors means the
build/install/test/report turnaround cycle for rawhide is 1-2 days
minimum. If some testers update from static-repos, the turnaround time
could be as short as 30 minutes. If this quick turnaround allows more
nightly rawhide trees to be installable, this tremendously aids our
overall testing ability and ultimate release quality.
Of course, it would be a huge mistake to point people directly to
koji/static-repos, because we can't afford to bog it down.
Thus a caching mirror of static-repos would allow us to more widely
advertise the option of updating rawhide directly from static-repos.
What do we need?
- 1 dedicated IP address.
- squid configured as reverse proxy server.
- ~15GB disk for squid cache.
- Metadata refresh monitoring daemon, can run as non-root anywhere on
the Internet. We could run this on a fedoraproject.org server. squid
needs to be configured to allow the IP address where the refresh daemon
is running to do the PURGE command.
- Bandwidth... although not very much, since this is a non-default repo,
few people will be using it.
What will Warren Provide?
- Sample squid configuration file.
- Time and attention to set this up.
- Continued maintenance of the refresh monitoring daemon. Since this
runs at fedoraproject.org, this requires no attention from warthog9.
How does this sound warthog9?
So what do we want to see by the Fedora 9 release? Here's a list I'd
like to see:
1) Remove all of our FC6 boxes (either by upgrade or move to RHEL)
2) Separate Test infrastructure - Right now we have people using test
boxes that connect to production databases and information. This needs
to stop. (I understand luke and dgilmore have had some discussions about
3) Finalized backup solution and koji share (we are out of room for both
backups and koji)
4) Further hardening of our systems. Implementing puppet has done a lot
of good but there's more that needs to be done. This includes making
sure all boxes come up as expected on reboot. Ensuring we have some
sort of management system in place for our xen guests that can run on
5) Further system replication - Anything related to distribution or the
primary website (including docs, and mirrors) should be able to run
while PHX is down. Its already pretty close to that.
6) New torrent server
7) New collaboration servers
8) Move hosted out of PHX and on to new server beach systems. This will
likely include creating a new "hosted" group.
9) FAS2 - This will be a big project and is one I'm hoping to accomplish
prior to F9 test1. Ricky has done some great work with it, we'll see
what it takes to finish it off.
10) Better systems integration - Many of our systems now support
different rss feeds and such. We can more easily integrate these
systems together with groups, koji, pkgdb, FAS2, bodhi, you name it.
11) Fewer new systems - This goes along with all the stuff we did to get
F7 and F8 ready. For F9 I'd like to see the team take a bit more time
taking each project to focus on that last 10%. Its always harder then
it seems and with the FAS switchover I feel its important.
:: whew ::
I'd like to spend time at next week's meeting (I'll likely not be at
this weeks meeting) talking about what is important for the individuals
in the rest of the team to get done. After that meeting we'll have a F9
milestone in place and populated with tickets.
What else do you guys want to do in the next 6 months?
Whats important to you not only as a Fedora Infrastructure member, but
as a contributor?
Here's the steps I took to reinstall the serverbeach servers:
0. make sure the hosts ip is allowed to access the repos on
infrastructure. and make sure puppet allows the host to connect to its
fileserver - in the fileserver.conf file in cvs
1. login to existing system as root
http://infrastructure.fedoraproject.org/rhel/RHEL5-x86_64/images/pxeboot/... \ /boot/vmlinuz-install
http://infrastructure.fedoraproject.org/rhel/RHEL5-x86_64/images/pxeboot/... \ /boot/initrd-install.img
3. run this command +/- ip address changes and path changes for the
grubby --add-kernel=/boot/vmlinuz-install \
--args="ks=http://skvidal.fedorapeople.org/hidden/sb1.ks lang= \
devfs=nomount ksdevice=link selinux=0 ip=220.127.116.11 \
gateway=18.104.22.168 netmask=255.255.255.192 \
dns=22.214.171.124,126.96.36.199" --title="install el5" \
4. run 'grub' and type:
savedefault --default=0 --once
5. reboot the system. Watch it via ping, when it starts pinging again
after the install run:
type in the vnc password - in this case 'vncinstall'
Yes i m interesting in that. But one thing i want to say about me that i m a fresher. What ever i can i will.
----- Original Message ----
From: Karsten Wade <kwade(a)redhat.com>
Sent: Tuesday, November 20, 2007 1:55:09 PM
Subject: Re: hosting git conversion of Fedora CVS tree on fedora infrastructure?
On Mon, 2007-11-19 at 12:01 +0100, Lennert Buytenhek wrote:
> The git tree is currently a read-only (slave) version of the CVS
> tree, and I expect it to stay that way for some time. But even
> Fedora isn't switching VCSes at this point, I think it would still
> make sense to have git/hg/random-other-VCS conversions of the Fedora
> CVS tree publically available ...
+1 to the general idea.
Are you interested in building and maintaining the solution in Fedora
Infrastructure? Just asking as a bystander ... :)
Karsten Wade, Developer Community Mgr.
Dev Fu : http://developer.redhatmagazine.com
Fedora : http://quaid.fedorapeople.org
gpg key : AD0E0C41
Never miss a thing. Make Yahoo your home page.
For those on the list that don't actively participate in discussion,
Fedora Infrastructure uses time-lines roughly related to the releases
that come out. This works out well for us as major changes to our
infrastructure typically are aligned well with a release. So in saying
that, here's a quick look back on stuff we did for fedora 8. This list
is FAR from exhaustive, but gives a good view:
Fedora 8 Website: A lot of stuff changed on the Fedora 8 website this
time around. Not as drastic of a change as last time in terms of looks,
but a lot of changes in how we build it. It has already been translated
into many languages. Its been replicated to multiple locations (with
one or two more to come). This served us well when PHX fell off the
planet during our release.
DB1 upgrade: DB1 was our only DB server for a long time. We upgraded
the OS and versions of Postgres and Mysql. Then split it up into two
boxes. One on DB1 (MySQL) and one on DB2 (Postgres). The two send
snapshots of eachother around. Its also worth noting that DB2 is now on
better hardware and is virtualized and run over iscsi.
Two clicks to live CD: A simple and obvious thing to do but took
coordination and work to actually implement. It has made our systems
more user friendly and allowed us to completely re-do what
download.fedoraproject.org is. Major props to Matt Domsch on that for
creating the ability in Mirror Manager.
Change Freeze: We officially had a change freeze this time around.
Aside from some failed hardware. It went well. We're gaining more
discipline in our infrastructure while still remaining adaptable to
change. A tricky task. But I hope to see more of it during Fedora 9.
Hosted: We implemented a hosted solution (still not quite "official"
but being worked on). Think about hosted for a moment. We haven't
announced it and basically word of mouth has made it what it is
already. We support multiple source control mechanisms and its
possibilities are quite limitless.
Ticketing system: Otrs is gone, Trac replaced it. Hurray! We
actually use this system. Its easy to use and its how I'm getting this
list of things we completed :) (note to someone that happens upon this.
OTRS is a great tool, just the wrong tool for our needs)
New backup system: We're using bacula now instead of backuppc. Its
still got some bugs and is in desperate need of a proper storage
backend. But it is something we setup and got working.
Package Database: Toshio wrote a package database where there just was
none before. It's very useful and will continue to become more useful
as we integrate our systems.
fedorapeople.org: Amazing how popular this turned out to be. People
host code, scm's, all sorts of stuff. Its been quite useful.
Ibiblio mirror: A small thing but has been useful. We have an ibiblio
mirror. It's proved we can run one outside of Red Hat's control and
it's even allowed us to fix up some mistakes we made quicker then
syncing from Phoenix.
Proxy Caching: Probably one of THE most useful (for Fedora
Infrastructure) parts of our release this time around has been proxy
caching. It has had a major impact on how often our application servers
actually serve pages. We've even made things easier on the wiki.
Open Infrastructure: We did do some major reorganization during the F8
release. We have more groups then ever, more sponsors then ever and
more contributors then ever in Infrastructure. We continue to need to
expand a bit and harden our actual procedures but the team is strong and
Voip: Our asterisk server was up and running in time for FudCon. I
should remind everyone that this went from "nothingness" to an open
working solution in a matter of days and ready for our virtual fudcon.
Major props to dgilmore, jcollie and skvidal. In Fedora 9 we'll build
some more dedicated boxes to this. Hopefully it can become a major
helpful tool for training, meetings, etc.
VPN: In response to our decentralization we've decided to implement a
vpn solution. Much of it is in place with more to come as we create
Respins: spins.fedoraproject.org is up and running. It's still very
basic but as we get more time, we'll be implementing more and more as
our new torrent box comes online.
start.fedoraproject.org: Well, our end is up anyway :)
translate.fedoraproject.org: A brand new for us website that, when
combined with transifex, will allow us to bring our software to more
people then ever.
maps: how cool are maps? http://fedoraproject.org/maps/ We got'em and
Upgrades and other stuff: Even after adding all of this stuff, we've
made a lot of our current systems better through virtualization and
Ok, I'm done. If there's something I've forgotten that you'd like to
add. It's a list, reply! I'm incredibly happy with all the work we've
done in the last 6 months.
Stay tuned for the "Fedora 9 - A release in preview". Start thinking of
things that are important to you and what you'd like to see done for
Fedora 9. We'll discuss it in that thread, then at the next meeting.