Meeting Log - 2007-11-15
by Ricky Zhou
14:59 < skvidal> who wants to have a meeting?
14:59 -!- jcollie [n=jcollie(a)161.210.6.122] has joined #fedora-meeting
14:59 -!- kwizart [n=kwizart@fedora/kwizart] has joined #fedora-meeting
14:59 < GeroldKa> what kind of meeting skvidal ?
14:59 * ricky
15:00 < skvidal> fedora-infrastructure meeting :)
15:00 < GeroldKa> ah OK
15:00 < GeroldKa> that's not my $favorite meeting today :-/
15:01 < abadger1999> Yes, let's.
15:01 < nnm> yep
15:01 -!- MrBawb [i=abob(a)guppy.drown.org] has joined #fedora-meeting
15:02 * ricky copies from an old log: < mmcgrath> dgilmore jeremy abadger1999 jima paulobanon mbonnet f13 mdomsch warren ricky skvidal ivazquez people I forgot: Ping!
15:02 * mdomsch needs to fix the calendar again after DST change
15:02 < skvidal> ricky: useful :)
15:02 * skvidal is here
15:02 < skvidal> oh crap, I'm supposedly in charge for this week
15:02 * lmacken
15:02 < skvidal> okie doke
15:02 < abadger1999> ha ha
15:02 < jcollie> yo
15:02 < abadger1999> sucker :-)
15:02 -!- G_ [n=njones@wikipedia/NigelJ] has joined #fedora-meeting
15:02 < warren> Even though I fixed the bug, I still fear crashes at this time.
15:02 < skvidal> let's see what the certs say
15:02 < skvidal> s/certs/tickets/
15:02 -!- clarkbw [i=clarkbw@nat/redhat/x-3b1efb9013e9f272] has quit "shocks the conscience"
15:03 < ricky> https://hosted.fedoraproject.org/projects/fedora-infrastructure/query?sta...
15:03 < ricky> Oops.
15:03 < ricky> I missed: https://hosted.fedoraproject.org/projects/fedora-infrastructure/query?sta...
15:04 < skvidal> so
15:04 < skvidal> guess what
15:04 < skvidal> we're still low on disk space
15:04 -!- hircus [n=michel(a)dhcp-cs-244-140.cs.indiana.edu] has quit "Ex-Chat"
15:04 < jeremy> shocker
15:04 < skvidal> on the plus side we do have the serverbeach servers and the rest of today, for me, is to get at least one of them online enough for us to use
15:04 < jeremy> go seth go!
15:04 < skvidal> now, once it's all xen and happy I need some suggestions on what we need to get over there first
15:05 < f13> I'm in another meeting :(
15:05 < ricky> Woohoo!
15:05 < mdomsch> skvidal, mirrorlist requests
15:05 < skvidal> we need to have a new app server
15:05 < ricky> So we said mirrormanager (or list at least)
15:05 < mdomsch> e.g. mirrors.fp.o
15:05 < skvidal> mdomsch: can our app servers talk to any db they need or to fas w/o having to be in phx?
15:06 < mdomsch> skvidal, if they can reach db1 over the vpn, then yes
15:06 < mdomsch> if they can't, no dice
15:06 < skvidal> mdomsch: yes, well the vpn is the other issue
15:06 < abadger1999> s/db1/db2/
15:06 < mdomsch> ok
15:06 < ricky> Can't seem to ping db1/2 from proxy3, at least.
15:06 < mdomsch> w/o the vpn, all we can do is scp the mirrorlist data out to the app servers not in phx
15:07 < skvidal> okay then it seems like I have to either find that password/key for the CA for the vpn
15:07 < mdomsch> fwiw, I'm ok with copying the data out from app3 -> app4 and new serverbeach appX
15:07 < skvidal> but for the long run this needs to be documented
15:08 -!- halfline_ [i=rstrode@nat/redhat/x-d3d704dad926c3de] has joined #fedora-meeting
15:08 < skvidal> setting up a vpn client isn't hard - but the CA signing is difficult :)
15:08 < skvidal> okay, next thing
15:08 < skvidal> ricky: what's the good word? did glezos have to jet?
15:08 * nirik recomends the use of the openvpn 'easy-rsa' scripts. Makes it very easy to handle openvpn certs.
15:09 < skvidal> nirik: yes, that's true
15:09 < skvidal> one problem
15:09 < ricky> skvidal: For transifex? Yeah, I'm waiting a bit on glezos, but I may have to do some digging about that error myself.
15:09 < skvidal> they don't work without the password on the CA
15:09 < nirik> sure. Thats gonna be needed no matter what. ;)
15:09 < skvidal> ricky: okay, thanks - it'd be good to get that back up sooner than later
15:09 -!- mclasen [n=mclasen(a)c-24-218-44-243.hsd1.ma.comcast.net] has quit "There must be some way out of here."
15:09 < ricky> Definitely.
15:09 < skvidal> ricky: is there anything I can do to help?
15:10 < lmacken> ricky: what error ?
15:10 -!- nim-nim [n=nim-nim@fedora/nim-nim] has joined #fedora-meeting
15:10 < skvidal> mdomsch, abadger1999, ricky, lmacken: what else can we do with what's in serverbeach?
15:11 < ricky> When a translation is submitted, it gives a UnicodeDecodeError.
15:11 -!- JSchmitt [n=s4504kr(a)p54B12824.dip0.t-ipconnect.de] has joined #fedora-meeting
15:11 < mdomsch> skvidal, another proxy
15:11 < skvidal> I know mike had some specific plans, and that's fine - I just wanted to see if there was anything else pressing
15:11 < lmacken> ricky: fun! I love those
15:11 < mdomsch> skvidal, maybe smolt backend
15:11 < mdomsch> since those boxes are beefy
15:11 < ricky> Heh- funny thing is- it doesn't happen on publictest5 - just app3.
15:11 < skvidal> mdomsch: yah - I'm going to see if I can find my spreadsheet
15:11 -!- wolfy [n=lonewolf@fedora/wolfy] has left #fedora-meeting ["Okay, who stopped the payment on my reality check?"]
15:11 < skvidal> mdomsch: indeed and smolt should be outside of phx,imo, too
15:12 < ricky> Maybe it's an effect of the sqlite => mysql migration.
15:12 < skvidal> okay, we can allocate them in xen once this is happy
15:12 < skvidal> okay next item
15:12 < skvidal> bacula
15:12 < skvidal> we're hitting the wall on bacula
15:12 < skvidal> and I want to know if there is anyone who is more versed in bacula than me
15:13 < skvidal> b/c I have some questions if there are
15:14 -!- G [n=njones@wikipedia/NigelJ] has quit Connection timed out
15:14 < ricky> Guess not at the moment :)
15:14 -!- bpepple [n=bpepple(a)adsl-75-60-205-159.dsl.wotnoh.sbcglobal.net] has quit "Ex-Chat"
15:15 -!- bpepple [n=bpepple(a)adsl-75-60-205-159.dsl.wotnoh.sbcglobal.net] has joined #fedora-meeting
15:15 < skvidal> okay
15:15 < skvidal> anything else? b/c if not we can close this early and I'll see if I can be useful
15:15 < ricky> Perhaps #bacula would be able to help a bit, if we can't find somebody.
15:15 < skvidal> ricky: yah - I know - I'll sort it out - I just wish there was more room for errors :)
15:17 -!- smooge [n=smooge(a)int.smoogespace.com] has quit "-ENOCAFFEINE"
15:17 < skvidal> okay seems like it is quiet, too quiet
15:17 < abadger1999> smolt would benefit from having a local database but I don't think we want to set that up without some thought.
15:18 < mdomsch> be useful
15:18 < skvidal> anythin else?
15:18 < warren> how is the storage going
15:18 < warren> for koji
15:18 < warren> after the purging
15:18 < skvidal> warren: no word since mike is away
15:18 < mdomsch> jima, are your map stats now unique IPs ?
15:18 < warren> ok
15:18 < skvidal> I've not heard anything new from max
15:18 < f13> purging is going well
15:18 < f13> we have another round of purges that will hit in a week or so
15:19 < f13> and that will give us a good indication if we look at the initial purge and the second p urge at what our expected growth will be over time.
15:19 < skvidal> purging is good
15:19 < jima> mdomsch: mirrorlist? yes.
15:19 < ricky> Hm, I don't know anything about this, but would db balancing help smolt?
15:19 < f13> we're obviously growing, but not as much as before.
15:19 < ricky> (Or is that not where the bottleneck is?)
15:20 < abadger1999> ricky: It might. But that's also something that I'd want to think about before doing... replication makes things more complex.
15:20 < f13> I also have word from my manager that once we have that idea of growth, and project out a year, we have budget to get needed storage, and needed backup system.
15:20 < ricky> Aha.
15:20 < f13> so when Mike gets back, I'll work with him on that.
15:21 < abadger1999> And I don't even know if we've added more indexes to the db tables yet. (which would be a much simpler fix to the db bottleneck.)
15:22 -!- JSchmitt [n=s4504kr@fedora/JSchmitt] has quit "Konversation terminated!"
15:23 < ricky> Wasn't there a db person in #fedora-admin working with smolt recently?
15:24 * dgilmore is here now
15:24 < abadger1999> There were a few. loupgaroublond -- do you know if the those db changes were implemented for smolt?
15:26 < ricky> And anybody have an update on anything at http://fedoraproject.org/wiki/Infrastructure/Schedule?
15:31 -!- mether [i=ask@fedora/mether] has quit Read error: 145 (Connection timed out)
15:33 < skvidal> I think we're closed out at this point, actually
15:34 < ricky> OK then :) Sending the log.
16 years, 5 months
Join Infrastructure Team?
by Timothy Roberts
Hello, I am interested in joining the Fedora Infrastructure team.
Experience:
Administration and installation of multiple home desktop and server
Gentoo Linux boxes.
Configuration of PaX and SELinux on Gentoo Hardened Servers.
Sorry that it's all Gentoo box experience, that's just what I used at
home prior to the Fedora switch.
I'm also very interested in gaining for *NIX admin experience =] So if
I can help out it'd be a great learning experience.
16 years, 5 months
blog praise for Infrastructure / Web teams
by Matt Domsch
Good work teams!
http://blog.linuxtoday.com/blog/archives/071113-111653.html
[snip]
November 13, 2007
Spinning a New Kind of Distro
When Fedora 8 came out last week, the first thing that came to mind
was this was the first release of Fedora in recent memory that didn't
have some sort of glitch associated with it.
You know the kind: early mirrored versions come out, someone gets wise
and then posts it on Digg or Slashdot, people start pulling down ISOs,
chaos ensues.
This time, however, there was none of that. Announcements went up,
mirror sites snapped on, and Fedora 8's first day was smoothly done.
This may seem like a weird thing to point out, but I think it offers
us a glimpse of the overall efficiency that's becoming more of a
factor in the Fedora Project overall.
--
Matt Domsch
Linux Technology Strategist, Dell Office of the CTO
linux.dell.com & www.dell.com/linux
16 years, 5 months
Wiki
by Paulo Santos
Hi all,
Now that everything calmed down, i've removed app2 from the wiki
loadbalancer so that the editing problems go away.
It should be picked up deployed at the next puppet run
Please let us know if you still find any problem.
Paulo
16 years, 5 months
Meeting Log - 2007-11-08
by Ricky Zhou
15:01 * jima
15:01 * ricky
15:01 -!- Netsplit zelazny.freenode.net <-> irc.freenode.net quits: yingbull
15:02 < abadger1999> Grr
15:02 < ricky> Crazy netsplits?
15:02 -!- Netsplit over, joins: yingbull
15:02 < abadger1999> Yesh. making me crazy
15:02 -!- giarc_w [i=hidden-u(a)gnat.asiscan.com] has joined #fedora-meeting
15:02 -!- mmcgrath changed the topic of #fedora-meeting to: Infrastructure -- Meeting time - Who's here?
15:03 * jima
15:03 * ricky
15:03 < fchiulli> frank
15:03 * mmcgrath is here
15:03 -!- jmtaylor [n=jason(a)c-76-112-115-86.hsd1.mi.comcast.net] has joined #fedora-meeting
15:04 * dgilmore is here
15:04 < jcollie> hello
15:04 * jima has to leave early
15:04 < mmcgrath> skvidal: paulobanon: lmacken: others?
15:04 < mmcgrath> jcollie: yo
15:05 < skvidal> pong
15:05 < jima> and i may have to wander off to do Real Work(tm).
15:05 < jcollie> real work sucks
15:05 * jcollie is jealous of mmcgrath dgilmore and abadger1999 :)
15:05 * nirik is in the rabble seats.
15:05 < jima> nirik: we're all rabble here. :P
15:06 < dgilmore> jima: why ?
15:06 < dgilmore> jcollie: why ?
15:06 < mmcgrath> Alrighty, lets get started.
15:06 * nirik might be interested in doing more infrastructure stuff someday...perhaps when pkgdb gets rid of the need to process cvsadmin requests. ;)
15:06 < jcollie> dgilmore: your new job of course :) (congrats by the way)
15:06 -!- mmcgrath changed the topic of #fedora-meeting to: Infrastructure -- Tickets (the Release)
15:06 < dgilmore> jcollie: :) thanks
15:07 -!- CyberSpy [n=cyberspy(a)cpe-065-191-024-225.nc.res.rr.com] has joined #fedora-meeting
15:07 * bpepple lurks in the background.
15:07 < ricky> Mass ticket-closing? :)
15:09 < mmcgrath> https://hosted.fedoraproject.org/projects/fedora-infrastructure/query?sta...
15:10 < dgilmore> mmcgrath: i think there all done :)
15:12 < skvidal> okay
15:12 < skvidal> so
15:12 < skvidal> mmcgrath has been netsplit into oblivion
15:12 < skvidal> or disconnected
15:12 < skvidal> so
15:12 < skvidal> ya
15:13 -!- Netsplit zelazny.freenode.net <-> irc.freenode.net quits: fab_away, gregdek, lancelan, Standbye, mmcgrathbot, couf_afk, clarkbw, so_solid_moo, mbacovsk, mmcgrath, (+2 more, use /NETSPLIT to show all of them)
15:13 < jima> oh man.
15:13 < skvidal> he just jabbered me to say 'well that sucks'
15:13 < jcollie> uhm
15:13 < skvidal> see
15:13 < dgilmore> :P
15:13 < ricky> Aha, there it goes.
15:13 < ricky> Hehe.
15:13 < jcollie> fedora needs a jabber server
15:13 < ricky> +1 to that idea :)
15:13 < jima> or irc.fedoraproject.org ;)
15:13 < skvidal> +1 to jabber
15:13 < dgilmore> skvidal: +1 to jabber
15:13 < ricky> Some projects have irc.project.org CNAME to Freenode.
15:13 < skvidal> tho a lot of red hat folks are bizarrely addicted to irc
15:13 -!- Netsplit over, joins: mmcgrathbot, paulobanon, mmcgrath, so_solid_moo, couf_afk, fab_away, Standbye, clarkbw, mbacovsk, gregdek (+2 more)
15:13 < dgilmore> conference rooms rock
15:14 < notting> skvidal: does 'too lazy to learn jabber' count?
15:14 < tibbs|h> notting: +1
15:14 < skvidal> notting: no
15:14 * jima hasn't done much with jabber lately (although he needs to get back into it)
15:14 < dgilmore> notting: theres nothing to learn
15:14 < mmcgrath> What did I miss?
15:14 < skvidal> mmcgrath: we're opening a jabber server
15:14 < ricky> 15:13:12 < jcollie> fedora needs a jabber server
15:14 < dgilmore> mmcgrath: jabber.fp.o
15:15 < mmcgrath> :: sigh ::
15:15 < skvidal> *love*
15:15 < mmcgrath> Ok, anyway, hurray on the release.
15:15 < dgilmore> mmcgrath: so looks like we got all those tickets done
15:15 < dgilmore> :)
15:15 < mmcgrath> we need to make sure the mirrors are ready *before* we switch the website.
15:15 < dgilmore> congrats to everyone that got things done
15:16 < skvidal> dgilmore: hey what about those of us who just got in the way?
15:16 < ricky> And we need to double check website links before too :/
15:16 < skvidal> dgilmore: don't we get a congrats, too?
15:16 < dgilmore> skvidal: thats why i watched
15:16 < dgilmore> not to get in the way
15:16 < dgilmore> skvidal: nope you get a slap upside your head :)
15:16 < skvidal> thanks
15:16 < skvidal> dgilmore: next fudcon you have to carry spot
15:17 < dgilmore> skvidal: i did that last one
15:17 < mmcgrath> Anyone have anything else to say about the release?
15:17 < skvidal> dgilmore: it's your job now :)
15:17 < skvidal> mmcgrath: yay?
15:17 < dgilmore> mmcgrath: we seem to be getting better at it
15:17 < skvidal> mmcgrath: fedora++?
15:17 < jcollie> werewolf rocks
15:17 < skvidal> forward the fedora?
15:17 < dgilmore> skvidal: thanks
15:17 < skvidal> Zod is pissed?
15:17 < dgilmore> Zod might go on the warpath
15:17 < jcollie> in the land of strange coincidences, my 6yo son chose to be a werewolf for halloween this year
15:17 < dgilmore> i say give it a month then we huke him
15:17 < ricky> Heh.
15:18 < dgilmore> s/huke/nuke/
15:18 < skvidal> dgilmore: you huke
15:18 < dgilmore> skvidal: i just make people puke
15:18 < skvidal> dgilmore: esp spot
15:18 < jima> argh!
15:18 < skvidal> I think we lost some people again
15:18 * spot ralfs
15:18 -!- G__ [n=njones@wikipedia/NigelJ] has joined #fedora-meeting
15:18 * dgilmore picks spot up
15:19 < jima> skvidal: my connection hosed for about two minutes, yeah.
15:19 < dgilmore> mike is dead in the water again
15:19 * lmacken rolls in late
15:19 * ricky is curious about SELinux coolness. (saw that ticket from lmacken)
15:19 < ricky> Well, speak of the devil...
15:19 < lmacken> yeah..
15:19 < dgilmore> so We made some good strides this release
15:19 < lmacken> it's probably not going to be a very easy task
15:20 * mmcgrath thinks so too.
15:20 < mmcgrath> Ok, it doesn't seem anyone has anything else.
15:20 < lmacken> most of our machines should be in permissive mode.. although for some odd reason that caused issues with mock, so we disabled it
15:20 * ricky plans to leave SELinux enabled on his computer for the first time this relase :)
15:20 < dgilmore> One thing we need to plan for is rebuilding the builders
15:20 < lmacken> we'll just have to keep an eye out for violations, and fix them as they come.. until we are satisfied that nothing will explode
15:20 < abadger1999> When are we going to unfreeze?
15:20 -!- k0k [n=k0k@fedora/k0k] has joined #fedora-meeting
15:21 < dgilmore> abadger1999: i say give it a week
15:21 * ricky double-checks slashdot for the article...
15:21 < dgilmore> mmcgrath: im thinking we should rebuild the builders in 3 weeks time
15:21 < dgilmore> do it on a weekend
15:22 < mmcgrath> No doubt, this release went even better then F7 did and way way better then FC6
15:22 -!- mmcgrath changed the topic of #fedora-meeting to: Infrastructure -- Open Floor
15:22 < mmcgrath> See I said it'd be quick :)
15:22 < mmcgrath> Anyone have anything they'd like / want to talk about?
15:22 -!- mmcgrath [n=mmcgrath(a)74-136-11-144.dhcp.insightbb.com] has quit "leaving"
15:22 < ricky> Guess not.
15:22 -!- mmcgrath [n=mmcgrath(a)74-136-11-144.dhcp.insightbb.com] has joined #fedora-meeting
15:22 < skvidal> hahah
15:23 < dgilmore> mmcgrath: just builders
15:24 < skvidal> dgilmore: what do we use to rebuild the builders?
15:24 < skvidal> dgilmore: do we need to first build builder builders?
15:24 < dgilmore> skvidal: either RHEL5 or F-8
15:24 < skvidal> and then how do we make those?!
15:24 < dgilmore> depending on the python bug
15:24 < skvidal> we'd need to build builder builder builders!
15:24 * mmcgrath votes RHEL5 if we can get away with it.
15:24 < skvidal> +1
15:25 * ricky wonders how we'll be moving away from FC6..
15:25 < dgilmore> mmcgrath: RHEL5 will be similiar to what we use today
15:25 < abadger1999> +1
15:25 < jcollie> +1 on RHEL5
15:26 < jcollie> unless there's going to be a Werewolf LTS edition :)
15:26 < dgilmore> so we should do it the weekend of Dec1 and 2
15:27 -!- rdieter_away is now known as rdieter
15:27 < dgilmore> we should consider what to do with hammer2 and xenbuilder1 they have been in use since not long after Fedora Extras 3 started
15:28 * jcollie accepts hardware donations :)
15:28 * jima too :)
15:28 < dgilmore> they are dual opteron 142's i think
15:28 < dgilmore> with 3gb ram
15:28 * jima would take 'em ;)
15:29 < jcollie> same here :)
15:29 < dgilmore> penguin computing 1u boxes
15:29 * jima has one of those, but it's only dual xeon
15:29 < jima> (w/ 2gb ram)
15:29 < mmcgrath> dgilmore: Once the bladeservers are in place, I'd think we could use them for test servers or additional expansion until they eventually die and get replaced.
15:29 < dgilmore> skvidal: do you remember when we got em
15:30 < dgilmore> mmcgrath: :) any word on that
15:30 < jima> okay, i'm off for the day -- everyone have a nice rest-of-day/evening!
15:30 < dgilmore> should we start beating people up
15:30 < dgilmore> seeya jima
15:30 < ricky> See you.
15:30 < skvidal> umm
15:30 < skvidal> yes
15:30 -!- G [n=njones(a)202-154-149-26.ubs-dynamic.connections.net.nz] has quit Connection timed out
15:30 < skvidal> gafton was in charge
15:30 < skvidal> so
15:30 < skvidal> a looooooooooooong time ago
15:30 < dgilmore> it was awhile ago
15:31 < dgilmore> ppc1 was gotten at the same time
15:31 < dgilmore> or shortly after
15:31 -!- hircus [n=michel(a)dhcp-cs-244-140.cs.indiana.edu] has joined #fedora-meeting
15:31 -!- fchiulli [i=824c400e(a)gateway/web/cgi-irc/ircatwork.com/session] has left #fedora-meeting []
15:32 < dgilmore> so we need to annouce the buildsys downtime soon give a few weeks notice
15:32 < dgilmore> do we have other systems on FC-6 we need to look at migrating
15:33 < ricky> Aren't app3/4 FC6?
15:35 < abadger1999> ricky: Yes. We have to migrate those as well.
15:35 < dgilmore> anyways lets talk about this on the list and get an action plan together
15:35 < dgilmore> for now lets finish the meeting and relax for a minute
15:36 < ricky> Sound good to me :)
15:36 < abadger1999> I think we have all the packages necessary to run the TG apps on RHEL5 built. Now we need to test mirrormanager, pkgdb, smolt, etc on publictest10 (RHEL5 box) and then migrate.
15:36 < ricky> Nothing depends on python 2.4?
15:36 < dgilmore> abadger1999: :)
15:36 < dgilmore> ricky: rhel5 is python 2.4
15:37 < abadger1999> RHEL5 and FC6 both use python2.4
15:37 < ricky> Oop, mixed that up.
15:38 -!- giarc_w [i=hidden-u(a)gnat.asiscan.com] has quit "Leaving"
15:38 < dgilmore> so for now
15:38 * jcollie loves foisting salesdroids off on my boss
15:38 < dgilmore> ===Meeting Over ====
15:38 < dgilmore> thanks yall
16 years, 5 months