15:00 -!- mmcgrath changed the topic of #fedora-meeting to: Infrastructure -- Who's
15:01 * yingbull is.
15:01 < mmcgrath> Alrighty everyone, who's around?
15:01 * warren here
15:01 * skvidal is here
15:01 * notting is here
15:01 < mmcgrath> jcollie: you around?
15:02 < jcollie> yup
15:02 < ivazquez> Pong.
15:02 < mmcgrath> cool, lets get started then
15:02 -!- mmcgrath changed the topic of #fedora-meeting to: Infrastructure -- Tickets
15:02 < mmcgrath> .tiny
15:02 < zodbot> mmcgrath: http://tinyurl.com/2hyyz6
15:02 * MostafaDaneshvar is there
15:02 < mmcgrath> First ticket:
15:02 < mmcgrath> .ticket 395
15:02 < zodbot> mmcgrath: #395 (Audio Streaming of Fedora Board Conference Calls) -
Fedora Infrastructure - Trac -
15:02 < mmcgrath> jcollie: whats the latest?
15:02 < warren> mmcgrath: is agenda set in stone? I want to add
s/cvsextras/packager/ after F9 release and next convenient scheduled outage.
15:03 < mmcgrath> warren: not at all, we pretty much go over tickets now, then a few
other items then open the floor.
15:03 < mmcgrath> warren: Remind me if I forget at the end.
15:03 < jcollie> not much has happened... someone horked up a bunch of my virtual
hosts at work so i've been busy rebuilding them
15:03 < mmcgrath> jcollie: virtual hosts at $DAYJOB or something related to fedora?
15:03 -!- Wakko666 [n=blentz(a)office.cardomain.com] has joined #fedora-meeting
15:03 < jcollie> $DAYJOB
15:03 -!- wolfy [n=lonewolf@fedora/wolfy] has left #fedora-meeting ["When you
breathe, you inspire. When you do not breathe, you expire."]
15:04 < mmcgrath> just checking ;-)
15:04 < mmcgrath> jcollie: k, so nothing new there. We'll move on.
15:04 < mmcgrath> .ticket 398
15:04 < zodbot> mmcgrath: #398 (elfutils `monotone' (mtn) error) - Fedora
Infrastructure - Trac -
15:04 * mmcgrath sees if rmcgrath is around.
15:04 < jcollie> i think that those of us that are interested in working on the
streaming should set up a separate meeting to divvy up some tasks
15:05 < mmcgrath> jcollie: not a bad idea. It sounds like you've done most of
the work already. Its just a matter of getting that last 10% done (and getting the fas
integration for the non streamins stuff)
15:05 < jcollie> yep
15:06 < mmcgrath> alrighty, we'll talk about 398 later if rmcgrath becomes
15:06 < mmcgrath> My understanding is that we can do it though.
15:06 < mmcgrath> .tickety 446
15:06 < mmcgrath> .ticket 446
15:06 < zodbot> mmcgrath: #446 (Possibility to add external links on spins page) -
Fedora Infrastructure - Trac -
15:06 < mmcgrath> doath
15:06 * dgilmore is here
15:07 < mmcgrath> dgilmore: I think last time we talked about you getting this up on
the wiki, any news on that?
15:07 < dgilmore> mmcgrath: i started and got sidetracked
15:07 < mmcgrath> dgilmore: ahh, you're still on it though?
15:07 < dgilmore> mmcgrath: will be done by the end of the weekend
15:07 < dgilmore> yep
15:08 < mmcgrath> AFAIK we just have the one spin so far.
15:08 < mmcgrath> dgilmore: sweet, thanks!
15:08 < mmcgrath> Ok, so thats really the end of it on the tickets side.
15:08 -!- mmcgrath changed the topic of #fedora-meeting to: Infrastructure -- New Wiki
15:08 < mmcgrath> Just a bit of a roundup on the wiki. I've been working on th
emigration script and docbook exporter script.
15:08 < mmcgrath> The people listed on
all met yesterday.
15:09 < mmcgrath> I'm going to schedule a few more meetings but a few bumps
aside it sounds like we are a GO.
15:09 < mmcgrath> dgilmore: are you on the FESCo ?
15:09 < dgilmore> mmcgrath: i am
15:09 < mmcgrath> dgilmore: at the next meeting would you find a delegate to sign up
to be the wiki liason for those sections of the wiki?
15:10 < mmcgrath> unless you want to do it :)
15:10 < dgilmore> mmcgrath: sure
15:10 < mmcgrath> Thanks.
15:10 < dgilmore> mmcgrath: i think jwb volunteered
15:10 * jwb wakes up
15:10 < jwb> huh?
15:10 * dgilmore notes he will largely be away next week
15:10 < mmcgrath> so https://publictest1.fedoraproject.org/wiki/FedoraMain
up and ready
15:10 < dgilmore> jwb: wiki migration?
15:10 < mmcgrath> dgilmore: ohhh yeah. you're right. he's on there.
15:11 < jwb> dgilmore, mmcgrath: oh, yeah. i put my name there
15:11 < mmcgrath> dgilmore: I'll be away next week as well.
15:11 < mmcgrath> jwb: sorry I missed it there.
15:11 < jwb> i have no idea what that means
15:11 < jwb> do i have to do something?
15:11 < jwb> it just said "contact point"
15:11 < dgilmore> jwb: you get to be whiped
15:11 < mmcgrath> jwb: not yet, we'll be scheduling a meeing again the week
after next. In the meantime just go through https://publictest1.fedoraproject.org/wiki/
and look for... doom.
15:11 < dgilmore> jwb: no Just be a point person.
15:12 < jwb> ok great
15:12 < jwb> either way, count me in
15:12 < mmcgrath> cool.
15:12 < mmcgrath> So thats really the latest on the wiki.
15:12 < mmcgrath> Anyone have any questions or concerns?
15:13 < mmcgrath> allrighty, moving on
15:13 -!- mmcgrath changed the topic of #fedora-meeting to: Infrastructure -- Next week
15:13 < mmcgrath> I'll be gone again next week for training. Its the last
traing session I actually have scheduled.
15:13 < mmcgrath> So during the days I'll be unavailable but I'll likely be
around most evenings.
15:14 < mmcgrath> Next topic
15:14 < dgilmore> mmcgrath: ill be in Brazil
15:14 * skvidal will be here
15:14 -!- mmcgrath changed the topic of #fedora-meeting to: Infrastructure -- nfs1
15:14 < skvidal> people can call/page me
15:14 < skvidal> ugh
15:14 < skvidal> nfs1
15:14 < mmcgrath> NFS1 got built yesterday in a hurry.
15:14 < mmcgrath> while working on xen2 the nfslock issues showed up again then the
whole box took a panic.
15:15 < mmcgrath> we'd been serving nfs directly from xen2 so when it goes down
the buildsystem goes down.
15:15 < mmcgrath> Anyway, we brought nfs1 up as a RHEL4 box to try to fix the nfs
issues, they might have fixed nfs issues but brought on ext3 issues since we're using
a 10T ext3 filesystem...
15:15 < mmcgrath> RHEl4 couldn't handle it and started to ioerror.
15:15 < mmcgrath> So I kicked a RHEL5 box and its now nfs1.
15:16 < mmcgrath> Its up, its running.
15:16 < mmcgrath> so far no nfslock issues, if we do get one though we're going
to contact steved while its in the bad state and give him a shell to look around.
15:16 < dgilmore> mmcgrath: lets get steved a shell on xen2 and let him monitor
15:16 < mmcgrath> Really nothing new here except that /mnt/koji is no longer
available on xen2, its now on nfs1.
15:17 < mmcgrath> dgilmore: you never know, perhaps magically moving to nfs1 fixed
our problem :)
15:17 < mmcgrath> we'll see.
15:17 * skvidal likes magic
15:17 < dgilmore> mmcgrath: we can hope
15:17 < dgilmore> skvidal: voodoo magic is best
15:17 < mmcgrath> heheh
15:17 * skvidal makes a doll of dgilmore
15:17 < mmcgrath> Ok, next item.
15:17 < dgilmore> skvidal: need some hair and fingernails?
15:18 -!- mmcgrath changed the topic of #fedora-meeting to: Infrastructure --
15:18 < mmcgrath> warren: ping?
15:18 < warren> hi
15:18 < mmcgrath> warren: so we're just going to rename cvsextras to packagers.
15:18 < mmcgrath> which involves
15:18 < warren> So f13 suggested we rename the group after F9 release, at the first
15:18 < warren> mmcgrath: packager, singular
15:18 < mmcgrath> 1) updating the db.
15:19 < mmcgrath> 2) updating the scripts (and upload script)
15:19 < mmcgrath> 3) updating the documentation
15:19 < mmcgrath> 4) testing?
15:19 < mmcgrath> I really don't think this will be that huge of a deal.
15:19 < mmcgrath> warren: I'll try to get it done sometime the week after F9
ships. Would you mind opening a ticket and bugging me about it that week?
15:19 < warren> mmcgrath: ok
15:20 < warren> mmcgrath: are there any other group names we should rename as well?
15:20 < f13> warren: lets keep the name that will make sense in the future given my
15:20 < warren> as long as we have an outage
15:20 < f13> warren: since I hope to get an intern to work on that this summer
15:20 < warren> f13: is this the one from 2 months ago or so?
15:20 < mmcgrath> warren: yeah we'll schedule an outage.
15:20 -!- Milanito [n=Matt(a)32.12.70-86.rev.gaoland.net] has quit Remote closed the
15:20 < f13> warren:
15:21 < mmcgrath> I'm not sure about other group names.
15:21 < warren> Let's ask around
15:21 < warren> there might be other names people want to do
15:21 < warren> f13: ok seems we need to figure out a bigger picture plan then
15:22 < warren> mmcgrath: will get back to you
15:22 < mmcgrath> warren: works for me, lets know well in advance of the outage know
because renaming cvsextras I have a pretty good idea of what needs to be done.
15:22 < mmcgrath> but for another group I might not know without more research.
15:22 < mmcgrath> k
15:22 < mmcgrath> warren: when its all ready just create a ticket and bug me. I
think we could have it done in an hour or two some night after F9 ships.
15:22 < f13> warren: well, we don't necessarly have to wait to rename
'cvsextras' to /somethihng/
15:23 < f13> warren: 'fedorapackager' or whatever seems fine, we can pick a
name that implies /greater/ rights later
15:23 < warren> f13: I do agree to rename it within the larger plan
15:23 < f13> or less rights.
15:23 < warren> f13: I'm still in favor of numbers instead of group names for
15:23 < warren> but we're getting off topic
15:23 < mmcgrath> <nod> you guys figure that out :)
15:24 < mmcgrath> anyone have any questions about that? We've got plenty of
time to discuss it.
15:24 < mmcgrath> if not I'll move on.
15:24 -!- mmcgrath changed the topic of #fedora-meeting to: Infrastructure -- app server
15:24 < mmcgrath> abadger1999: ping?
15:25 < mmcgrath> I believe toshio wants to upgrade sqlalchemy and python-fedora
before the freeze. I think we're on schedule to do that but I'll have to get
ahold of him to find out for sure.
15:25 < mmcgrath> which brings me to the next topic
15:25 < abadger1999> here.
15:25 < mmcgrath> oh
15:25 < mmcgrath> there you are :)
15:25 < abadger1999> Today.
15:26 < mmcgrath> abadger1999: want to talk about all of that right quick?
15:26 < abadger1999> Want to do it today/tonight.
15:26 < abadger1999> It would be to both app servers.
15:26 < mmcgrath> abadger1999: "both" ?
15:26 < mmcgrath> which?
15:26 < abadger1999> app4 and app5
15:26 < mmcgrath> app2 is also running tg apps
15:26 < abadger1999> Oh. I was not aware.
15:27 < dgilmore> mmcgrath: i want the new python-fedora
15:27 < abadger1999> then we wantto upgrade that as well.
15:27 < abadger1999> We can issue a new python-fedora for everything if we want.
15:27 < mmcgrath> abadger1999: your call I'm not sure what we'd be comitting
to with the new package.
15:27 < abadger1999> Or we can update that when we do the other updates from RHEL,
15:27 < mmcgrath> what are the risks / benefits?
15:27 < mmcgrath> and will this break transifex?
15:28 < abadger1999> All the FAS1 stuff has been removed. Only FAS2 stuff is in
15:28 < abadger1999> there's been some rearrangement of modules but the old
imports should work with a deprecation warning.
15:28 < mmcgrath> k
15:29 < mmcgrath> sounds pretty low risk, and we can always revert if required.
15:29 < abadger1999> <nod>
15:29 < mmcgrath> abadger1999: I'm fine for any time this week on that, if you
want me around just ping me or call me.
15:29 < mmcgrath> abadger1999: anything else on that?
15:30 < abadger1999> Nope that's it.
15:30 < mmcgrath> cool.
15:30 -!- mmcgrath changed the topic of #fedora-meeting to: Infrastructure -- app5
15:30 < mmcgrath> So yeah, I don't know WTF is going on but app5 is rebooting...
15:30 < mmcgrath> this comes after the move from x86_64 to i686
15:30 -!- Milanito [n=Matt(a)32.12.70-86.rev.gaoland.net] has joined #fedora-meeting
15:30 < mmcgrath> we're talking between 20-40 times a DAY.
15:30 < mmcgrath> very very strange.
15:30 < mmcgrath> I've got a stack trace, I'm going to try to take it to the
virt guys soon.
15:31 < mmcgrath> side note though, aside from that, the i686 move (also done on
app2) has proved quite successful memory usage (and therefor swap) is way down.
15:31 < mmcgrath> its been a big win for us.
15:31 < mmcgrath> anyone have any questions / comments?
15:31 -!- viking-ice [n=johannbg(a)dsl-149-97-225.hive.is] has joined #fedora-meeting
15:31 < mmcgrath> Alrighty, then I'll move on to the last topic of the day
before opening the floor.
15:32 -!- mmcgrath changed the topic of #fedora-meeting to: Infrastructure -- Change
15:32 < mmcgrath> This one's for everyone
15:32 < mmcgrath> REMEMBER
15:32 < mmcgrath> DO NOT MAKE ANY CHANGES WITHOUT GETTING APPROVAL ON THE LIST
15:32 < mmcgrath> and remember, you'll be asked the question "Why can't
this wait until after F9 gets released"
15:33 < mmcgrath> I'll be gone much of next week which makes the freeze
15:33 < warren> under pain of
15:33 * mmcgrath looks up exact date again
15:33 < mmcgrath> warren: under pain of CURSE!
15:33 < skvidal> under pain of pain
15:33 < warren> I can advise against pain.
15:34 < mmcgrath> There we go, so the freeze starts the 15th. The release is on the
29th. The freeze is lifted on the 30th.
15:34 < yingbull> pain's scary.
15:34 < mmcgrath> Obvious exceptions include outages and things already scheduled
for the release (mm type stuff, the website, etc)
15:34 < mmcgrath> other then that. you must get approval on the list before making
changes. If you're working on tickets and people get cranky, just let them know
we're under a freeze.
15:34 < skvidal> mmcgrath: it's okay to bring up new boxes unrelated to the
15:34 < ivazquez> Should we hold it until the Monday following?
15:35 < skvidal> mmcgrath: ie: if I get all the statics, etc from BU
15:35 < mmcgrath> skvidal: take it to the list! But yeah, that shouldn't be a
15:35 < skvidal> mmcgrath: will do
15:35 < mmcgrath> ivazquez: I thought about that, in the past though, for the most
part, things settled down from the release the day after.
15:36 < ivazquez> Okay.
15:36 < mmcgrath> anyone have any questions / comments about that?
15:36 < mmcgrath> alrighty, then we'll open the floor
15:36 -!- mmcgrath changed the topic of #fedora-meeting to: Infrastructure -- Open Floor
15:36 * dgilmore has nothing right now
15:36 < mmcgrath> Anyone have anything they'd like to discuss? A release is
coming, lots of stuff has been going on.
15:37 < notting> how are we ensuring the ability to get the bits to our tier0
15:37 < mmcgrath> notting: same as we had been, AFAIK our mirror system (and I2) is
15:37 < notting> that's not what mdomsch's mail implied
15:37 < mmcgrath> notting: which email and when?
15:38 < notting> to mirror-list
15:38 < dgilmore> notting: no, it implied a work around was setup
15:38 < dgilmore> mmcgrath: a couple of hours ago
15:38 < dgilmore> mmcgrath: 2 boxes were setup for Tier 0 I2 masters
15:39 < mmcgrath> <nod> that was my understanding. We don't have the
permanent solution in place but a working one.
15:39 < mmcgrath> we'll probably want to make sure the working one stays as is
until after the 30th.
15:39 < dgilmore> mmcgrath: they have static I2 routes to dl.fedora.redhat.com
15:39 < mmcgrath> notting: I'll follow up with mdomsch to make sure whats going
15:39 < mmcgrath> .any mdomsch
15:39 < zodbot> mmcgrath: mdomsch was last seen in #fedora-meeting 1 hour, 9
minutes, and 24 seconds ago: *** mdomsch has quit IRC (Remote closed the connection)
15:39 -!- Milanito [n=Matt(a)32.12.70-86.rev.gaoland.net] has quit Remote closed the
15:40 < mmcgrath> Alrighty, anyone have anything else to discuss?
15:40 < dgilmore> nope
15:40 < mmcgrath> k, we'll close the meeting in 30
15:41 < mmcgrath> 15
15:41 < mmcgrath> 5
15:41 < notting> dgilmore: so, the static route & download1 are
bandwidth-unclogged enough to sync OK?
15:41 < dgilmore> Thanks mmcgrath :)
15:41 < notting> dgilmore: with no one using it, are we sure that download1 is
15:41 < dgilmore> notting: i believe so.
15:42 < mmcgrath> notting: we'll have to talk to them to find out for sure.
15:42 * mmcgrath just sent an email to mdomsch to verify.
15:42 < notting> maybe i'm paranoid
15:42 < mmcgrath> notting: you're once bitten.
15:42 < mmcgrath> nothing wrong with that. The fact is the whole I2 connection
thing showed a major problem we have. and the fact that galgoci is the _only_ person at
Red hat with access and no how to fix it is much less comforting.
15:43 < dgilmore> notting: i believe download1 is down
15:43 < mmcgrath> even his backup told us that he was really the only guy we'd
want doing that.
15:44 -!- k0k [n=k0k@fedora/k0k] has quit Connection timed out
15:44 < mmcgrath> notting: we'll just have to wait and see if what we think
mdomsch said is actually the way things are. That mirror has been down since before Feb
20th so its hard to say the state of things (considering they only just recently got back
up and running)
15:44 < mmcgrath> Anywho, not much we can do but check with the mirrors list and
mdomsch and make sure everyone's happy with where things are for the release.
15:44 < mmcgrath> Anyone have anything else? If not we'll close the meeting in
15:45 -!- Milanito [n=Matt(a)32.12.70-86.rev.gaoland.net] has joined #fedora-meeting
15:45 < mmcgrath> 10
15:45 -!- mmcgrath changed the topic of #fedora-meeting to: Infrastructure -- Meeting
15:45 < mmcgrath> Thanks for coming everone!