#fedora-meeting: Infrastructure (2012-09-27)
Meeting started by nirik at 18:00:01 UTC. The full logs are available at
* hey hey! (nirik, 18:00:01)
* New folks introductions and Apprentice tasks (nirik, 18:02:59)
* Applications status / discussion (nirik, 18:04:04)
* Sysadmin status / discussion (nirik, 18:31:46)
* Private Cloud status update (nirik, 18:35:56)
* Security FAD update (nirik, 18:39:24)
* Upcoming Tasks/Items (nirik, 18:40:22)
* 2012-10-09 to 2012-10-23 F18 Beta Freeze (nirik, 18:40:34)
* 2012-10-23 F18 Beta release (nirik, 18:40:35)
* 2012-11-13 to 2012-11-27 F18 Final Freeze (nirik, 18:40:35)
* 2012-11-20 FY2014 budget due (nirik, 18:40:35)
* 2012-11-22 to 2012-11-23 Thanksgiving holiday (nirik, 18:40:35)
* 2012-11-26 to 2012-11-29 Security FAD (nirik, 18:40:35)
* 2012-11-27 F18 release. (nirik, 18:40:37)
* 2012-11-30 end of 3nd quarter (nirik, 18:40:41)
* 2012-12-24 to 2013-01-01 Red Hat Shutdown for holidays. (nirik,
* 2013-01-18 to 2013-01-20 FUDCON Lawrence (nirik, 18:40:45)
* Open Floor (nirik, 18:41:41)
Meeting ended at 18:48:16 UTC.
Action Items, by person
People Present (lines said)
* nirik (115)
* threebean (29)
* dgilmore (25)
* abadger1999 (24)
* lmacken (12)
* smooge (12)
* zodbot (5)
* relrod (5)
* herlo (3)
* akshaysth (2)
* _love_hurts_ (2)
* jwb (1)
* ricky (0)
* mdomsch (0)
* skvidal (0)
* CodeBlock (0)
18:00:01 <nirik> #startmeeting Infrastructure (2012-09-27)
18:00:01 <zodbot> Meeting started Thu Sep 27 18:00:01 2012 UTC. The chair is nirik.
Information about MeetBot at http://wiki.debian.org/MeetBot
18:00:01 <zodbot> Useful Commands: #action #agreed #halp #info #idea #link #topic.
18:00:01 <nirik> #meetingname infrastructure
18:00:01 <zodbot> The meeting name has been set to 'infrastructure'
18:00:01 <nirik> #topic hey hey!
18:00:01 <nirik> #chair smooge skvidal CodeBlock ricky nirik abadger1999 lmacken
dgilmore mdomsch threebean
18:00:01 <zodbot> Current chairs: CodeBlock abadger1999 dgilmore lmacken mdomsch
nirik ricky skvidal smooge threebean
18:00:13 <nirik> who alls around for a infra meeting?
18:00:16 * _love_hurts_ present
18:01:26 <smooge> here
18:01:40 <dgilmore> hola
18:01:43 * nirik waits for more folks to wander in
18:01:48 * lmacken
18:02:04 * herlo
18:02:39 * threebean
18:02:57 <nirik> ok, I guess lets go ahead and get started...
18:02:59 <nirik> #topic New folks introductions and Apprentice tasks
18:03:04 <nirik> Any new folks this week?
18:03:14 <nirik> or apprentices like to discuss questions/tickets/whatever?
18:03:57 <nirik> ok...
18:04:04 <nirik> #topic Applications status / discussion
18:04:14 <nirik> Any application/devel news this week or upcoming soon?
18:04:15 * akshaysth sneaks in
18:04:41 <nirik> we did just update python-bugzilla in production and moved
gather-easyfix and review-stats from staging to production to handle that.
18:04:53 <nirik> Please be on the look out for any bugzilla related issues in our
18:05:02 * threebean brought a list of stuff! http://www.fpaste.org/Y77R/
18:05:23 <nirik> nice. ;)
18:05:42 <nirik> can you spew them to the channel? (since fpaste might expire
someday or whatever)
18:05:47 <threebean> right, right
18:05:53 * nirik doesn't mind the flood as long as freenode doesn't. ;)
18:06:02 <threebean> exciting stuff that happened:
18:06:03 <threebean> - new messages being emitted from releng, pkgdb2branch, and
18:06:03 <threebean> - updated to latest fedmsg-0.5.0 in all of staging last week.
18:06:03 <threebean> - You can get fedmsg messages publicly at
18:06:06 <threebean> tcp://hub.stg.fedoraproject.org:9940. I'll write more on
how to do this after
18:06:09 <threebean> the freeze starts.
18:06:11 <threebean> - deployed a new /packages app with some nice bugfixes
18:06:14 <threebean> things to get done before the freeze (12 days until freeze):
18:06:19 <threebean> - datanommer -
18:06:22 <threebean> - koji plugin -
18:06:25 <threebean> - fix busmon in stg, deploy to production (app0*).
18:06:27 <threebean> - update fedmsg to 0.5.0 everywhere in production.
18:06:30 <threebean> This is where we stand now -> http://www.fpaste.org/fENr/
18:06:44 <nirik> cool. ;) Exciting stuff in the messaging lang.
18:06:45 <nirik> land
18:07:02 <nirik> can meetbot report the name of meetings and/or what channel they
18:07:08 <threebean> yes
18:07:26 <threebean> just need to put the new fedmsg on value03
18:07:28 <nirik> cool.
18:08:10 <smooge> tres cool
18:08:48 <nirik> ok, any other application related news?
18:08:57 <nirik> abadger1999: you were going to try and get a pkgdb release pushed
18:09:12 <nirik> relrod: where's the fas new release standing?
18:09:13 <abadger1999> nirik: yeah -- sometime before freeze.
18:09:28 * nirik is looking forward to dropping all those hotfixes. ;)
18:10:03 <abadger1999> nirik: heh -- I already see a few features that might come in
as hotfixes after the release is out :-(
18:10:14 <nirik> yeah, it's neverending. ;)
18:10:56 <abadger1999> nirik: I plan on making this release purely about hotfixes.
Merging all those in. then checking that the files in the release match what's
actually running on the servers.
18:10:59 <nirik> abadger1999 / threebean: speaking of packages/pkgdb...
can that be closed? or is there
further action we can take there?
18:11:11 <abadger1999> Then push it to stg and production in pretty rapid
18:11:29 <nirik> abadger1999: sounds good. Then we can get back on a more regular
release cycle hopefully.
18:11:32 <threebean> nirik: not yet.. almost there.
18:12:08 <abadger1999> nirik: Well -- I'm thinking that that is further out
18:12:29 <abadger1999> nirik: I am very sorely tempted to go for the pkgdb rewrite
to be leaner.
18:12:30 <nirik> Oh, I guess application related: Smolt retirement. We pinged suse
and some other folks that were using it... Suse says no problem with retiring it, it
wasn't even in their two more recent releases, and they removed it from the older one
they still support.
18:12:39 <abadger1999> nirik: and just handle acls (as in that ticket)
18:12:53 <nirik> abadger1999: ok. That makes sense.
18:13:18 <abadger1999> yay! (smolt retirement)
18:13:33 <abadger1999> nirik: Any people we need to contact on our side?
18:13:35 <nirik> I think we still want to wait for word from the other distros.
18:13:41 <nirik> for a bit.
18:13:54 <nirik> but the question is: should we try and retire before f18? or
18:14:13 <nirik> or phased approach... remove client from f18 and retire server
18:14:15 <abadger1999> Should we already be looking into a cgi to /dev/null new data
submited to smolt?
18:14:36 <nirik> yeah, would be good to have that ready.
18:14:45 * threebean seems to recall puiterwijck was going to look at that
18:14:54 <abadger1999> I'd say, remove the server and replace with /dev/null
asap. Remove client in F18 or F19 -- sooner is better.
18:14:58 <nirik> yeah, puiterwijk was going to look into it.
18:15:01 <abadger1999> Cool.
18:15:28 <abadger1999> If he gets bogged down with other work he can let me know and
I'll look into it.
18:15:35 <nirik> I'll see about removing it from comps now.
18:15:38 * dgilmore can remove smolt from f18 right now
18:15:48 <abadger1999> It shouldn't be a very hard thing to code.
18:16:01 <nirik> dgilmore: cool... I think we just need to nuke it from comps...
18:16:10 <dgilmore> nirik: there is a few things to do
18:16:24 <dgilmore> nirik: but we will have to keep the server up until f17 is EOL
18:16:26 * nirik notes we have https://fedoraproject.org/wiki/Smolt_retirement
18:16:44 <dgilmore> nirik: since any install of f17 will prompt to have he profile
18:16:57 <nirik> dgilmore: yeah, we were going to look at replacing it with a sink
service that accepts the data and drops it... along with a landing page that says what
happened to it.
18:17:01 <dgilmore> though i guess that will just fail behind the scenes
18:17:27 <nirik> also a redirect so people who go to their profile or whatever get a
18:17:29 <dgilmore> nirik: if we plan to kill it we should get it out of f18 beta
18:18:20 <nirik> I agree.
18:18:25 <nirik> dropping it from comps should do that.
18:18:55 <nirik> dgilmore: you want to do that? or shall I?
18:19:00 <dgilmore> drop from comps, block from f18 probably also some other changes
18:19:07 <dgilmore> nirik: I will
18:19:21 <nirik> do we want to set a time now for replacing the server with the EOL
18:19:31 <nirik> dgilmore: awesome. Thanks.
18:19:35 <dgilmore> nirik: thats less time critical
18:19:37 <smooge> when is beta
18:19:38 <dgilmore> but we should
18:19:45 <dgilmore> smooge: 3 weeks i think
18:19:51 <smooge> I would say that is when we replace it
18:19:52 <dgilmore> maybe 2 weeks
18:19:57 <smooge> the server side
18:19:57 <nirik> 2012-10-23
18:20:06 <nirik> (unless we slip)
18:20:18 <dgilmore> my schedule doesnt have room for slips
18:20:23 <lmacken> smolt is currently required by func?
18:20:29 <nirik> lmacken: ?
18:20:32 <nirik> it is?
18:20:41 <nirik> that seems odd.
/usr/lib/python2.7/site-packages/func/minion/modules/hardware.py: # this will fail
if smolt is not installed. That's ok. hal_info will
18:21:22 <lmacken> yeah, weird
18:21:39 <smooge> ha
18:21:41 <abadger1999> still work.
18:21:51 <smooge> hal_info will still work?
18:21:51 <abadger1999> So -- looks safe to remove the dep
18:21:58 <nirik> hal and smolt... two friends sitting in rocking chairs at the old
18:22:07 <herlo> lol
18:22:22 <abadger1999> okay--- maybe not quite
18:22:23 <smooge> I thought we took hal out to the cornfield last release
18:22:26 <abadger1999> It will remove functionality
18:22:38 <abadger1999> (without smolt, getting hw data will return an empty )
18:23:06 <nirik> so, how about we plan for replacing the server after beta... but
before final. Would give time to announce it, etc.
18:23:06 <smooge> I have the feeling that we can fix that by taking the code out of
smolt and sticking it in func. easy peasy.. [is playing programmer manager today]
18:23:30 <nirik> like say... 2012-11-07
18:23:45 <nirik> thats a bit more than a month for notice.
18:24:27 <nirik> or is that not enough time?
18:25:33 <dgilmore> nirik: sounds fine :)
18:25:43 * dgilmore has just taken smolt out the back of the shed
18:26:12 * relrod here, late - just got home from class
18:26:29 * abadger1999 wonders if dgilmore uses an axe or a shotgun
18:26:30 <nirik> proposal: set 2012-11-07 as the date to move the server over to
18:26:37 <abadger1999> +1
18:26:40 <lmacken> so yeah, func needs the hardware enumeration code from smolt...
18:26:47 <smooge> abadger1999, he uses a spoon. it takes longer
18:26:53 <abadger1999> lmacken: That code looks broken to me too...
18:27:00 <lmacken> abadger1999: it's definitely a mess...
18:27:15 <abadger1999> import smolt ; smolt.Hardware() => looks like it
doesn't work with current smolt code
18:27:22 <dgilmore> abadger1999: it was vicious and bloody
18:27:35 <relrod> nirik: late response, but the FAS release is going well. Tomorrow
ends the translations period, but i'll be at ohio linuxfest. Will probably wrap up the
final release Monday.
18:27:43 <nirik> relrod: cool. ;)
18:27:48 <threebean> relrod: awesome :)
18:28:17 <lmacken> abadger1999: smolt.Hardware() works for me
18:28:44 <nirik> yeah, we will need to figure out how best to fix it in func... ;)
we can do that out of band?
18:28:50 <nirik> any other application/devel news?
18:29:00 <lmacken> I'm working on getting a new bodhi bugfix release out before
18:29:12 <lmacken> fixing up some of those push ordering issues that we hit with the
big f18 alpha push
18:29:35 <nirik> cool. Any news on 2.0? or not much on it recently?
18:30:14 <lmacken> not much 2.0 dev lately, more putting out fires in 1.0
18:30:18 <threebean> lmacken: thanks for helping stress test fedmsg the other day,
too. sometime when we're both free, can we revisit that?
18:30:45 <nirik> ok, still hopefully a 2.0 early release after f18 goes out? it
would be cool to have it in for f19...
18:30:48 <lmacken> threebean: yeah, let's turn that code into a benchmarking
18:30:54 * threebean nods
18:31:03 <lmacken> nirik: hopefully around that timeframe
18:31:08 <nirik> cool cool.
18:31:16 <nirik> #topic Sysadmin status / discussion
18:31:17 <relrod> oh I have one more thing
18:31:22 <nirik> #undo
18:31:22 <zodbot> Removing item from minutes: <MeetBot.items.Topic object at
18:31:29 <relrod> python-flask is out of epel-testing now :)
18:31:29 <nirik> relrod: go ahead...
18:31:42 <nirik> ok. Nice.
18:31:46 <nirik> #topic Sysadmin status / discussion
18:31:50 <relrod> (so fedorahosted-app work can start up again too)
18:31:59 <nirik> So, lets see... what do we have in sysadmin land this week/upcoming
18:32:10 <nirik> I have scheduled some mass reboots for monday/tuesday next week.
18:32:17 <nirik> to get us up to date on kernel/glibc, etc.
18:32:48 <smooge> I have been dealing with IBM and hardware issues in general
18:33:22 <nirik> we are also going to be cleaning up the production wiki db
monday... and then we can push some more plugins into it from staging.
18:33:32 <nirik> smooge: yeah, no fun. ;(
18:33:47 <nirik> we have 4 new machines pending getting their remote console
18:33:58 <smooge> I have developed a craving for cigarettes and whisky
18:34:06 <nirik> 1 at osuosl... 1 for a virthost12 and 2 for new kernel builders.
18:34:09 <dgilmore> smooge: i offer you corn
18:34:58 <nirik> we got the new bus instance for production setup and ports into
18:35:41 <nirik> lots of regular stuff.
18:35:55 <nirik> which leads on to...
18:35:56 <nirik> #topic Private Cloud status update
18:36:12 <nirik> we have our euca cloudlet and our openstack cloudlet both up and
18:36:35 <nirik> we are testing things on them... skvidal is going to try and run a
ftbfs run on the euca one soon.
18:37:00 <nirik> if anyone would like to help us test on them, see me or skvidal for
access after the meeting.
18:37:16 <threebean> do you have a timeframe for testing?
18:37:22 * akshaysth nods
18:37:28 <nirik> I am almost 100% going to reinstall the openstack one in a few
weeks when folsom comes out.
18:37:35 <_love_hurts_> I'm interested in cloud things
18:37:57 <nirik> threebean: well, on openstack it will get reinstalled in a few
weeks. ;) On euca I think we are more stable, so instances there might be able to exist
18:38:18 <threebean> ok :)
18:38:36 <nirik> keeping in mind the cloudy mindset... that instances are
transitory... save off your data to remote or a persistent volume if you care about it.
18:39:24 <nirik> #topic Security FAD update
18:39:46 <nirik> ok, so I think we are just going to go with the orig dates and I
should move that forward seeing if we have budget for everyone, etc.
18:39:52 <nirik> I will try and work on that in the coming week.
18:40:02 <nirik> If you haven't updated your info on the wiki page, please do
18:40:13 <nirik> https://fedoraproject.org/wiki/FAD_Infrastructure_Security_2012
18:40:22 <nirik> #topic Upcoming Tasks/Items
18:40:32 <nirik> Here's what I have for upcoming:
18:40:34 <nirik> #info 2012-10-09 to 2012-10-23 F18 Beta Freeze
18:40:35 <nirik> #info 2012-10-23 F18 Beta release
18:40:35 <nirik> #info 2012-11-13 to 2012-11-27 F18 Final Freeze
18:40:35 <nirik> #info 2012-11-20 FY2014 budget due
18:40:35 <nirik> #info 2012-11-22 to 2012-11-23 Thanksgiving holiday
18:40:35 <nirik> #info 2012-11-26 to 2012-11-29 Security FAD
18:40:37 <nirik> #info 2012-11-27 F18 release.
18:40:41 <nirik> #info 2012-11-30 end of 3nd quarter
18:40:44 <nirik> #info 2012-12-24 to 2013-01-01 Red Hat Shutdown for holidays.
18:40:45 <nirik> #info 2013-01-18 to 2013-01-20 FUDCON Lawrence
18:40:48 <nirik> any other items people would like to schedule or note?
18:41:41 <nirik> #topic Open Floor
18:41:48 <nirik> Any one have anything for open floor?
18:42:05 <dgilmore> just that my hours over the next 2 months will be all over the
18:42:11 <nirik> ideas/comments/feedback/bacon?
18:42:19 <herlo> rub some bacon on it :)
18:42:38 <nirik> dgilmore: are there going to be specific days you are offline? feel
free to let me know if there's anything I can help with while you are offline.
18:43:02 <dgilmore> nirik: around FUDCon i mostly wont be online
18:43:14 <dgilmore> im taking most of the week of Oct 8th off
18:43:31 <dgilmore> im taking off from Dec 4-11
18:43:33 <nirik> ok. Just wanted to know if there's composes or whatever that
need doing. ;) Shouldn't be then...
18:43:41 <dgilmore> and there will be flight days in there
18:43:58 <dgilmore> nirik: i tried to schedule things so ill be around for composes
18:44:03 <nirik> also jwb offered to assist with composes or updates.
18:44:08 <dgilmore> yeah
18:44:57 <jwb> i did
18:45:00 <nirik> ok, if nothing else, we can call it a meeting. ;)
18:45:12 <threebean> real quick -> I was looking at modules-staging earlier today
and hoping that we could clean it out before beta freeze... but a lot of that already
happened today. :)
18:45:38 <nirik> threebean: yeah, would be good to clean out as much of that as we
can before freeze.
18:45:58 * threebean nods
18:47:06 <nirik> ok, will set the meeting fuse for 1 min... ;)
18:48:13 <nirik> ok, thanks for coming everyone. Lets keep on truckin in
#fedora-admin, #fedora-noc and/or #fedora-apps. ;)
18:48:16 <nirik> #endmeeting