============================================ #fedora-meeting: Infrastructure (2013-10-31) ============================================

Meeting started by nirik at 19:00:07 UTC. The full logs are available at http://meetbot.fedoraproject.org/fedora-meeting/2013-10-31/infrastructure.20... .

Meeting summary --------------- * welcome y'all (nirik, 19:00:07)

* New folks introductions and Apprentice tasks (nirik, 19:02:50)

* Applications status / discussion (nirik, 19:06:25) * still working on askbot upgrade in stg (nirik, 19:07:40) * bodhi bug 960642 hopefully quashed. (nirik, 19:07:54) * fedocal in stg is 0.3.0 prerelease. Please test. (nirik, 19:08:53) * LINK: https://gist.github.com/ralphbean/7204706 (threebean, 19:17:20)

* Applications status / discussion (nirik, 19:19:33)

* Sysadmin status / discussion (nirik, 19:27:50) * LINK: https://github.com/ansible/ansible-examples (abadger1999, 19:35:02)

* Upcoming Tasks/Items (nirik, 19:40:29) * LINK: https://apps.fedoraproject.org/calendar/list/infrastructure/ (nirik, 19:40:29)

* Open Floor (nirik, 19:44:03) * apprentice feedback mail will optionally go to list next time. (nirik, 19:48:04)

Meeting ended at 19:48:55 UTC.

Action Items ------------

Action Items, by person ----------------------- * **UNASSIGNED** * (none)

People Present (lines said) --------------------------- * nirik (82) * abadger1999 (27) * threebean (24) * pingou (23) * lmacken (19) * anshprat (18) * ianweller (15) * masta (7) * smooge (7) * croberts (7) * zodbot (4) * dgilmore (4) * puiterwijk (0) * mdomsch (0) * relrod (0) -- 19:00:07 <nirik> #startmeeting Infrastructure (2013-10-31) 19:00:07 <zodbot> Meeting started Thu Oct 31 19:00:07 2013 UTC. The chair is nirik. Information about MeetBot at http://wiki.debian.org/MeetBot. 19:00:07 <zodbot> Useful Commands: #action #agreed #halp #info #idea #link #topic. 19:00:07 <nirik> #meetingname infrastructure 19:00:07 <nirik> #topic welcome y'all 19:00:07 <nirik> #chair smooge relrod nirik abadger1999 lmacken dgilmore mdomsch threebean pingou puiterwijk 19:00:07 <zodbot> The meeting name has been set to 'infrastructure' 19:00:07 <zodbot> Current chairs: abadger1999 dgilmore lmacken mdomsch nirik pingou puiterwijk relrod smooge threebean 19:00:19 * pingou here 19:00:26 * masta waves 19:00:36 * abadger1999 here 19:00:41 <threebean> hi :) 19:00:41 * anshprat here 19:00:44 <smooge> Hello from my phone 19:00:48 * lmacken 19:00:59 <ianweller> hi 19:01:05 <ianweller> also from phone :I 19:01:12 <ianweller> :o * 19:01:44 <nirik> hello smooge's phone. meet ianweller's phone. :) 19:02:14 <dgilmore> hola 19:02:21 <dgilmore> ¿como estas? 19:02:43 <nirik> :) 19:02:50 <nirik> #topic New folks introductions and Apprentice tasks 19:03:05 <nirik> any new folks want to say hi, or apprentices have questions or comments? 19:03:10 <masta> yes 19:03:18 <masta> I'm new - hello all 19:03:24 <anshprat> hi masta 19:03:40 * croberts waves 19:03:48 * masta waves back 19:03:50 <nirik> hey masta. Welcome 19:03:58 <masta> glad to be here 19:04:08 <nirik> you wanted to help out with the aarch64 stuff with dgilmore ? or did you have other interests as well? 19:04:44 <dgilmore> nirik: im going to get him involved in helping me 19:04:48 <dgilmore> not just aarch64 19:04:54 <masta> will probably just focus on aarch64, and while doing that watch and learn from all of you... pickup tasks where I can 19:04:55 <nirik> excellent. 19:05:13 <nirik> well, welcome and do ask questions as things come up, etc. 19:05:16 <croberts> nirik: i closed that wiki ticket with the timeout i will be working on picking up more and working on them 19:05:42 <nirik> croberts: great. I can dig out the wiki related ones if you want those specifically. ;) 19:05:45 <masta> I'm sure I'll be helping rel-eng alot... that is my goal 19:05:51 <croberts> nirik: yes please :) 19:05:51 <anshprat> askbot - upstream says they have hardcoded version on one of the dependency - and that could be the issue. 19:06:11 <anshprat> so I need to figure out how to fix that without hardcoding version dependency 19:06:11 <nirik> anshprat: hey. :) Back from vacation? 19:06:25 <nirik> #topic Applications status / discussion 19:06:38 <anshprat> nirik: :) yes, but haven't been able to spend much time yet. 19:07:23 <lmacken> bodhi bug #960642 popped back up this week, but I am fairly confident that I squashed it yesterday. 19:07:29 <nirik> anshprat: no worries. I reenabled puppet on it (since it mails every 30min about puppet being off... we can disable it again when you are ready to work on it) 19:07:40 <nirik> #info still working on askbot upgrade in stg 19:07:44 <anshprat> nirik: alright 19:07:54 <nirik> #info bodhi bug 960642 hopefully quashed. 19:08:06 <nirik> Any other application news? 19:08:09 <pingou> fedocal on stg is running a pre-release of 0.3.0 19:08:33 <lmacken> working on the bodhi2 masher as a fedmsg consumer, which will let us take advantage of twisted deferred callback chains, easy threading, etc. 19:08:41 <pingou> and the unit-tests coverage of pkgdb2 bumped to 97% today (with all the precautions we should take on this measurement) 19:08:51 <lmacken> pingou: win! 19:08:53 <nirik> #info fedocal in stg is 0.3.0 prerelease. Please test. 19:09:21 <pingou> lmacken: but that does imply fedmsg becomes a source of information that we rely on, right? 19:09:22 <nirik> lmacken: cool. How far are we from tossing up a cloud instance or something where we can see bodhi2. ;) 19:09:49 * croberts goes afk for a few 19:10:11 <lmacken> pingou: yeah, to trigger a push it would be. we could also potentially avoid the public bus by having the bodhi-client send the message locally on a releng box, etc. 19:10:30 <lmacken> but I think we could do it w/o changing any of the current bodhi masher messages 19:10:41 <nirik> someone at the door here, brb 19:10:42 <pingou> cool 19:11:17 <threebean> lmacken: cool. if you need to change them though, we can update fedmsg.meta to match it no problem. 19:11:25 <threebean> agility! 19:11:27 <lmacken> nirik: there are no widgets or templates ported, which is probably a little ways off. the json api is solid at the moment, and I'm going to focus on getting headless mashing & API working first 19:11:38 <lmacken> python-fedora compat, etc. 19:11:57 <lmacken> threebean: cool. I'd also want to ensure that the masher message is signed by someone in releng 19:12:01 <smooge> Sounds good 19:12:24 * threebean nods 19:12:46 <threebean> lmacken: yeah, we can make the required cert group readable by an appropriate releng group. 19:12:52 <ianweller> datagrepper 0.2.1 still has most of the new 0.2.x features broken in production, i'll be working on that this weekend hopefully 19:13:02 <lmacken> threebean: perfect. 19:13:02 <ianweller> (at least, openid login is broken) 19:13:18 <ianweller> (this is due to it being two servers and only having tested it in staging on one) 19:13:26 <lmacken> is /raw still sluggish? 19:13:32 <ianweller> yeah i don't know why it is. 19:13:39 <ianweller> it was fine when we upgraded and restarted apache 19:13:40 <lmacken> are we using indexes? 19:13:44 <nirik> cool. 19:13:47 <ianweller> upgraded datagrepper / restarted apache that is 19:13:52 <anshprat> puiterwijk: any news on postja? 19:14:17 <ianweller> so, datagrepper getting really slow / impossible to use is still an open issue 19:14:59 <anshprat> :! 19:15:25 <threebean> lmacken: yes 19:15:31 <smooge> Ianweller if response >20 reboot 19:15:43 <lmacken> threebean: I'm not seeing any in the model? 19:15:46 <ianweller> smooge: basically 19:15:52 <threebean> lmacken: nope. manually created. 19:16:00 <lmacken> :( 19:16:03 <lmacken> on which cols? 19:16:07 <threebean> timestamp 19:16:14 <threebean> maybe the user and package relations too.. I forget. 19:16:26 <nirik> so whatever became of the idea of checking datagrepper messages stored vs logs or whatever to see if we are dropping anything? I guess thats pending daragrepper being fully working? 19:16:40 <threebean> yeah, I've been running that as of earlier this week. 19:16:49 <threebean> zero dropped messages detected so far. 19:16:55 <lmacken> I'd probably index user.name for sure 19:16:55 <threebean> (I'm just checking koji, though). 19:17:03 <nirik> threebean: nice! 19:17:15 <nirik> could we run a check on other stuff too, just to be paranoid? 19:17:18 <pingou> that's nice 19:17:20 <threebean> https://gist.github.com/ralphbean/7204706 19:17:25 <lmacken> threebean: excellent. 19:17:34 <threebean> everyone run that ^^ 19:17:42 <threebean> erm.. please. 19:17:43 <pingou> at the same time ? :) 19:18:31 <threebean> pingou: well, maybe not ;p 19:18:37 <pingou> doh :( 19:18:54 <threebean> I dunno.. it *should* be able to take it, but, as we're saying, datagrepper is feeling sluggish. 19:19:33 <nirik> #topic Applications status / discussion 19:21:31 <smooge> I am not to touch hardware today 19:21:51 <ianweller> threebean: i wonder if the datanommer db is getting deadlocked somewhere 19:22:37 <ianweller> also, surprisingly, that script you posted runs fine on KU EECS fedora machines. apparently the koji client is installed 19:22:41 <abadger1999> ianweller: to test that -- do you have two datanommer frontend machines? do they both get slow at the same time? 19:22:47 <threebean> ianweller: :D 19:22:51 <abadger1999> ianweller: does restarting one make both fast again? 19:23:00 <pingou> threebean: I got one missed 19:23:04 <ianweller> abadger1999: we have two datagrepper frontend machines. i haven't actually done any real testing, so everything i'm saying is just hearsay 19:23:15 <abadger1999> k 19:23:44 <threebean> pingou: hm. did the script output which thing was missed? 19:23:51 <pingou> * For 2013-10-31 19:20:40.236530 fedmsg did not have https://koji.fedoraproject.org/koji/buildinfo?buildID=475069 19:23:57 <threebean> pingou: and.. was it a race condition? 19:24:05 <pingou> threebean: that's all I have 19:24:14 * threebean check datagrepper for that build 19:24:14 <ianweller> threebean: 475071 koji didn't have 19:24:36 <lmacken> we should sync up the model with the db tweaks by hand, like: __table_args__ = (sa.Index('index_name', 'colname'),) 19:24:44 <threebean> +1 19:24:58 <lmacken> because there may be some low-hanging indexable fruit 19:25:51 <nirik> ok, any other applications news? 19:26:10 <nirik> pingou: how's the jenkins working with the new f19? 19:26:28 <pingou> nirik: so far didn't see/hear any problem 19:26:34 <nirik> cool. 19:26:43 <pingou> more a request for a F20 host :) 19:26:51 <pingou> but I'm not sure we have the IP for it 19:27:06 <nirik> we can get some more I think... 19:27:09 <pingou> so I'm thinking replace F18 by F20 when F20 is beta/release 19:27:11 <nirik> just need to reserve them 19:27:21 <pingou> ok, I'll need to see that with you then :) 19:27:31 <nirik> yep. Ping me out of meeting and we can try and do it. 19:27:50 <pingou> not today, but around half november would be good for me 19:27:50 <nirik> #topic Sysadmin status / discussion 19:27:56 <nirik> pingou: sounds good. 19:28:03 <nirik> lets see... on the sysadmin side... 19:28:35 <smooge> Hardware 100 smooge 0 19:28:45 <nirik> abadger1999: want to share any ansiblefest stuff? 19:28:48 <smooge> :) 19:28:53 * pingou takes the magnet from smooge 19:28:57 <nirik> we have our new bladecenter up and I am trying to get iscsi working on it. 19:29:25 <abadger1999> Lets see... I think we could move the fasClient runs from pull based on each host to push based from lockbox01. 19:29:35 <abadger1999> That might help with our nightly fasClient emails 19:30:17 <nirik> abadger1999: that does mean things have to be ssh reachable from lockbox01, but if they aren't a missed fasClient run is likely the least of their problems. 19:30:33 <abadger1999> nirik: yep. 19:30:57 <abadger1999> nirik: If we have things that are permanently not reachable via ssh from lockbox01, ansible won't be able to work with them at all 19:31:03 <nirik> yep. 19:31:04 <abadger1999> we'll have to configure those hosts differently. 19:31:35 <abadger1999> * f13 pointed me at authprogs https://pypi.python.org/pypi/authprogs as a possible better way to control normal users and scm repos on pkgs.fp.o and fedorahosted. 19:31:38 <nirik> we only have 2 of those that I can think of... the sign-vault machines. On those we only enable sshd when we need to push changes. It's off most of the time. 19:31:49 <abadger1999> It looks promising, puiterwijk was going to look into it. 19:31:53 <abadger1999> <nod> 19:31:57 <nirik> cool 19:32:11 <abadger1999> nirik: do we run fasClient on those two machines regularly or infrequently? 19:32:45 <abadger1999> If regularly, we probably should just leave them pull-based. 19:32:46 <nirik> I'd have to look, I think it runs there... but we could do it as needed, there's few people with access. 19:33:09 * croberts is back 19:33:14 <abadger1999> If infrequently, we can just do fasClient runs at the same time as we explicitly are pushing changes to them. 19:33:19 <abadger1999> <nod> 19:33:21 * nirik nods 19:34:34 <nirik> ok, anything else sysadmin wise? 19:34:36 <abadger1999> For those who don't know, there's the ansible-examples github 19:34:44 <pingou> oh cool 19:34:51 <nirik> yeah. 19:35:02 <abadger1999> https://github.com/ansible/ansible-examples 19:35:19 <abadger1999> It's supposed to have examples that might be good starting points. 19:35:20 <pingou> git grep to the rescure! 19:35:23 <pingou> rescue* 19:35:37 <nirik> yeah, git grep is something I use everyday. ;) 19:35:41 <nirik> very handy 19:36:07 <abadger1999> ansibleworks is going to start up a web service that makes uploading, downloading, rating, etc community provided roles easier in a few months. 19:36:17 <abadger1999> (it'lll be called ansible-galaxy) 19:36:36 <abadger1999> that's about all the big exciting stuff :-0 19:36:38 <nirik> should be a nice source of stuff... 19:36:45 <nirik> abadger1999: whats the licensing on that? 19:36:50 <anshprat> nirik: http://tech-blog.flipkart.net/2013/10/hostdb/ - will we benefit from something like this rather than just inventory in pvt git? 19:36:55 <nirik> or can people specify? 19:37:43 <nirik> anshprat: we have talked about inventory software in the past, but never found one that really was that nice... I've not looked at hostdb tho. 19:37:46 <anshprat> its a " tool to help manage data center inventory and write applications around it. - See more at: http://tech-blog.flipkart.net/2013/10/hostdb/#sthash.qyJQYcfn.dpuf" 19:38:02 <abadger1999> nirik: for the exaples in ansible-galaxy? I don't know for sure -- I imagined it would be uploader-specifies an OSS approved license. 19:38:11 <abadger1999> like googlecode/sourceforge. 19:38:19 <nirik> abadger1999: ok, just curious 19:38:35 <nirik> anshprat: can take a look and discuss on list? 19:38:39 <abadger1999> Rather than "unless specified, this is CC-BY" like stackexchange, etc. 19:38:41 <anshprat> nirik: sure 19:38:54 <abadger1999> Probably should ping the ansibleworks guys to see if that's the case, though. 19:39:17 <nirik> abadger1999: we do want to make sure, because we would likely add to our repo and it's public... 19:39:36 <abadger1999> Yeah. 19:40:29 <nirik> #topic Upcoming Tasks/Items 19:40:29 <nirik> https://apps.fedoraproject.org/calendar/list/infrastructure/ 19:40:37 <nirik> anything anyone wants to schedule or note? 19:40:46 <nirik> I'll note we slipped another week on f20 beta... 19:40:52 <nirik> so, we are frozen another extra week. :( 19:40:53 <pingou> :( 19:40:56 <threebean> :( 19:41:14 <smooge> Jan 1st 19:41:22 <nirik> smooge: of what year? :) 19:41:26 <threebean> smooge: we taking bets? 19:41:34 <anshprat> we have 3 freezes per release I guess ? 19:41:43 <nirik> anshprat: yep. alpha, beta, final. 19:41:55 <nirik> usually they are 2 weeks. 19:42:09 <nirik> but alpha was 3 this time, and beta is 4 now. 19:42:21 <anshprat> so a quarter an year of freezes, atleast.. :! 19:42:40 <nirik> yeah. 19:43:04 <nirik> in a number of ways the freezes are nice tho... forces you to plan things and work on stuff thats not immediately pushing out. 19:43:12 <nirik> just anoying when they get long 19:43:54 <anshprat> yeah.. 19:44:03 <nirik> #topic Open Floor 19:44:10 <nirik> anyone have any items for open floor? 19:44:21 <nirik> questions, comments, ideas, book suggestions? 19:44:42 <anshprat> nirik: the feedback from apprentices.. 19:44:51 <anshprat> any comments / words on those? 19:45:07 <anshprat> (the monthly mail that you sent..) 19:45:15 <nirik> yeah, some. I suppose I should republish out anything from there that's interesting... 19:45:28 <nirik> but we will be doing a new one tomorrow. ;) 19:45:31 <nirik> or... 19:45:41 <nirik> I could mail apprentices and ask/set replies to the list? 19:46:01 <nirik> then everyone could see and comment? or is that too open for feedback? 19:46:11 <pingou> nirik: propose both 19:46:24 <anshprat> yeah, am fine with that.. 19:46:46 <nirik> pingou: yeah, I can... 19:46:47 <croberts> nirik: can i send you an email for a list of the wiki tickets 19:47:00 <nirik> croberts: sure, or I can send you one with them. ;) 19:47:08 <nirik> or we can meet over in #fedora-admin after the meeting... 19:47:59 <croberts> ok sounds good :) 19:48:04 <nirik> #info apprentice feedback mail will optionally go to list next time. 19:48:17 * nirik will close out things in a few if no further comments. 19:48:52 <nirik> thanks for coming everyone! 19:48:55 <nirik> #endmeeting