20:00 < mmcgrath> #startmeeting Infrastructure
20:00 < zodbot> Meeting started Thu Apr 1 20:00:34 2010 UTC. The chair is
mmcgrath. Information about MeetBot at
http://wiki.debian.org/MeetBot.
20:00 < zodbot> Useful Commands: #action #agreed #halp #info #idea #link #topic.
20:00 -!- zodbot changed the topic of #fedora-meeting to: (Meeting topic:
Infrastructure)
20:00 < mmcgrath> #topic Who's here?
20:00 -!- zodbot changed the topic of #fedora-meeting to: Who's here? (Meeting topic:
Infrastructure)
20:00 * nirik is lurking around.
20:01 * sijis is sorta here
20:01 -!- MrTom [~quassel@fedora/MrTom] has quit Remote host closed the connection
20:01 < mmcgrath> Ok
20:02 < mmcgrath> #topic Fedora 13 Beta
20:02 -!- zodbot changed the topic of #fedora-meeting to: Fedora 13 Beta (Meeting topic:
Infrastructure)
20:02 < smooge> here
20:02 < smooge> is waiting for Lenova to get a support tech
20:02 < mmcgrath> #link
https://fedorahosted.org/fedora-infrastructure/report/9
20:02 < mmcgrath> So as you saw from the list, we slipped by a week.
20:02 * ricky
20:03 < mmcgrath> so the change freeze also slips by one week.
20:03 < mmcgrath> .ticket 2058
20:03 < zodbot> mmcgrath: #2058 (Verify Mirror Space) - Fedora Infrastructure - Trac
-
https://fedorahosted.org/fedora-infrastructure/ticket/2058
20:03 < mmcgrath> I closed that just yesterday, we're in good shape on the
mirrors
20:03 < mmcgrath> .ticket 2059
20:03 < zodbot> mmcgrath: #2059 (Release Day ticket) - Fedora Infrastructure - Trac
-
https://fedorahosted.org/fedora-infrastructure/ticket/2059
20:03 < mmcgrath> Just a tracking ticket
20:03 < mmcgrath> .tiny 2060
20:03 < zodbot> mmcgrath: Error: '2060' is not a valid url.
20:03 < mmcgrath> .ticket 2060
20:03 < zodbot> mmcgrath: #2060 (Verify releng permissions) - Fedora Infrastructure
- Trac -
https://fedorahosted.org/fedora-infrastructure/ticket/2060
20:03 < mmcgrath> >:-|
20:03 < mmcgrath> smooge has that one
20:03 < mmcgrath> .ticket 2061
20:03 < zodbot> mmcgrath: #2061 (MM redirects) - Fedora Infrastructure - Trac -
https://fedorahosted.org/fedora-infrastructure/ticket/2061
20:03 < mmcgrath> No mdomsch again, I wonder if the time change has made this slot
bad for him.
20:03 < mmcgrath> smooge: do you want to take that one for now?
20:04 < mmcgrath> I think you may have done that for the Alpha
20:04 < ricky> I thought the redirects were automatic now or something
20:04 < mmcgrath> ricky: I think they are but mdomsch seemed to want to verify that
so I'm not sure if it's done and we don't have to worry about it, or if
it's an ongoing thing
20:04 < mmcgrath> we should check to see if that can be removed from our list
altogether
20:05 < ricky> OK
20:05 < ricky> Here's the comment from mdomsch about it:
https://fedorahosted.org/fedora-infrastructure/ticket/1993
20:06 < mmcgrath> so in theory if the bug is fixed, we don't have to worry about
it?
20:06 * mmcgrath will make note to talk to sr. domsch next time he's on.
20:06 < mmcgrath> ok
20:06 < smooge> ok
20:06 < mmcgrath> .ticket 2062
20:06 < zodbot> mmcgrath: #2062 (Infrastructure Change Freeze) - Fedora
Infrastructure - Trac -
https://fedorahosted.org/fedora-infrastructure/ticket/2062
20:06 -!- RodrigoPadula [~rodrigopa(a)189.106.8.220] has quit Quit: Saindo
20:06 < mmcgrath> This one's in effect already
20:06 < smooge> sorry I lost net connection
20:06 < mmcgrath> .ticket 2063
20:06 < zodbot> mmcgrath: #2063 (Lessons Learned) - Fedora Infrastructure - Trac -
https://fedorahosted.org/fedora-infrastructure/ticket/2063
20:06 < mmcgrath> smooge: now orries
20:06 < mmcgrath> And lessons learned isn't until after the release.
20:06 < mmcgrath> So that's the release stuff
20:06 * mmcgrath notes it's only 6 minutes since th emeeting started
20:06 < mmcgrath> that could be a new record for release tickets.
20:07 < mmcgrath> anyone have any questions or concerns about the release?
20:07 < mmcgrath> ricky: do you think websites is in good shape?
20:07 < mmcgrath> sijis: ^^
20:08 -!- J5 [~johnp(a)c-65-96-175-112.hsd1.ma.comcast.net] has quit Ping timeout: 265
seconds
20:08 < sijis> yeah. just repo stuff.
20:08 < ricky> Yup, just need to get the banners ready this weekend
20:08 < sijis> tryign to finish up get-* pages..
20:08 < sijis> which can be seen in
https://stg.fp.o
20:08 < mmcgrath> sijis: how'd that go for you?
20:08 < ricky> sijis: Mind sending a link to websites-list? Looking good
20:08 < mmcgrath> the staging environment I mean.
20:09 < sijis> ricky: we need to chat about banner that too. i think tyhey are
trying something different.
20:09 < sijis> mmcgrath: yeah, it was easy to setup. thanks
20:09 < sijis> ricky: will send to list
20:09 < smooge> mmcgrath, you need me to take 2061 correct?
20:10 < mmcgrath> smooge: yeah, we're pretty sure it's done already, just
needs a verification
20:10 < mmcgrath> Ok, well with that we'll move on.
20:10 < mmcgrath> #topic Search Engines
20:10 -!- zodbot changed the topic of #fedora-meeting to: Search Engines (Meeting topic:
Infrastructure)
20:10 < smooge> okie dokie
20:10 < mmcgrath> Doah, no a-k
20:10 < mmcgrath> I thought I saw him earlier. We'll skip this.
20:11 < mmcgrath> #topic Monitoring
20:11 -!- zodbot changed the topic of #fedora-meeting to: Monitoring (Meeting topic:
Infrastructure)
20:11 < mmcgrath> I've been poking around and changing some basic behaviors in
nagios.
20:11 < mmcgrath> Stuff like flagging a host as dead if nrpe is down, and not going
off of pings.
20:11 < mmcgrath> I'm presently (as in just before the meeting started) trying
to get some decent servergroup views in order.
20:12 < mmcgrath> just lots of general config management and things
20:12 < mmcgrath> Anyone have any questions or comments on that?
20:12 < mmcgrath> I'm disappointed that we don't have a combined trend and
monitoring solution
20:12 < mmcgrath> but I have to say, collectd has been awesome.
20:12 < mmcgrath> we've learned several things about the environment just by
using it
20:12 < sijis> so collectd = trend?
20:12 < smooge> I like collectd too
20:13 < mmcgrath> sijis: yeah, have you seen it?
20:13 -!- skvidal [~skvidal(a)sethost.sethdot.org] has joined #fedora-meeting
20:13 < smooge> I can see why so many comapnies wrap nagios + something like it into
an overall product
20:13 < mmcgrath> it's been interesting.
20:13 < sijis> yeah, right after you noticed some db issues on smolts, i think
20:13 < mmcgrath>
https://admin.fedoraproject.org/collectd/bin/index.cgi?hostname=memcached...
20:13 < mmcgrath> even stuff like that has been helpful
20:13 < mmcgrath> we've had a pretty solid 98% hit ratio in memcached.
20:14 < mmcgrath> and, surprisingly to me, all of our memcached stats fit in les
then 90M of space.
20:14 < mmcgrath> I figured it would be much higher then that
20:14 < mmcgrath> but anyway, yeah. Monitoring.
20:14 < mmcgrath> any questions or comments or suggestions about our monitoring
systems?
20:15 < mmcgrath> I really want us to get smarter alerts.
20:15 < mmcgrath> right now when the fit hits the shan, everything kind of goes nuts
and that's just not helpful.
20:15 < mmcgrath> I also want to start taking more metrics like MTTF and MTTR for
our apps.
20:15 < mmcgrath> which is kind of tricky to do at the moment because of all of our
entry points.
20:16 < mmcgrath> Ok, I'll assume nothing there?
20:16 < mmcgrath> #topic Blade Center
20:16 -!- zodbot changed the topic of #fedora-meeting to: Blade Center (Meeting topic:
Infrastructure)
20:16 < mmcgrath> So I'm working to get a new blade center.
20:16 < smooge> whee
20:16 < mmcgrath> Due to some internal accounting bits, it's taking longer than
expected. I'm still hopeful I'll be flying out to Phoenix in mid april (just a
few weeks from now) to install it.
20:17 < mmcgrath> jwb: you around?
20:17 < smooge> do you need me out there?
20:17 < mmcgrath> smooge: naw, and I don't think we have cash to send you anyway
:)
20:17 < smooge> I could hitch-hike?
20:17 < mmcgrath> I *hope* it will be pretty straight forward, I have that and a few
other servers to install.
20:17 < mmcgrath> smooge: maybe you could just will yourself there :)
20:17 -!- mgoldmann [~mgoldmann@redhat/jboss/mgoldmann] has quit Quit: shutdown -h now
20:18 < mmcgrath> Ok, with that I'll open the floor
20:18 < mmcgrath> #topic Open Floor
20:18 -!- zodbot changed the topic of #fedora-meeting to: Open Floor (Meeting topic:
Infrastructure)
20:18 < smooge> no my teleportation skills have been pretty bad lately
20:18 < mmcgrath> anyone have anything they'd like to discuss?
20:19 < smooge> I missed anything on a week delay
20:19 < skvidal> mmcgrath: if the new blade center comes in - will we have some
extra machines?
20:19 < mmcgrath> skvidal: until December we might. We'll be migrating off the
old BC to the new one.
20:19 < mmcgrath> then we'll be removing the old one.
20:19 < skvidal> darn
20:19 < mmcgrath> but yeah, capacity will be up, whats up?
20:20 < skvidal> just thinking about some of the koprs stuff
20:20 < skvidal> and where we're going to find the capacity for that
20:20 < skvidal> esp disk space and builders
20:20 < jwb> mmcgrath, hm?
20:20 < mmcgrath> skvidal: ah, well. I suspect it'all take a bit to get it all
together but I can certainly put it in next years budget for Q1.
20:20 < mmcgrath> and we can do some trails now. We might be able to find space for
all that.
20:21 < mmcgrath> the new builders are significantly beefier then the old.
20:21 < mmcgrath> jwb: hey, so. PPC stuff. Just wanted to make you aware of my
intentions and see what we can do.
20:21 < smooge> mmcgrath, how much would it be to keep the old bladecenter under
warranty even if slow/old
20:21 < mmcgrath> So the new blade center has two PPC blades in it for EPEL
building.
20:21 < smooge> yeah!
20:21 < mmcgrath> that means all our old PPC blades will be EOLed at the end of the
year. Do you happen to have any use for them outside of the colo?
20:22 < jwb> mmcgrath, why EOLed?
20:22 -!- sharkcz [~dan(a)plz1-v-4-17.static.adsl.vol.cz] has quit Quit: Ukončuji
20:22 < mmcgrath> smooge: it'd be a few thousand if they allow it. I actually
extended it to be supported for this year, it's on its 4th year.
20:22 < mmcgrath> jwb: no longer supported.
20:22 < jwb> mmcgrath, the HW or ppc as a release?
20:22 < mmcgrath> the hardware
20:22 < jwb> mmcgrath, so what are you planning on doing for F12 ppc builds?
20:22 < mmcgrath> it'd probably run just fine, but since its not under warranty
(and because the power drops are pretty pricy) we won't be keeping two blade centers
in there at the moment.
20:22 < jwb> because F12 will still be active at the end of the year
20:23 < mmcgrath> jwb: we'll still have some non-blade builders.
20:23 < jwb> oh, ok
20:23 < mmcgrath> ppc4 still has I think 2 years left, certainly 1 year.
20:23 < jwb> mmcgrath, i can't really host the blades anywhere myself. i can
ask around but needing a BC sort of makes it hard to make use of them
20:24 < mmcgrath> jwb: yeah that's what my fear was as well. Doesn't hurt
to ask though. If you happen to find someone let us know.
20:24 < jwb> will do
20:25 < mmcgrath> Ok, and with that does anyone have anything else to discuss?
20:25 < smooge> not me
20:25 < mmcgrath> allllrighty!
20:25 < mmcgrath> #endmeeting
20:25 -!- zodbot changed the topic of #fedora-meeting to: Channel is used by various
Fedora groups and committees for their regular meetings | Note that meetings often get
logged | For questions about using Fedora please ask in #fedora | See
http://fedoraproject.org/wiki/Meeting_channel for meeting schedule
20:25 < zodbot> Meeting ended Thu Apr 1 20:25:51 2010 UTC. Information about
MeetBot at
http://wiki.debian.org/MeetBot .
20:25 < zodbot> Minutes:
http://meetbot.fedoraproject.org/fedora-meeting/2010-04-01/fedora-meeting...
20:25 < mmcgrath> Thanks for coming everyone!
20:25 < zodbot> Minutes (text):
http://meetbot.fedoraproject.org/fedora-meeting/2010-04-01/fedora-meeting...
20:25 < zodbot> Log:
http://meetbot.fedoraproject.org/fedora-meeting/2010-04-01/fedora-meeting...