infrastructure May 2012

infrastructure@lists.fedoraproject.org

42 participants
48 discussions

httpd log files on hosted03
by Seth Vidal 30 May '12

30 May '12

our logfiles on hosted03 for httpd appear to be not being pruned/compress/reaped. on systems using the log-per-day mechanism of rotatelogs from apache - we normally keep the logs for N days then reap any older than whatever. I can nuke the older ones and setup a cron job to do that automagically - just need an okay to do so. -sv

2 1

torrent json stats
by Seth Vidal 29 May '12

29 May '12

accidentally pushed this change but I updated the torrents stats generator to generate the old .json files from the old tracker - so that the spins site will work. -sv

4 3

RE: freeze break: block hits from a domain thats referring too much
by Matt Domsch 27 May '12

27 May '12

+1 -----Original Message----- From: Kevin Fenzi [kevin(a)scrye.com<mailto:kevin@scrye.com>] Sent: Saturday, May 26, 2012 10:09 PM Central Standard Time To: Fedora Infrastructure Subject: freeze break: block hits from a domain thats referring too much Our proxy boxes have jumped up to max httpd processed due to a ton of referring hits from a single domain/page. I'm going to redirect them to 403 for now and see if that brings processes down on the proxies. It's not causing an outage, but it could impact things if it keeps going on. It may also be affecting smolts.org. Retroactive +1s? ;) diff --git a/modules/fedora-web/files/redirects.conf b/modules/fedora-web/files/redirects.conf index feba7d0..26a2b36 100644 --- a/modules/fedora-web/files/redirects.conf +++ b/modules/fedora-web/files/redirects.conf @@ -10,6 +10,10 @@ RewriteRule ^/([^/]+/)?legal/licenses https://fedoraproject.org/wiki/Legal:Licen RewriteRule ^/([^/]+/)?legal/trademarks http://fedoraproject.org/wiki/Legal:Trademark_guidelines [R=301,L] RewriteRule ^/([^/]+/)?legal https://fedoraproject.org/wiki/Legal:Main [R=301,L] +# Drop distributed web referrer hits from klopodavonatyl.net +RewriteCond %{HTTP_REFERER} ^http://(www\.)?klopodavonatyl.net/.*$ +RewriteRule .* - [F] + # Comment this when there is a prerelease available # RewriteRule ^(/.*)?/get-prerelease$ $1/get-fedora [R=302] kevin

1 0

freeze break: block hits from a domain thats referring too much
by Kevin Fenzi 27 May '12

27 May '12

Our proxy boxes have jumped up to max httpd processed due to a ton of referring hits from a single domain/page. I'm going to redirect them to 403 for now and see if that brings processes down on the proxies. It's not causing an outage, but it could impact things if it keeps going on. It may also be affecting smolts.org. Retroactive +1s? ;) diff --git a/modules/fedora-web/files/redirects.conf b/modules/fedora-web/files/redirects.conf index feba7d0..26a2b36 100644 --- a/modules/fedora-web/files/redirects.conf +++ b/modules/fedora-web/files/redirects.conf @@ -10,6 +10,10 @@ RewriteRule ^/([^/]+/)?legal/licenses https://fedoraproject.org/wiki/Legal:Licen RewriteRule ^/([^/]+/)?legal/trademarks http://fedoraproject.org/wiki/Legal:Trademark_guidelines [R=301,L] RewriteRule ^/([^/]+/)?legal https://fedoraproject.org/wiki/Legal:Main [R=301,L] +# Drop distributed web referrer hits from klopodavonatyl.net +RewriteCond %{HTTP_REFERER} ^http://(www\.)?klopodavonatyl.net/.*$ +RewriteRule .* - [F] + # Comment this when there is a prerelease available # RewriteRule ^(/.*)?/get-prerelease$ $1/get-fedora [R=302] kevin

3 3

Yet another freeze break: reviewstatus
by Kevin Fenzi 25 May '12

25 May '12

I'd like to revert puppet commits: 55103ec6ce3a80234391229f6b10e220fa13e72d 5b131d0e489f5349d5cef6a8e0ffdc2febda0441 This reverts review-stats.py to the version that we had before monday, which was working with the old python-bugzilla (which we have already reverted to). I've tested it locally and it does only make 6 bugzilla calls when used with the python-bugzilla version we have reverted to, so we shouldn't be causing undue load on bugzilla. I'd also like to: diff --git a/modules/review-stats/manifests/init.pp b/modules/review-stats/manifests/init.pp index 9a016a5..0376a77 100644 --- a/modules/review-stats/manifests/init.pp +++ b/modules/review-stats/manifests/init.pp @@ -57,7 +57,7 @@ class review-stats::build inherits review-stats { command => "/usr/bin/python /usr/local/bin/pyGetReviewByFlags.py -t /usr/local/share/review-stats -d user => "apache", minute => [ 0, 30 ], - ensure => "absent", + ensure => "present", } } to re-enable it. +1s? kevin

3 2

Questions regarding Koji, fedpkg and SCM
by Jeremy Davis 25 May '12

25 May '12

Hello List, I hope I picked the correct mailing list to ask these types of questions. In my environment we are trying to setup a koji build system using the same structure that you all use for the Fedora project. This involves using fedpkg (modified for our environment) to import the SRPM into the SCM and then tell koji to build. We have everything working with our environment but have to do a couple of things manually. Mainly the SCM creation and adding a tag for the package in koji before the user is allow to add their srpm to the SCM and build with koji (get the error that it can't tag if a tag is not created before hand). Once we create the SCM and add the tag into Koji everything works as it should based on the documentation I have read. My question is this. Do you by chance have any tools that automate the SCM creation and koji tag adding? Is this how you all manage packages in Fedora? Do you have to manually create SCM and koji tags when a user wants to add a new package to Fedora? Any assistance you could provide would be greatly appreciated. Note: This is for an internal package build system for my company and will only be used as such. I am willing to share what I have done as the fedpkg will also be used to push the packages into our Spacewalk environment and also push those packages to systems within Spacewalk (with some custom code that I am adding to fedpkg and renaming that command to fit into our naming convention). Thank you for your time and have a great day! Regards, Jeremy

3 3

[Change Request] Fix pkgdb privileged operations for old python-bugzilla
by Toshio Kuratomi 24 May '12

24 May '12

As part of two situations I've pushed out a hotfix to pkgdb and would like two +1's for the change. The situations: * Continuing work to get our apps working with the old python-bugzilla package from yesterday's problems between infrastructure and bugzilla * Unknown causes of downtime for the fas servers. We're currently trying to determine what's causing the second situation to recur ~ every two hours. As part of that we're trying to fix anything that is out of the ordinary. So I've pushed this fix out and verified it works with the reporter. If I could get two +1's for after the fact application during change freeze, that would be great. The change: When PackageDB makes changes to who owns a package or creates a new package it needs to login to bugzilla and find out if the new owner has a bugzilla account. Those connections were failing for the same reason as fas's export-bugzilla was failing yesterday: the older version of python-bugzilla does not know how to get this information out of the new version of bugzilla. The fix is the same as yesterday's fix to export-bugzilla: for now, use the RHBugzilla3 compatibility API to get this information. https://fedorahosted.org/fedora-infrastructure/ticket/3304 diff --git a/modules/hotfix/files/pkgdb/utils.py b/modules/hotfix/files/pkgdb/utils.py index 021a80f..8c121c5 100644 --- a/modules/hotfix/files/pkgdb/utils.py +++ b/modules/hotfix/files/pkgdb/utils.py @@ -51,7 +51,7 @@ from cherrypy import request from sqlalchemy import select, and_ from sqlalchemy.exceptions import DataError -from bugzilla import Bugzilla +from bugzilla import RHBugzilla3 import memcache # The Fedora Account System Module @@ -218,9 +218,8 @@ def get_unprivileged_bz(): bz_server = config.get('bugzilla.queryurl', config.get('bugzilla.url', 'https://bugzilla.redhat.com')) bz_url = bz_server + '/xmlrpc.cgi' - _unpriv_bugzilla = Bugzilla(url=bz_url, cookiefile=None) + _unpriv_bugzilla = RHBugzilla3(url=bz_url, cookiefile=None) return _unpriv_bugzilla - def get_bz(): '''Retrieve a connection to bugzilla @@ -236,7 +235,7 @@ def get_bz(): bz_url = bz_server + '/xmlrpc.cgi' bz_user = config.get('bugzilla.user') bz_pass = config.get('bugzilla.password') - _bugzilla = Bugzilla(url=bz_url, user=bz_user, password=bz_pass, + _bugzilla = RHBugzilla3(url=bz_url, user=bz_user, password=bz_pass, cookiefile=None) return _bugzilla -Toshio

4 5

[PATCH] - add db-fas01 to dns - add to puppet - based in larger part off of db01's recipe
by Seth Vidal 24 May '12

24 May '12

From: Seth Vidal <skvidal(a)fedoraproject.org> --- manifests/nodes/db-fas01.phx2.fedoraproject.org | 18 ++++++++++++++++++ modules/bind/files/master/126.5.10.in-addr.arpa | 4 ++-- modules/bind/files/master/phx2.fedoraproject.org | 3 ++- 3 files changed, 22 insertions(+), 3 deletions(-) create mode 100644 manifests/nodes/db-fas01.phx2.fedoraproject.org diff --git a/manifests/nodes/db-fas01.phx2.fedoraproject.org b/manifests/nodes/db-fas01.phx2.fedoraproject.org new file mode 100644 index 0000000..ac90bf9 --- /dev/null +++ b/manifests/nodes/db-fas01.phx2.fedoraproject.org @@ -0,0 +1,18 @@ +node db01-fas.phx2.fedoraproject.org { + $bacula5 = "true" + # re-enable this once we have db's setup. + # $backup_db_often = true + include phx + include db + include appDB-el6 + collectd::collectd { 'log02': } + collectd::postgres { 'postgres': + databases => ['fas'] + } + + # Workaround for bug 768055 + semanage_fcontext { '/usr/lib64/nagios/plugins/check_disk': + type => 'nagios_unconfined_plugin_exec_t' + } + +} diff --git a/modules/bind/files/master/126.5.10.in-addr.arpa b/modules/bind/files/master/126.5.10.in-addr.arpa index 39e2b4d..9dbb2e2 100644 --- a/modules/bind/files/master/126.5.10.in-addr.arpa +++ b/modules/bind/files/master/126.5.10.in-addr.arpa @@ -2,7 +2,7 @@ $TTL 86400 @ IN SOA 126.5.10.in-addr.arpa. hostmaster.fedoraproject.org. ( ; Make SURE to update the serial # when changes are made. Format is YYYYMMDDNN ; (NN is just a counter to handle multiple changes in one day). - 2012031900 ; Serial + 2012052400 ; Serial 28800 ; Refresh 14400 ; Retry 1000000 ; Expire @@ -109,7 +109,7 @@ $TTL 86400 96 IN PTR download04.phx2.fedoraproject.org. 97 IN PTR download05.phx2.fedoraproject.org. 98 IN PTR darkserver01.phx2.fedoraproject.org. -99 IN PTR unused. +99 IN PTR db-fas01.phx2.fedoraproject.org. 100 IN PTR unused. 101 IN PTR unused. 102 IN PTR unused. diff --git a/modules/bind/files/master/phx2.fedoraproject.org b/modules/bind/files/master/phx2.fedoraproject.org index c0ebff5..707eae8 100644 --- a/modules/bind/files/master/phx2.fedoraproject.org +++ b/modules/bind/files/master/phx2.fedoraproject.org @@ -3,7 +3,7 @@ $TTL 3600 ; Make SURE to update the serial # when changes are made. Format is ; YYYYMMDDNN (NN is just a counter to handle multiple changes in one ; day). - 2012051001 ; Serial + 2012052401 ; Serial 3600 ; Refresh 600 ; Retry 1000000 ; Expire @@ -62,6 +62,7 @@ db5 IN CNAME db05 db-wiki IN CNAME db05 db-smolt IN CNAME db05 db-insight IN CNAME db05 +db-fas01 IN A 10.5.126.99 dhcp01 IN A 10.5.125.43 download01 IN A 10.5.126.93 download02 IN A 10.5.126.94 -- 1.7.2.1

3 3

Plan for tomorrow's Fedora Infrastructure meeting (2012-05-24 18UTC)
by Kevin Fenzi 24 May '12

24 May '12

The infrastructure team will be having it's weekly meeting tomorrow, 2012-05-24 at 18:00 UTC in #fedora-meeting on the freenode network. Suggested topics: #topic New folks introductions and Apprentice tasks. If any new folks want to give a quick one line bio or any apprentices would like to ask general questions, they can do so here. #topic two factor auth status #topic Fedora 17 Release tickets #topic Applications status / discussion Check in on status of our applications: pkgdb, fas, bodhi, koji, community, voting, tagger, packager, dpsearch, etc. If there's new releases, bugs we need to work around or things to note. #topic Upcoming Tasks/Items #info 2012-05-08 to 2012-05-29 FINAL FREEZE #info 2012-05-29 - F17 release #info 2012-06-01 - nag fi-apprentices. #info 2011-06-03 - gitweb-cache removal day. #info 2012-06-04 - class B reboots? #info 2012-06-05 - class A reboots? #info 2012-06-08 OOW: osuosl01.fedoraproject.org #info 2012-06-17 OOW: sign-vault02.phx2.fedoraproject.org #info 2012-06-21 to 2012-07-04 Kevin is off on trains and boats. #topic Meeting tagged tickets: https://fedorahosted.org/fedora-infrastructure/report/10 #topic Open Floor Submit your agenda items, as tickets in the trac instance and send a note replying to this thread. More info here: https://fedoraproject.org/wiki/Infrastructure/Meetings#Meetings Thanks kevin

1 1

Freeze Break: shutdown search01-dev on virthost05 - setup db-fas01 on virthost05
by Seth Vidal 24 May '12

24 May '12

This will be a first step to migrate to a private fas postgres instance. In order to free up mem and procs I would like to shutdown search01-dev on virthost05 and bring up db-fas01 on there. The goal would be a private postgres db server for fas alone. I will be kickstarting a vm on virthost05 and then adding entries to puppet/dns to accommodate this. I need some +1's to break the freeze to make this happen. thanks, -sv

3 2

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

infrastructure May 2012