Trac logs..
by Mike McGrath
So our trac syncing has been failing lately. After some debugging I
discovered that A) some projects are logging to projects/*/log/trac.log
and B) we're doing so with debug enabled. Some of these files are HUGE.
Pungi's is over 1.1G. So here's what I'd like to propose.
1) Remove all of those log files.
2) Disable logging (only a small handful have enabled it)
3) Set the default log level on all future and current projects to ERROR
instead of DEBUG
4) After the freeze look at how to properly log rotate these files.
Hosted1 is currently frozen but seeing as how this is impacting backups,
can I get two +1's ?
-Mike
14 years
[PATCH] Ensure that rsync backups are only running one at a time
by Mike McGrath
---
configs/db/backup-dbs | 2 ++
1 files changed, 2 insertions(+), 0 deletions(-)
diff --git a/configs/db/backup-dbs b/configs/db/backup-dbs
index 83ed6c8..7af66fb 100755
--- a/configs/db/backup-dbs
+++ b/configs/db/backup-dbs
@@ -20,6 +20,8 @@ mv $DEST/$HOSTNAME.new $DEST/$HOSTNAME
# Sync out
for host in db01 db02 db03; do
+ # Sleep if any other rsyncs are already
+ while ssh $host "pgrep rsync" | grep -q [0-9]; do sleep 10; done
if [ "$host" != $HOSTNAME ]; then
su - dbbackup -c "ssh $host mkdir -p $DEST/$HOSTNAME"
su - dbbackup -c "rsync -azr --bwlimit=5000 -e ssh $DEST/$HOSTNAME/* $host:$DEST/$HOSTNAME/"
--
1.6.6.1
14 years
Change request: smolt_new db
by Mike McGrath
I forgot to cleanup after myself on db1 a few weeks back. as a result
there's a smolt_new database that doesn't need to be there.
2 +1's to drop it?
-Mike
14 years
[PATCH] rate limiting rsync copies
by Mike McGrath
This is causing disk contention without a limit
---
configs/db/backup-dbs | 2 +-
1 files changed, 1 insertions(+), 1 deletions(-)
diff --git a/configs/db/backup-dbs b/configs/db/backup-dbs
index a3a12fa..83ed6c8 100755
--- a/configs/db/backup-dbs
+++ b/configs/db/backup-dbs
@@ -22,7 +22,7 @@ mv $DEST/$HOSTNAME.new $DEST/$HOSTNAME
for host in db01 db02 db03; do
if [ "$host" != $HOSTNAME ]; then
su - dbbackup -c "ssh $host mkdir -p $DEST/$HOSTNAME"
- su - dbbackup -c "rsync -azr -e ssh $DEST/$HOSTNAME/* $host:$DEST/$HOSTNAME/"
+ su - dbbackup -c "rsync -azr --bwlimit=5000 -e ssh $DEST/$HOSTNAME/* $host:$DEST/$HOSTNAME/"
fi
done
--
1.5.5.6
14 years
Freeze tomorrow
by Mike McGrath
Just a reminder... we freeze tomorrow!
-Mike
14 years, 1 month
Possibility of Fedora sponsoring a Freenode server
by Nick Bebout
What is everyone's thoughts about us sponsoring a Freenode server? Their
general requirements are noted on http://freenode.net/hosting_ircd.shtml
and I've discussed them with mmcgrath and he has said they seem doable,
probably on telia1 or osuosl1. Basically we'd be providing them a xen
guest with about 1.5 or 2 GB ram, client servers should expect to sustain
around 1Mbps of traffic, with much higher peaks. Total usage for client
servers is usually around 320 gigabytes per month, while hidden hubs are
generally much lower overall.
We'd be giving them at least a user account on it, they prefer sudo if
possible. They handle the day-to-day administering of the ircd, either
scp'ing new binaries or compiling the ircd on the server itself, and
copying new config files to there.
I've talked to their sponsor liaison and they seem to be interested. Mike
and I wanted to discuss it on here before I send the actual "proposal"
email.
I think it would be a good idea, especially since we use them so much for
our IRC channels, it'd be nice to contribute a server back to the network.
Nick
14 years, 1 month
Re: main problems of fedora?
by Jose Manimala
On 3/18/2010 12:10 AM, Mike McGrath wrote:
> On Wed, 17 Mar 2010, Gergely Buday wrote:
>
>> Hi there,
>>
>> is there a list of known deficiencies of fedora - I mean those which
>> can be cured by developing new software, rather than fixing bugs? Or,
>> if such list does not exist, can you name a few missing item from
>> fedora? Think of relatively small would-be software, without a GUI.
>>
>
> One of the new initatives in Fedora is called Fedora Engineering Services
> - http://fedoraproject.org/wiki/Fedora_Engineering_Services
>
> If you're a fast learner who just wants to work on various technical and
> other issues but don't care what, sign up. We can always use more help.
>
> -Mike
Wow... this is the kind of work I was looking for... Can I sign up?
--
Kind Regards
Jose M Manimala
http://josemanimala.eu.org
Ph: +919884065040
GPGkeyID: 1BE49F39
14 years, 1 month
[PATCH] expose MM crawler logs on bapp01
by Matt Domsch
My goal here is to expose the mirrormanager crawler logs, which are on
bapp01:/var/log/mirrormanager/crawler/
via the web URL:
http://admin.fedoraproject.org/mirrormanager/crawler/
Does this look right to do that? I can't easily test this in staging,
so want an additional set of eyes.
Thanks,
Matt
>From d6f45bf2fa9165712f9ce9032050c51c68e75db0 Mon Sep 17 00:00:00 2001
From: Matt Domsch <mdomsch(a)puppet01.phx2.fedoraproject.org>
Date: Wed, 17 Mar 2010 15:47:17 +0000
Subject: [PATCH] MM: expose crawler logs on bapp01
---
manifests/servergroups/proxy.pp | 6 ++++++
modules/mirrormanager/manifests/init.pp | 16 ++++++++++++++++
.../templates/mirrormanager-crawlerlogs.conf.erb | 4 ++++
3 files changed, 26 insertions(+), 0 deletions(-)
create mode 100644 modules/mirrormanager/templates/mirrormanager-crawlerlogs.conf.erb
diff --git a/manifests/servergroups/proxy.pp b/manifests/servergroups/proxy.pp
index 33f83f4..8efbd45 100644
--- a/manifests/servergroups/proxy.pp
+++ b/manifests/servergroups/proxy.pp
@@ -679,6 +679,12 @@ class proxy {
proxyurl => "http://localhost:10008",
}
+ mirrormanager::proxy::crawlerlogs { "admin.fedoraproject.org/mirrormanager":
+ website => "admin.fedoraproject.org",
+ path => "/mirrormanager/crawler",
+ proxyurl => "http://bapp01",
+ }
+
mirrormanager::proxy::mirrorlist { "mirrors.fedoraproject.org":
website => "mirrors.fedoraproject.org",
proxyurl => "http://localhost:10002",
diff --git a/modules/mirrormanager/manifests/init.pp b/modules/mirrormanager/manifests/init.pp
index 4666231..979170a 100644
--- a/modules/mirrormanager/manifests/init.pp
+++ b/modules/mirrormanager/manifests/init.pp
@@ -278,6 +278,22 @@ define mirrormanager::proxy::mirrorlist(
}
}
+define mirrormanager::proxy::crawlerlogs(
+ $website,
+ $path,
+ $proxyurl
+) {
+ include httpd::base
+
+ file { "/etc/httpd/conf.d/$website/mirrormanager-crawlerlogs.conf":
+ owner => "root",
+ group => "root",
+ mode => 0644,
+ content => template("mirrormanager/mirrormanager-crawlerlogs.conf.erb"),
+ notify => Service["httpd"],
+ require => Httpd::Website[$website],
+ }
+}
class mirrormanager::sync {
package { 'mirrormanager-client': ensure => present }
diff --git a/modules/mirrormanager/templates/mirrormanager-crawlerlogs.conf.erb b/modules/mirrormanager/templates/mirrormanager-crawlerlogs.conf.erb
new file mode 100644
index 0000000..e9b8769
--- /dev/null
+++ b/modules/mirrormanager/templates/mirrormanager-crawlerlogs.conf.erb
@@ -0,0 +1,4 @@
+RewriteEngine On
+
+RewriteRule ^<%= path %>(.+) <%= proxyurl %><%= path %>$1 [P,QSA,L]
+ProxyPassReverse / <%= proxyurl %>/
--
1.5.5.6
--
Matt Domsch
Technology Strategist, Dell Office of the CTO
linux.dell.com & www.dell.com/linux
14 years, 1 month