I possibly fixed Redmine...

Matt Wagner matt.wagner at redhat.com
Mon Apr 9 14:01:59 UTC 2012


Howdy,

I meant to send this email out on Friday, but got distracted with other
tasks.

For my ongoing task to look into moving Redmine to OpenShift in order to
make it not unbearably slow[1], I logged into the Rails VM powering
Redmine today and tried to figure out what was so horribly broken. I had
in the past observed that it was using swap, but on Friday, it wasn't.
In fact, it had a lot of free RAM. So why in the world were page loads
taking so long?!

I _think_ I identified the problem and fixed it. I noticed that Google
was crawling our site, and that serving certain pages from the
repository browser were taking about 30 seconds to load. What was more
remarkable was that it would load the 30-second page, followed
immediately by a bunch of <200ms pages.

Then I hit a wiki page, which the logs claimed was taking <100ms to
process, and watched as it just sat in my browser, "Waiting for
www.aeolusproject.org." And then, tailing the log, I saw another repo
browser request finish processing, and right then, my page loaded.

This makes me think that the repo browser was somehow blocking thin. I
confess to not knowing the specifics of what does/doesn't block thin,
but my theory is that something it was doing to call out to disk as it
compared git repos was causing thin to block, causing other requests to
back up.

I tried throwing up a robots.txt to keep Google out of the repo browser,
but it didn't pick up the change immediately, and this happened to come
on the heels of me having questioned whether anyone actually used the
repo browser anyway. So I temporarily disabled the repo browser module.

Redmine feels fast now. It's still sometimes taking a couple of seconds,
but that's more in line with a normal website. I'm yet to see any pages
take an absurd amount of time to load, and I haven't sit a single Apache
timeout error.

This does somewhat undermine my push for us to move to OpenShift, which
was largely in response to our current solution being unacceptably slow.
I still think there's value, but I think the issue is somewhat (pardon
the pun) clouded now, since we did identify some downsides to switching
away from our current system. The pros ("Redmine is usable") really
outweighed the cons (IMHO), but now that Redmine is potentially usable
again, I think it's less clear-cut.

Let me know how things work, and if anyone actually misses the repo
browser. I suspect that GitHub presents a better browser, and that if
anyone actually wanted to use the ability to link Redmine issues to
commits, we could write a Redmine plugin that just received a
post-receive hook from GitHub:
http://help.github.com/post-receive-hooks/ as opposed to continually
mirroring the entire codebase and serving absurdly-slow blocking
pages.


-- Matt


[1]
https://www.aeolusproject.org/redmine/projects/aeolus/wiki/Moving_Redmine_to_OpenShift



More information about the aeolus-devel mailing list