On Tue, Sep 20, 2011 at 05:09:55PM -0400, Chris Lalancette wrote:
All,
This is a patch series to remove condor from the conductor. In short,
condor presents problems for our project because it is an external project, it
is written in C++ (while most of our developers are ruby), and it is too complex
for our current needs.
The new way we do scheduling is described pretty well in patch 1, so I
won't delve into it here. Since the last posting, there have been a few more
bugs fixed:
1) jprovazn finally found an ActiveRecord incantation that works for
re-establishing a connection to a database, so the dbomatic subprocesses don't
trample over each other
2) I had to fix several bugs when running dbomatic under systemd. I can go
into the gory details if you want, just ping me on IRC :)
3) imain discovered that the broken Download Key links are independent of this
patchset, so we don't have to worry about that.
There are also some outstanding TODO items, but which can be postponed until
later:
1) We are still not properly generating the instance name. In condor, we did
this by generating a UUID that would be the instance name, then truncating that
UUID to the maximum length that deltacloud told us an instance name could be
for a particular backend. I'm thinking we want to do the same here.
2) imain believes we can do away with the Task class completely now. That
would be a nice cleanup
3) When launching large deployments, the "Launch Deployment" page can be slow
to return. That's because in the background we are issuing one deltacloud call
at a time to launch each instance in the deployment. We probably want to fork
off a separate process for each deployment so we don't hold up the UI process.
I've personally tested the patches using the EC2 backend, where things seem to
be working well. I was able to launch 5 deployments for a total of 15
instances simultaneously without issue. I also tested up to a 4 assembly
deployable which seemed to work.
At this point, I'm relatively happy with this patchset and I think we should
commit it barring any major problems found in testing. The enhancements above
can be done as additional patches on top of this.
Please review and ACK.
ACK'd it is. This is working well for me in my testing. Like Chris
says we have some follow up things to address but in general this is
working very well and is far more simple than using condor for this
task.
Thank you Chris and others for all your work on this.
Ian