On 11/02/10 - 10:27:11AM, Ladislav Martincik wrote:
On Nov 1, 2010, at 7:52 PM, Ian Main wrote:
On Mon, 2010-11-01 at 10:07 -0400, Chris Lalancette wrote:
On 11/01/10 - 02:32:02PM, Ladislav Martincik wrote:
Hi Chris,
just curious, why do you think that file observation is good solution?
I guess the question is, why do you think it is not? If you were to do something like pipes, that would just be a file anyway, so you wouldn't really get much efficiency gains.
Ok. Let's assume that file storage is a good solution. How do u assure that:
- When the file gets big it will not blow your memory consumption
Could be a problem, in theory. Disk storage is so unbelievably cheap, though, that I'm not inclined to worry too much about it. When we switch to QMF, this will go away.
- What if somebody deletes the file by accident or change its content (backup, logrotate)
We could handle this (though we don't today). Inotify can also tell you when your file goes away, so we could try to handle that intelligently. That being said, you have similar problems with almost anything; what happens if your network socket closes unexpectedly?
- What if you miss some messages? Fault tolerant.
We already handle this. We store the place in the file that we last parsed from, and on startup we look for messages we missed. Again, QMF will also handle this.
Now, I agree, we eventually want to switch to QMF; that will allow us to do it over the network, meaning we can have condor running on a different machine from the aggregator. But at the moment I don't really see a compelling need to move to anything but QMF.
I think the discussion should about Condor being the background job processor. IMHO choosing simple DelayedJob gem/plugin would make things much more simpler and faster to implement from beginning.
Having sufficient message queue would make things even much better.
I know you weren't here when we made this decision, but suffice it to say that removing condor and going with something else is absolutely not an option. Condor is known to scale well (to hundreds of thousands of jobs), and that is what we want to target for our scheduling.
I agree, there's nothing wrong with the file observation method we are using now. The format is good (XML) and we can use eg inotify or other mechanism (kqueue on bsd) to be notified of new data.
AFAIK QMF is still not prime time for condor, though I should poke in on that front again now as it's been some time.
Ian
My main goal for sub. of rb-inotify has been the fact that I do most of the dev. work on my MacBook and having to comment all the time rb-inotify to be able to run server is just painful.
Right. I'm fine with replacing rb-inotify with fssm, I just want to make sure we have the packages and it works before we do that.