Hey List,
Here's where i stand with EKG so far. According to my handy bean counter, i've spent a total of 14 hours on EKG so far. This puts me behind schedule just a bit. I think i predicted about 16 hours to redo the scanner, and another 4-8 for the grapher. (Max, that's an official 2 weeks, like we discussed.)
So here's what i have going on right now.
I've broken down what we're looking at into 'sources' which are places where things happen, and 'facts' which are actions someone undertook. Each fact has one source. Since the details of the data model aren't hammered out, i'm going to add getting sqlalchemy-migrate setup to my todo list.
So far there's an abstract base class called mailman, that handles getting gzipped mbox files of logs from various sources, and then there are some simple functions that break it down into facts in the database. The reason for this is that in order to write a new mailman for a new source, you simply need to feed it some URLs, and a regex or two it can use to parse the web pages that link to the mbox files. This idea can be extended to other ways of getting facts from other sources. Right now, since we're focused on mailing lists, it uses a few idioms related to that, which will have to be factored out at some point.
As for as the code base goes, it needs all the usual refinements, like using the config file i nicely set up, setting up a python egg so we can package and install it, a bit of refactoring some nasty bits, and probably performance help.
I'll have a stab at making sure we can parse pretty much every mailing list on the v2 settings list tomorrow morning.
-Yaakov