This is a bit further reaching than what we discussed at FOSDEM. My comments as follows.
2009/2/11 Max Spevack <mspevack(a)redhat.com>:
Let's get away from the language debate, and discuss the design
and
architecture of the code. Maybe the language falls out of that
discussion.
=======
Ignoring what currently is implemented in v1 and v2, here is what I care
about:
I can airlift the design from v2 into this, without any difficulty.
(1) Pulling data down from wherever *exactly once* and storing it in
a
database forever after that.
That's goal #1 for today.
(2) The primary feature of the config file is a "list of
projects",
which represents the projects that EKG will analyze. For each project
in the list, the pieces of data that are collected will probably expand
over time, so it's important to take this into account when we
architect. Maybe we're only starting with mailing lists, but we'll
eventually expand to commits, maybe to bugs, etc. Basically I want the
program to think of something like Spacewalk not just as the spacewalk
mailing list(s), but the various other sources of data that exist.
I have a few ideas on how to do this. I'm gonna use a few ideas i figured out in
Devshell. (When all you have is a hammer, everything looks like a nail.)
I can create classes that handle certain kinds of data streams, namely, mailing lists,
bugzilla, trac, wikis, etc... Then each project is a composite object of these classes.
Finally, we can find a way to serialize an object to a config file like file, that can be
used to combine pieces like this, without requiring anything complicated.
(3) Having an "update my data" feature that basically looks
at the list
of projects that EKG cares about, and can say "well, my data on
Spacewalk hasn't been updated since January 12th, so let me go get all
the new data" or "oVirt has just been added to the list, so let me go
and get everything that exists".
I'm not 100% sure about the best way to go about this, but it doesn't seem
difficult.
(4) Being able to analyze the data in time slices. Monthly,
quarterly,
and yearly are what interests me most. I'd like to be able to get that
either from a calendar point of view (year starts on January 1) or a Red
Hat fiscal year point of view (year starts on March 1).
We talked about this. This is going to be the #2 feature, after the #1 bit.
(5) Some use cases:
* There are 100 projects in the database. Show me the report of the
last quarter for all of them.
* Here is a list of 10 projects that are most important to Brian
Stevens. Show me the breakdown for the last two years, with quarterly
deltas.
* Show me all projects where the external mailing list percentage is
greater than 50%.
* Show me all the projects where @redhat.com is committing more than 75%
of the code.
* Show me the projects where the total volume of mailing list traffic
has increased/decreased by more than 25% since the last quarter.
This sounds like business reporting. It's that wishlist bit that goes beyond a simple
prototype and simple working solution. Some of these use cases should be pretty simple to
do. Some of them may require more work.
-Yaakov