On Wed, Jun 30, 2010 at 12:11 AM, Ian Weller <ian(a)ianweller.org> wrote:
This proposal comes after talking with Mike McGrath of
Infrastructure
and also with Max.
The issue that Mike brought up (that I think Nathaniel also brought up)
was that we would be heavily duplicating data. (We have just over 2 GB
of bzip2'd data for the month of May 2010 alone, and put in a database
with proper indexing would make querying fast, but would offer no
additional benefits than just downloading the logs directly and would at
least double, if not triple the storage we dedicate to website logs.)
So Max and I came up with a different way of how datanommer should work
based on my reunderstanding of his goals and the caveats I got from
Infrastructure. Instead of an application that simply collects log data,
I (we) should write an application that uses already-existing APIs to
download data from applications commonly used within free software
communities. It should not be focused on fetching data, but should
instead be focused on creating presentations around the data (read:
generating charts, to start with).
I already have some code written up and pushed to our git repo. The
datanommer runnable is hardcoded right now since I haven't bothered
creating a configuration file structure yet, but you can look at the
backend and see how flexible it's planned to be. Code is at:
http://git.fedorahosted.org/git/?p=datanommer.git;a=summary
(It may be cached to not have any commits listed yet still. The URLs
work just fine though.)
If you run bin/datanommer, you get a graph that looks like this
(slightly dated data):
http://ianweller.fedorapeople.org/test.png
Thoughts, everybody?
Yeah what I was trying to do is quite different from this.
Probably not interested in helping but I'd like to see where you take it.
--
Ian Weller <ian(a)ianweller.org>
() ascii ribbon campaign - against html e-mail
/\ www.asciiribbon.org - against proprietary attachments
_______________________________________________
datanommer mailing list
datanommer(a)lists.fedorahosted.org
https://fedorahosted.org/mailman/listinfo/datanommer