Yaakov Nemoy napisal(a):
Hi Yaakov,
While we want to look at 'heros' at some point, the best
thing we
might want are the following stats:
* for the entire membership, which domains are they coming from?
...which is implemented already.
* for these categories of membership (core, active, passive) which
domains are they coming from?
OK. It was mentioned in my first mail in this thread:
- who is the core member of the community (by e-mail domain)?
- who is a regular member of the community (by e-mail domain)?
- who is a temporary (passive) member of the community
(by e-mail domain)
and I agree that it's a good way of showing the stats later on.
* which domain is crossing the category barrier, namely, are more
jboss people talking to passive members or are they mainly talking to
users using gmail addresses.
OK.
All of these needs to be looked at temporaly too. You may write the
algorithm that calculates all these for a particular slice. We'll then
either feed it the entire list, or also segments of the list divided
up by convenient time frames. We need to see the big picture, but also
how does it change.
Indeed. The algorithm should only need formatted input. No matter if
the input is weekly or yearly.
> In case of weekly/monthly/quarterly slices show the dynamics:
> - are the core/regular/temporary groups changing?
> Are the numbers changing? Are the e-mail domains changing?
> Is the density in groups changing? Present these results
> on line charts.
This can be done two ways. I put in a concept of a general purpose
delta calculator, so we can pass any theoretical data and get the
deltas between time segments. Line charts and other graphs can
visualize this data for us cleanly as well.
That's exactly how I see it. I think that the main barrier here
is my language :)
>
> II. Basic data structure
>
> Connecting users on the basis of the structure of replies has two
> dimensions:
> - number of connections to other users
> - number of mails sent by each user to any other user
I think this part of the analysis is flawed. Details inline.
...
We can't determine easily if an email is sent to a particular
user. We
can only look at replies as a metric for that.
Right, that's what I had in mind. The problem it's rather a matter
of unprecise language than the concept itself.
> I consider filling such a data structure with actual data as
one
> of the mile stones of my project. But, let's assume that it is filled,
> so we can continue our thought experiment.
I would also like to, and i think just outright prefer, to see this as
an aggregate of domains. Namely, rather than emailX and emailY, let's
see domainX and domainY.
I agree that showing such stats for domains is interesting, but I think
that the set of domains is not large enough to make it a good input
for the algorithm which aims at distinguising core/regular/temporary
groups.
I would rather say: run the algorithm against single users and then
generalise for domains.
In order to do this, we would need to save the in-reply-to fields of
the email. This is not terribly difficult, just a pointer. I think the
tricker thing is to reconstruct that tree out of the database.
Exactly. Just wanted to ask you if there are any additional fields
that we need to add to the database. I am not in too deep into the
mail threading problems but I think that there are some good libraries
out there, so it shouldn't be a big problem as long as we have
corresponding fields in the database.
You may also want to look at the following scenarios.
Core member posts, regular replies, core member replies in return.
In such case there are two incrementations going on:
- for 'Core member posts, regular replies': CRS+=1
- for 'regular replies, core member replies in return': CRS+=1
core - regular relation may be reversed as well: regular - core.
Single counter does the job here.
> A bit of social networks analysis. Show a map of connections
between
> users including the information on how strong the ties between them
> are (based on the number of mails exchanged).
Would be nice, but i would discourage you from making this a required
element of the proposal application.
OK :)
I also think that this is very well thought out, and i am looking
forward to your application.
Thanks, the application should arrive soon :)
--
Regards,
Wojtek Walczak,
http://tosh.pl/gminick/