Hey List,
for some odd reason, when i went to run scanner.py last night on max's settings file, i got about 8000 facts for a time period spanning several years. This seemed odd, but since i started it at around bed time expecting it to take all night, i didn't get a chance to debug it. My apologies max.
I'm gonna have a look at it in a bit.
-Yaakov
2009/3/25 Yaakov Nemoy loupgaroublond@gmail.com:
Hey List,
for some odd reason, when i went to run scanner.py last night on max's settings file, i got about 8000 facts for a time period spanning several years. This seemed odd, but since i started it at around bed time expecting it to take all night, i didn't get a chance to debug it. My apologies max.
I'm gonna have a look at it in a bit.
-Yaakov
Running the following query on the data i've generated:
select facts.*, sources.cache_file from facts, sources, streams where facts.source_id = sources.id and sources.stream_id = streams.id and streams.name = 'fedora-devel-list'
Shows that sources.cache_file (namely <year>-<month>.txt.gz) often only has one entry for some months, which was the original problem i had reported earlier with RH's mailman.
I have a few ideas what we can do.
1) Complain bitterly to RH's tech support about this. 2) Get permission to do an ssh pull of the archives, i can write a subclass of Source to handle this 3) Write an instance of Source todo what the version 2 was doing, namely pulling stuff from websites. 4) Other...
Any particular course of action that would be best?
-Yaakov
On Wed, 25 Mar 2009, Yaakov Nemoy wrote:
select facts.*, sources.cache_file from facts, sources, streams where facts.source_id = sources.id and sources.stream_id = streams.id and streams.name = 'fedora-devel-list'
Shows that sources.cache_file (namely <year>-<month>.txt.gz) often only has one entry for some months, which was the original problem i had reported earlier with RH's mailman.
Does that mean the archives are incomplete, or that they are formatted in some broken way? What exactly is the problem? I'm sorry if you wrote this before and I don't remember.
I have a few ideas what we can do.
- Complain bitterly to RH's tech support about this.
I'd file a ticket with helpdesk@redhat.com for sure, if you can explain the problem and what you think the fix is, that would really help.
--Max