On Mon, Apr 20, 2015 at 12:03:20PM +0200, Adrian Reber wrote:
On Fri, Apr 17, 2015 at 09:05:03AM -0600, Kevin Fenzi wrote:
> On Fri, 17 Apr 2015 17:01:45 +0200
> Pierre-Yves Chibon <pingou(a)pingoured.fr> wrote:
>
> > So it seems the last update of MM2 has fixed the issue I was seeing
> > from time to time of the application becoming unresponsive due to DB
> > hanging.
> >
> > So here is the current status as far as I see it:
> >
> > * Mirrorlist -> In prod and apparently fine
> > * MM2 UI -> Running in stg and seems all good to me
> > * Crawler -> Adrian Reber has put some great work on this, afaics he
> > seems to have fixed the biggest issues but I'll let him confirm if
> > everything works now or not
> > (@Adrian, I know there are features you want to make to the crawler
> > but let's get it working as it is for 1.0 and improve it
> > afterward ;-))
I would also say, that the crawler is ready for production. There is a
one line change I would like to get into an upcoming release, but it is
not really required at this point. PR will come today.
> > * Pickle generation -> code wise I believe it works the question is:
> > does it generate the same pickle as MM1
> > ! Crons: iirc there are still a couple of crons that are not running
> > properly and returning errors
Which ones?
From this night:
----
Subject: Cron <root@mm-crawler01> /usr/bin/mm2_crawler --threads 50 > /dev/null
2>&1
/bin/sh: line 1: 18523 Killed /usr/bin/mm2_crawler --threads 50
/dev/null 2>&1
----
Subject: Cron <root@mm-backend01> cd /usr/share/mirrormanager2 &&
/usr/bin/mm2_get_internet2_netblocks
/var/lib/mirrormanager/internet2_netblocks.txt
gzip: /tmp/get_i2_netblocks.RS1rA07F/rib.gz: unexpected end of file
short file (empty packet) at zebra-dump-parser/zebra-dump-parser.pl line 98.
gzip: /tmp/get_i2_netblocks.RS1rA07F/rib.gz: unexpected end of file
short file (empty packet) at zebra-dump-parser/zebra-dump-parser.pl line 98.
gzip: /tmp/get_i2_netblocks.RS1rA07F/rib.gz: unexpected end of file
short file (empty packet) at zebra-dump-parser/zebra-dump-parser.pl line 98.
gzip: /tmp/get_i2_netblocks.RS1rA07F/rib.gz: unexpected end of file
short file (empty packet) at zebra-dump-parser/zebra-dump-parser.pl line 98.
gzip: /tmp/get_i2_netblocks.RS1rA07F/rib.gz: unexpected end of file
short file (empty packet) at zebra-dump-parser/zebra-dump-parser.pl line 98.
gzip: /tmp/get_i2_netblocks.RS1rA07F/rib.gz: unexpected end of file
short file (empty packet) at zebra-dump-parser/zebra-dump-parser.pl line 98.
gzip: /tmp/get_i2_netblocks.RS1rA07F/rib.gz: unexpected end of file
short file (empty packet) at zebra-dump-parser/zebra-dump-parser.pl line 98.
gzip: /tmp/get_i2_netblocks.RS1rA07F/rib.gz: unexpected end of file
short file (empty packet) at zebra-dump-parser/zebra-dump-parser.pl line 98.
gzip: /tmp/get_i2_netblocks.RS1rA07F/rib.gz: unexpected end of file
short file (empty packet) at zebra-dump-parser/zebra-dump-parser.pl line 98.
unable to retrieve netblock list.
----
These are the only two I can see right now.
> So, that would be another, higher level test:
>
> Make staging scan directories for latest content.
> Make staging make a pkl based on that.
> Check it against the prod one
I am not sure if the resulting pickles can be that easily compared as
there have been changes all around MM2 which might return slightly
different results which are nevertheless both valid results.
I don't think we changed the pickles themselves, for example mirrorlist barely
changed and is currently working with MM1's pickle.
Pierre