Hey guys, I wasn't able to get mash to run locally so I never got profiling
of it done this vacation. But I did see several obvious ways to optimize
the multilib code. So here's a patch that will probably help make that
piece of the puzzle faster.
There's a few conceptual changes:
1) Only create the lists of packages and files once, when we create the
class rather than every time we instantiate it (or worse, everytime we
run the select() method which is what it was doing before).
2) Use frozensets() instead of lists wherever we're doing a containment test
('string' in set_of_packages). Sets are a hashed lookup whereas lists
have to be searched linearly.
3) Reduce the calls to fnmatch. fnmatch uses a regex under the hood so it's
not the cheapest operation in the world (although the regex compilation
is cached so it's not the worst thing either). I was able to change
a few fnmatches into containment tests instead and others I was able to
put behind a cheaper conditional which skips those tests altogether if
the cheaper conditional is satisfied
Without profiling a mash run I don't know how much this will speed up
mashing but talking to people it seemed like the multilib portion was taking
30-45 minutes to complete and since this was all low hanging fruit it seemed
like a good place to start.
-Toshio
Show replies by date