Hey guys, I wasn't able to get mash to run locally so I never got profiling of it done this vacation. But I did see several obvious ways to optimize the multilib code. So here's a patch that will probably help make that piece of the puzzle faster.
There's a few conceptual changes:
1) Only create the lists of packages and files once, when we create the class rather than every time we instantiate it (or worse, everytime we run the select() method which is what it was doing before). 2) Use frozensets() instead of lists wherever we're doing a containment test ('string' in set_of_packages). Sets are a hashed lookup whereas lists have to be searched linearly. 3) Reduce the calls to fnmatch. fnmatch uses a regex under the hood so it's not the cheapest operation in the world (although the regex compilation is cached so it's not the worst thing either). I was able to change a few fnmatches into containment tests instead and others I was able to put behind a cheaper conditional which skips those tests altogether if the cheaper conditional is satisfied
Without profiling a mash run I don't know how much this will speed up mashing but talking to people it seemed like the multilib portion was taking 30-45 minutes to complete and since this was all low hanging fruit it seemed like a good place to start.
-Toshio