Circling back here...
I reduced the density of builders. There's now 8 builders per host instead of 10. This means that there's now only 32 ppc64le builders instead of 40. However, everything seems a good deal more performant and stable now. So I think the tradeoff is worthwhile.
If folks who were seeing issues could keep an eye out now that would be appreciated. If it's better/worse/thesame would be useful feedback.
Thanks,
kevin