On 12/25/2011 05:25 AM, Dennis Gilmore wrote:
When we were testing build were happening really fast, once we
up the build jobs things have become really slow
at 1:19 utc and at 5:18 utc four hours later the buildrequires are
still being installed. australia is completely io bound. i think we
need to see how we can spread the io load. ~30 hosts reading and
writing to the 4 spindles just saturates the disk io.
Is your RAID aligned properly? I have found that making sure that your
RAID and FS are aligned properly makes a big difference to performance,
at least on my workloads. Have a look here:
Just taking the default options when setting up the RAID and FS
typically hammer the IOPS performance down to that of a single disk.
i guess options
are find a way to add more spindles. move /var/lib/mock to sdcard, see
if we can get some kind of san that has alot of smaller fast disks.
That will make it an order of magnitude worse. A typical class 10 SD
card only manages about 20 random write IOPS.
some 2.5" usb drives one for each builder. some other idea? is there
anyway we could add 4-8 more disks to australia. the size of the
matters little. gaining more iops by adding more spindles would help.
You could just replace the disks with some decent SSDs and be done with
it (aligning the RAID and FS as explained on the above linked page still
applies just the same). Especially since 15K rpm disks are now highly
uncompetitively priced against SSDs.