Dear Conductor developers,
Tired of running endless rspec and cucumber tests and having to wait
almost forever for them to finnish?
Here is (maybe) the solution! :)
As you know, RSpec tests are very database heavy, so it probably does
make sense to put postgresql data directory into a ramdisk-based filesystem.
The same applies to cucumber tests, but there is a difference that
cucumber is much more solr-heavy than db-heavy.
Let's then put a solr data directory into a tmpfs and see what happens.
Here are some "benchmarks" for you to have a picture of what's going on.
RSpec:
With pgsql on tmpfs:
$ rake
Finished in 79.749916 seconds
With pgsql on ext4 (and luks):
$ rake
Finished in 470.17935 seconds
So, rspec is on tmpfs about 489.5 percent faster.
Cucumber:
With solr/data on tmpfs and pgsql on ext4
$ rake cucumber
1m47.025s
With solr/data on tmpfs and pgsql on tmpfs
$ rake cucumber
1m44.647s
This is not a significant improvement compared to last result.
With solr/data on ext4 and pgsql on tmpfs
$ rake cucumber
5m18.378s
Well, it seems that cucumber is really solr-heavy.
And for completness, here are tests with both solr and pgsql on ext4,
which is probably the most common setup:
$ rake cucumber
5m24.583s
Well, it looks like the solr on tmpfs can provide ~156.5 percent speed
improvement.
There are also scripts for automated setup. Please note that these
scripts should be run from cucumber/src directory.
Please, give it a shot and also any feedback would be really appreciated.
Best regards,
Maros Zatko
mzatko(a)redhat.com
Show replies by date
#!/bin/bash -x
if ! mount | grep /var/lib/pgsql; then
/sbin/service postgresql stop
mount -t tmpfs tmpfs /var/lib/pgsql
chown postgres:postgres /var/lib/pgsql
chmod 700 /var/lib/pgsql
/sbin/service postgresql initdb
cat > /var/lib/pgsql/data/pg_hba.conf <<EOL
local all all trust
host all all 127.0.0.1/32 trust
host all all ::1/128 ident
EOL
/sbin/service postgresql start
su - postgres -c "echo -e \"CREATE USER dcloud WITH PASSWORD
'v23zj59an' CREATEDB;\n\\q\" | psql"
rake dc:setup
else
echo Postgresql is already on tmpfs
fi
#!/bin/bash -x
if ! mount | grep solr/data ; then
rake sunspot:solr:stop RAILS_ENV=cucumber
rake sunspot:solr:stop
sudo mount -t tmpfs tmpfs solr/data
rake sunspot:solr:start
rake sunspot:solr:start RAILS_ENV=cucumber
rake sunspot:reindex
else
echo Solr/data is already on tmpfs
fi
#!/bin/bash -x
if mount | grep /var/lib/pgsql; then
/sbin/service postgresql stop
umount /var/lib/pgsql
/sbin/service postgresql start
else
echo Postgresql is not on tmpfs
fi
#!/bin/bash -x
if mount | grep solr/data ; then
rake sunspot:solr:stop RAILS_ENV=cucumber
rake sunspot:solr:stop
sudo umount solr/data
rake sunspot:solr:start
rake sunspot:solr:start RAILS_ENV=cucumber
rake sunspot:reindex
else
echo Solr/data is not on tmpfs
fi