Hello everyone,
Here are mongo-benchmark results for 1-brick volumes formatted with (1) ext4: http://file.brq.redhat.com/~eshishki/HekaFS/hekafs.ext4.html (2) xfs: http://file.brq.redhat.com/~eshishki/HekaFS/hekafs.xfs.html
Legend: A) "no crypt xlator": crypt and oplock xlators are not invoked. B) "crypt xlator with trivial cipher transform": crypt and oplock xlators are invoked, cipher transform is represented by empty operator. EOFs (end-of-files) are fully handled. C) "crypt xlator with AEC-CBC cipher transform": crypt and oplock xlators are invoked, cipher transform is represented by AEC_CBC_encrypt of OpenSSL library.
The link to mongo documentation can be found at the end of tables.
---------------------------------------------------------------
For the client/server configuration (1) above I have also measured performance of writing/reading a large file (800M):
dd writing: (dd if=/dev/zero of=/mnt/gluster/largefile bs=4K count=200000)
A) 178.57 s, 4.6 MB/s (no crypt xlator) B) 226.02 s, 3.6 MB/s (crypt xlator with trivial cipher transform) C) 254.02 s, 3.2 MB/s (crypt xlator with AES-CBC cipher transform)
B/A = 1.26 C/A = 1.42
dd reading: (dd if=/mnt/gluster/largefile of=/dev/null bs=4K)
A) 11.2394 s, 72.9 MB/s (no crypt xlator) B) 13.4892 s, 60.7 MB/s (crypt xlator with trivial cipher transform) C) 26.4549 s, 31.0 MB/s (crypt xlator with AES-CBC cipher transform)
B/A = 1.20 C/A = 2.36
RAW dd reading of ext4 large file on the server: (dd if=/root/exp2/edward/largefile of=/dev/null bs=4K)
9.69625 s, 84.5 MB/s
------------------------------------------------------------------
Server and client volfiles for (A) and (B) attached. HekaFS sources: git://git.fedorahosted.org/CloudFS.git (branch "aes").
Common comments: Current implementation of encryption support leads to performance drop from 1.28 till 2.36 times depending on operations (for bricks formatted with ext4). The most affected operation is reading a largefile. The least affected operation is creating a large set of small files.
Thanks, Edward.
Common comments: Current implementation of encryption support leads to performance drop from 1.28 till 2.36 times depending on operations (for bricks formatted with ext4). The most affected operation is reading a largefile. The least affected operation is creating a large set of small files.
Looked at another way, performance with encryption is 42-78% of performance without. That actually doesn't seem too bad, considering that some people need encryption regardless of the cost. Consider people who use Tahoe-LAFS, for example, despite the fact that it has absolutely *atrocious* performance characteristics. It would be interesting to see how these numbers scale up with more concurrent application threads, or as clients/servers are added. Since you don't mention a network type, am I correct in assuming that these numbers were all from client and server processes together on a single machine?
On 01/31/2012 04:29 PM, Jeff Darcy wrote:
Common comments: Current implementation of encryption support leads to performance drop from 1.28 till 2.36 times depending on operations (for bricks formatted with ext4). The most affected operation is reading a largefile. The least affected operation is creating a large set of small files.
Looked at another way, performance with encryption is 42-78% of performance without. That actually doesn't seem too bad, considering that some people need encryption regardless of the cost. Consider people who use Tahoe-LAFS, for example, despite the fact that it has absolutely *atrocious* performance characteristics. It would be interesting to see how these numbers scale up with more concurrent application threads, or as clients/servers are added. Since you don't mention a network type, am I correct in assuming that these numbers were all from client and server processes together on a single machine?
Just a note, it would be great to make sure that we do benchmark runs on XFS & RHEL6.2 as a base (on our new storage hardware :))
Ric
On 01/31/2012 10:29 PM, Jeff Darcy wrote:
Common comments: Current implementation of encryption support leads to performance drop from 1.28 till 2.36 times depending on operations (for bricks formatted with ext4). The most affected operation is reading a largefile. The least affected operation is creating a large set of small files.
Looked at another way, performance with encryption is 42-78% of performance without. That actually doesn't seem too bad, considering that some people need encryption regardless of the cost. Consider people who use Tahoe-LAFS, for example, despite the fact that it has absolutely *atrocious* performance characteristics. It would be interesting to see how these numbers scale up with more concurrent application threads, or as clients/servers are added. Since you don't mention a network type, am I correct in assuming that these numbers were all from client and server processes together on a single machine?
Client and server were different machines with Gigabit network connection. I have updated hardware info on the tables with the results:
client: ThinkPad x61, Intel(R) Core(TM)2 Duo CPU T7300 2.00GHz. Ethernet controller: Intel Corporation 82566MM Gigabit Network Connection
server: HP xw4600, Intel(R) Core(TM)2 Quad CPU Q9300 2.50GHz. Ethernet controller: Broadcom Corporation NetXtreme BCM5755 Gigabit Ethernet PCI Express
cloudfs-devel@lists.fedorahosted.org