I've posted some new encryption code to the "aes" branch of the git repo.
http://git.fedorahosted.org/git/?p=CloudFS.git;a=summary
This isn't intended to be a final authoritative anything, or to displace
any of the things Edward is doing. It's just a demonstration that the
CTR-mode approach can be made to work. Besides avoiding the
read-modify-write and contention issues of the previous approach, it's
much simpler and seems much faster (though I haven't done complete
performance testing yet). It supports AES-128, AES-192, or AES-256,
with the key given as a text string, hex string, or path to a
hex-encoded file. The basic mechanism sort of combines CTR mode and
ESSIV, and thus shares a couple of important weaknesses.
(1) Like all such methods, the result is vulnerable to a known-plaintext
attack. If someone knows both the plaintext and ciphertext for a range
of a file then they can easily derive the keystream for that range and
use it to view that range for the rest of the file's lifetime even if
the contents change.
(2) More seriously, CTR mode is vulnerable to tampering. Without even
being able to read the file contents, the storage provider can flip any
bit in the plaintext merely by flipping that bit in the ciphertext.
Note that neither of these vulnerabilities should be exploitable by
anyone but the storage provider; they both require access to the
ciphertext, and the in-flight encryption (separate from this) should
prevent anyone except them and the user from having access to that. I
think that's sufficient for many environments, especially in a private
cloud where the storage provider shares organizational accountability
with the tenants. If it's not, then the only way to solve (1) is to
store per-file keys in a separate distributed store that's secure enough
for that purpose and make the result fast/reliable enough for general
use. For (2) I think we can reduce a "flip one specific bit" problem to
a "flip some random bit in the same block" problem pretty easily, but
for a real solution we need to detect the tampering and reject the
tampered result. That would require storing checksums or similar with
similar constraints to the per-file keys for (1). This puts us into the
same problem space that Tahoe-LAFS tries to address, and despite many
more people working on that for many more years they have yet to come up
with a solution that's anywhere near adequate for primary storage. I
propose that we consider only small incremental improvements for now,
and defer the ultra-secure stuff until version two at least.
So, let me know what you think, either of the code or the commentary,
and we'll go from there.