[guardian-dev] IOCipher Status Update

[Hans-Christoph Steiner]

Hey Stephen,

I'm cc'ing guardian-dev to have this discussion continue there.

Thanks, this is great.  I've already integrated your libsqlfs changes with
some cleanups and pushed them to the official repo.  I also made a
'c_perf_key' test that uses the encryption just to see the speed differences.

I'm integrating the IOCipher patch now, did you have any tests written up for
that already?  I renamed begin() and complete() to beginTransaction() and
completeTransaction() since the words 'begin' and 'complete' seemed too vague
in the context of the VirtualFileSystem class.  We'll work to document that so
people can make good use of it.

It would great if you could spend the remaining time (~10 hours) working on
improving the lock situation so that we can have concurrent reads, or maybe
even SQLite3 3.7's WAL, if that's not too ambitious.  This chunk is probably
more complicated than the write work you just did, so it might just be that
you end up giving us direction for how to implement it.  FYI, I added read
speed tests to tests/c_perf.c and they are already a lot faster than writes,
looks like they take about 40-50% of the time of a similarly arranged write.

In semi-related news, I got our secure jenkins instance to automatically run
valgrind on all of the tests, so hopefully that'll keep us on the ball in
terms of crashes and buffer overflows.

.hc

On 01/15/2013 06:14 PM, Stephen Lombardo wrote:
> Hi Everyone,
> 
> I just wanted to summarize where we are with our IOCipher work so far. Nick and I started to dig into the performance issues with IOCipher and have identified three areas for immediate improvement.
> 
> 1. When issuing writes into libsqlfs, there can be a significant overhead due to repeatedly opening and committing transactions[1]. If multiple sequential write calls are wrapped in a single transaction, it is possible to increase write performance considerably (2x - 10x improvement).
> 
> 2. The use of a large sqlfs block size (128K) decreases performance for small writes considerably because the full 128K block must be pulled from the database, modified, and then re-written on each update. In addition, the use of the default SQLCipher page size (1K) adds extra overhead for encryption / decryption operations when large blocks of data must be updated. We have found that simultaneously decreasing the sqlfs block size and increasing the SQLCipher page size can substantially increase performance.
> 
> 3. In general, using a BufferedOutputStream improves write performance by grouping small writes together into larger writes. In particular, using a BufferedOutputStream with a buffer size aligned with the sqlfs block size of 8K provides the best overall performance.
> 
> Based on these findings, we would recommend the following changes:
> 
> 1. That the block size and page size be set to 8K in the libsqlfs library used with IOCipher[2]. That should provide an across-the-board improvement in performance for small-to-medium sized writes, balancing performance with large writes.
> 
> 2. Provide guidance to developers to use BufferedOutputStream instead of the raw FileOutputStream when performing write operations. It may even be desirable to create an IOCipher-specific subclass of java.io.BufferedOutputStream that adjusts the default constructor to use an 8K buffer for optimal performance.
> 
> 3. Expose the sqlfs begin / complete functions to give developers an additional option to wrap file operations in a transaction when doing large numbers of write operations[3], i.e.
>    
>     vfs.begin();
>     // performance sensitive write operations here
>     vfs.complete();
> 
> In conjunction with the previous changes this would provide the fastest possible performance when blocking transactions are not a concern.
> 
> Please let us know if you have any comments or questions on these suggestions.
> 
> From a timing standpoint, we've expended about 15 hours on testing and performance tuning. Would you like us to continue working on any other issues at this time?
> 
> Thanks!
> 
> Cheers,
> Stephen
> 
> [1] https://github.com/sjlombardo/libsqlfs/commit/1590037700ac8916b606233161d4232206603607
> [2] https://github.com/sjlombardo/libsqlfs/commit/d8ca435c8d1674e6b3f65d8a796325aae66bfaa8
> [3] https://github.com/sjlombardo/IOCipher/commit/22e518067813aced5ae923a4fd98725962786635
>