libBitStream fulfills a very useful function which I find missing in many predefined language libraries: the ability to write and write values of artibrary bit lengths to and from memory.

The project is the first C# projec I'm putting on the site and will be used in my next projects libYCC and YCCDemo where it is used to store compact luminance and chrominance data in blocks of 8-bits per sample or less.

libBitStream also allows the basic integer and real types (except decimal) and strings to be stored and retrieved meaning the library is ideal for any application that needs to keep data compact whilst wanting to make storage and retrieval easy.

libBitStream has been developed using the Mono libraries and MonoDevelop under Linux. The final library will be tested under .NET to confirm compatibilty.

Developer Blog

12th February 2009

I started developing libBitStream yesterday alongside libYCC, however I only began implementing the library towards the end of the day so the only progress made was to store values in arbitrary bit-length blocks.

Today, I've almost finished the library. Calling methods can store integer and real values (except decimal because I've yet to discover how to decode decimal values, the others are easy) and strings alongside arbitrary length data. I've also written the class for reading and tested both to satisfaction.

The only features I would like to add are:

• Make use of ArrayList to make managing memory more efficient. At the moment, every time the BitStreamWriter buffer is filled, it creates a new larger byte array and copies the data from the old to the new.

  For small amounts of data this is fine but for half a Gigabyte it's unacceptable. Firstly because the library needs 1+ Gb to actually perform the transfer and then there's the wait whilst copying. Using an ArrayList to refence each block of data means that not only is memory kept to the amount required only for storage, but there's no delay when creating another block when the existing storage has been filled.

• Allow BitStreamReader to read directly from a file, either in its entirety or block by block (using a fixed block size) The latter might be preferrable when the storage requirements for the file are huge and memory is limited.

13th February 2009

I've implemented the ArrayList idea but have yet to test it - can't see it not working though. I've also added a constructor for BitStreamWriter that allows the user to define the size of the sub-storage (or blocks) used to hold the data. The default value is 4096 bytes which is fine for most data, but allows the developer to optimize the routine where they have a good idea of the granularity of the data they're using.

To give you an idea: If a routine required 256MBytes of data to be written using libBitStream, under the default block size the storage resize routine would be called 65,536 times. If the developer chose to increase the block size to 1MByte the resize routine would only be called 256 times. Whilst the overhead of using the default block size would probably be negligible, there's no reason not to take advantage of optimizing how the code works if the facility's there.

I've also implemented File based input for BitStreamReader and file based output for BitStreamWriter to make the transition from permanent to temporary storage easier.

Downloads

Windows binary .DLL Linux binary .DLL Windows source Linux source HTML Documentation
You can also read the documentation here.

I have tested this library with Visual Studio 2008 Express on Microsoft Windows and it seems to be fully compatible. If you have any problems using the libraries please email me.