FileFormats BinaryWriter

BinaryWriter provides easy functionality for storing data in files or sending it over the network. It is fairly low-level, and in theory it is possible to use only its byte-related functionality and have your own way of writing more complex data types like 32 bit integers or strings, but it saves LOADS of time to use an already existing, consistent implementation.

However, if for any reason the ability to read/write files produced by BinaryWriter is needed in a system without BinaryWriter - anything outside of .NET, basically - the following documents the important details about how data is stored. Only a small subset of BinaryWriter's full range is actually used, partially for compatibility reasons.

Generally, little endian encoding is used (LSB first).

bool is written as a single byte, 0 for false and 1 for true.

byte is written as is.

an array of byte is written as is, without length prefixing - this needs to be tracked elsewhere.

an array of char is written as an array of byte, transformed using an Encoding. Currently, default encoding is used, but transition to UTF8 is considered.

int is written as 4 bytes, LSB first: 259 => 0x03, 0x01, 0x00, 0x00

float is first punned to int, then written as 4 bytes, LSB first: 0.23 => 0x3e6b851f => 0x1f, 0x85, 0x6b, 0x3e (IEEE 754 float representation)

string is prefixed with its length encoded in 7-bit variable length integer, followed by bytes obtained from Encoding (see array of char)

7-bit variable length integer

This encoding is the same one as used in MIDI files and is somewhat similar to UTF8's variable length encoding.

If the number is less than 128, it is written as is, as a single byte.

Otherwise, its least significant 7 bits are written to a byte, with the highest bit set to 1, this is repeated. As soon as only 7 bits or less are left to be written, the last byte is written with the highest bit low (0).

This is the source code verbatim from Microsoft's source code repository:

protected void Write7BitEncodedInt(int value) {

        // Write out an int 7 bits at a time.  The high bit of the byte,

        // when on, tells reader to continue reading more bytes.

        uint v = (uint) value;   // support negative numbers

        while (v >= 0x80) {

            Write((byte) (v | 0x80));

            v >>= 7;

        }

        Write((byte)v);

    }

    internal protected int Read7BitEncodedInt() {

        // Read out an Int32 7 bits at a time.  The high bit

        // of the byte when on means to continue reading more bytes.

        int count = 0;

        int shift = 0;

        byte b;

        do {

            // Check for a corrupted stream.  Read a max of 5 bytes.

            // In a future version, add a DataFormatException.

            if (shift == 5 * 7)  // 5 bytes max per Int32, shift += 7

                throw new FormatException(Environment.GetResourceString("Format_Bad7BitInt32"));

            // ReadByte handles end of stream cases for us.

            b = ReadByte();

            count |= (b & 0x7F) << shift;

            shift += 7;

        } while ((b & 0x80) != 0);

        return count;

    }