Abhishek Goyal abgoyal@gmail.com writes:
Having dealt with custom binary formats for large amounts of data quite a bit, I tend to recommend against doing that if standard options can instead be used: there are a large number of pitfalls in going binary, both when writing the data, as well as when reading it back;
A message full of excellent points. I failed to think of hdf5/protobuf and all the rest of the great suggestions. So I will second the notion that a custom binary format is very likely not a good idea.