Bill Gaylord chibill110@gmail.com writes:
Hello,
I am working on making a binary format for rtl_power to save space onmy server for observing the spectrum. I am wondering if anyone else would be interested in this idea. I am hoping keep the same format in terms of how the data is organized in the file but instead of using text use some form of binary format. For example for the date and time I think using a unix epoch timestamp would work.
I am hoping to receive any comments or criticism.
My suggestions are:
Think about whether you want the binary format to be portable. This means being more careful about types and also endianness.
Post a draft file format for comments.
With a binary format for an existing text format, it would be nice to have programs (pipes ideally) that convert text->binary and binary->text.
For portability, I think the obvious answer is that the file format should be independent of CPU type, compiler, and specifically endiannesss. Thus you are writing a format that could be used over the network (even if it's only intended to be stored). This means you have to choose fixed-width types, and you have to store in a particular endianness. I am 99% sure that the network protocol standard is big endian, and I think you should follow that. I think there is precedent in things like sqlite. For pgsql I think so, but there is a culture of having to use pg_dump to change versions (and as a result pg_dump is 100% satisfactory; see my comment above about converters). I don't know about mysql.