Schema Library

Ready-made EByte schemas for real formats. Each one comes with a real sample file, and on every change to EByte the tests decode that sample and encode it back, checking every byte. Download a schema, point ebyte at your own file, and you’ll know within a line whether it matches.

All nine are in the schemas folder on GitHub, along with a note on each schema’s limits.

Using One

ebyte validate-schema --schema pcm-wav.schema.json
ebyte inspect --schema pcm-wav.schema.json --input recording.wav --format text

A file that matches gives a complete trace and exit code 0. A file that doesn’t gives one line of JSON naming the field and the byte offset where it stopped matching, and exit code 2. The folder-check example runs the WAV schema over a whole folder.

What Each Schema Covers

Since 0.3.0 the schemas derive their sizes, so a file whose stored size, offset or checksum is wrong fails at that field. To read such a file anyway, delete that field’s derive and it becomes an ordinary integer.

WAV. The 44-byte header of a plain PCM file, with the RIFF, fmt and data chunks in that order. Files with extra chunks, such as a LIST chunk of metadata, are reported as trailing data. Since 0.3.0 riff_size and data_size are derived, so they follow the amount of audio; when you change the sample rate, update byte_rate and block_align yourself.

WAV, any chunks. Needs EByte 0.2.0. Reads the RIFF header and then every chunk, fmt , data, LIST or anything else, in whatever order they come, each as an id, a size and a raw body. Chunks can be added, removed or reordered in the document, and since 0.3.0 riff_size and every chunk size are derived. Chunks with an odd size, padded with a byte the size doesn’t count, aren’t supported and stop the schema at the pad byte.

BMP. Uncompressed 24-bit images with the 40-byte info header, stored bottom-up or top-down. Paletted, 32-bit and compressed BMPs are rejected at the field that differs. Since 0.3.0 file_size and image_size are derived; width and height must still match the pixel data.

ZIP, any entries. Needs EByte 0.2.0. Reads local entries while they start with PK 03 04, then central directory entries while they start with PK 01 02, then the end record, so any number of stored or compressed files works. Archives using data descriptors, ZIP64 or split volumes aren’t supported. Since 0.3.0 every length, both entry counts and the directory size and offset are derived. Each entry’s local header offset, its sizes as copied into the directory, and its CRC-32 (which covers the uncompressed file) are still edited by hand, as the derived sizes case study shows.

PNG. Needs EByte 0.4.0. Reads the signature and then every chunk, IHDR, tEXt, IDAT, IEND or any other, as a type and raw data. Every chunk length and CRC-32 is derived, so text and metadata chunks can be edited, added or removed and the file stays valid. Image data stays zlib-compressed hex: EByte can’t edit pixels, and changing the size in IHDR without new image data gives a file with correct CRCs that viewers reject.

pcap. Needs EByte 0.3.0. Reads a classic little-endian, microsecond libpcap capture: the global header, then every packet’s timestamps, lengths and raw frame. incl_len is derived from the frame; orig_len is kept as stored, because a capture can be shorter than the packet. Big-endian or nanosecond captures and pcapng files are rejected at the magic number. To see inside frames, extend the packet with your protocol’s fields, as the sensor case study does.

Float WAV. Needs EByte 0.6.0. A 32-bit float WAV (format 3) with only fmt and data chunks; every sample is an f32 value in the JSON, so audio can be edited as numbers. riff_size and data_size are derived; byte_rate and block_align are edited by hand.

8-bit BMP. Needs EByte 0.6.0. An uncompressed paletted BMP: the palette is every byte from the end of the 40-byte header up to pixel_offset, kept exactly. Colours can be edited in place; changing the palette’s size means editing pixel_offset and colors_used by hand.

One-file ZIP. An archive holding exactly one stored (uncompressed) file, from the local header to the end record. A second file stops the schema inside the second file’s header. Since 0.4.0 every length, size and offset and both copies of the CRC-32 are derived, because stored data isn’t compressed; a damaged file now fails with checksum_mismatch.

What’s Coming

Repeating blocks arrived in 0.2.0, derived sizes and offsets in 0.3.0 and checksums in 0.4.0. 0.5.0 added ebyte diff, which compares two files through any of these schemas. 0.6.0 added floats, bit fields and fields that run up to a declared offset, used by the float WAV and 8-bit BMP schemas. Since 0.7.0 every schema here can also be tried in the browser demo with its sample file. As each one lands, the schemas here get wider and new formats join the list. To suggest a format, use the contact page.