Enable packing and unpacking structures with user-specified gaps between fields

[kagalenko-m-b]

Following the discussion in #34, I implemented the suggested syntax for packing to/unpacking from a binary buffer while skipping some parts of it. The usage may be illustrated by this fragment from runtests.jl:

@io struct RawData
    A::UInt32
    dummy_1::UInt8
    dummy_2::UInt16
    B::UInt16
    dummy_3::UInt32
    C::UInt128
    dummy_4::UInt16
    dummy_5::UInt64
    D::UInt8
end align_packed

@io struct GappedStruct
    A::UInt32
    _::3
    B::UInt16
    _::4
    C::UInt128
    _::2
    _::8
    _D::UInt8
end align_packed

A,B,C,D = 1,2,3,4
gapped_struct =  GappedStruct(A, B, C, D)
raw_data = RawData(A, 0xBE, 0xBEEF, B, 0xDEADBEEF, C,  0xBEEF, 0xDEADBEEFDEADBEEF,  D);
unpacked_raw_data = RawData(A, 0, 0, B, 0, C, 0, 0, D);

buf = IOBuffer()
pack(buf, raw_data)
seekstart(buf)
@test unpack(buf, GappedStruct) == gapped_struct
buf = IOBuffer(zeros(UInt8, packed_sizeof(GappedStruct)), read=true, write=true)
pack(buf, gapped_struct)
seekstart(buf)
@test unpack(buf, GappedStruct) == gapped_struct
seekstart(buf)
@test unpack(buf, RawData) == unpacked_raw_data

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 95.75%. Comparing base (1b79251) to head (4d73132).
⚠️ Report is 8 commits behind head on master.

Additional details and impacted files

@@            Coverage Diff             @@
##           master      #35      +/-   ##
==========================================
+ Coverage   94.53%   95.75%   +1.22%     
==========================================
  Files           1        1              
  Lines         128      165      +37     
==========================================
+ Hits          121      158      +37     
  Misses          7        7              

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[ararslan]

Why make this optional rather than just always querying field_gaps(T)? If T has gaps, wouldn't setting gaps equal to anything but its default value be wrong?

[kagalenko-m-b]

First of all, thank you for thoughtful comments. Now, to the substance:

Why make this optional

I had in mind the use case when I read in a structure with gaps and then write it out as a normal 'dense' structure (or vice-versa). This also allows run-time computation of gaps, which could be potentially useful.

[ararslan]

I wonder if it would make sense to unconditionally retain all gap information, even if there are no gaps, in which case all entries of the array would be 0. Then you wouldn't need to condition on isempty(field_gaps(T)) and would know that it always has size fieldcount(T) + 1.

[kagalenko-m-b]

I wonder if it would make sense to unconditionally retain all gap information, even if there are no gaps,
in which case all entries of the array would be 0.

I was thinking about the loop where I read fields one by one. If gap values array is empty, then the compiler can figure out that the condition need not be checked at every loop iteration.

[ararslan]

It might make sense for that to be an error, but I guess it doesn't hurt to handle it as you've done.

[kagalenko-m-b]

I allowed several gaps in unbroken succession to make this more "ergonomic" when one takes some large struct and gets rid of unneeded fields, but then changes his mind and retains a previously skipped field. No manual summing of field lengths also reduces the chances to make an error.

[ararslan]

Thinking about this more, can you really say that padding occurring before the first field or after the last field is part of the structure? I could see a case for allowing padding following each field (so you have at most fieldcount(T) gaps) but also having leading (or vice versa) feels weird. Is there precedent in other languages, do you know?

[kagalenko-m-b]

I do not know about the precedents, but I think that allowing space after the last field is useful when reading repeated structs from a file. Otherwise one would have to skip() manually.