Decoder can't catch errors in CAT021/RE (wrong length) and silently skips to next datablock

[ifsnop]

Reported length of the datablock is 5D = 93 bytes. There are 93 bytes on the datablock.

root@host# echo "15 00 5D F3 1B 7B 6B D3 A7 04 14 D8 01 01 00 00 8A 01 0E E6 37 53 01 13 CA 6C 39 64 F2 54 62 13 54 62 1E 17 18 31 F3 15 B0 12 0F A0 05 C8 40 00 14 08 E9 7D 51 54 62 51 10 12 31 C3 0C E0 15 00 5A F3 1B 73 6B D3 A7 04 14 D8 01 01 00 00 08 01 0F 08 FA 94 01 D8 65 36 4D 20 67 54 62" | xxd -r -p | asterix_d -d asterix.ini
[...]
ERROR:   Mar 17 15:03:15 2026 (asterixformat.cxx) DataItem::parse needed length=2 , and there is only 0 : [  ]
ERROR:   Mar 17 15:03:15 2026 (asterixformat.cxx) Wrong length in DataItem format for CAT021/I040
ERROR:   Mar 17 15:03:15 2026 (asterixformat.cxx) Data Record bytes: [ 62 ]
ERROR:   Mar 17 15:03:15 2026 (asterixformat.cxx) Wrong data length in Explicit. Needed=11 and there is 7 bytes.

Everything is ok until the decoding of the Reserved Expansion:

08 FA 94 01 D8 65 36 4D 20 67 54 62

08 bytes of length reported, but 12 bytes in the content. So decoder ends in the 8th byte, and feeds the remaining bytes (20 67 54 62) to the decoder.

Then, I honestly don't know what the decoder process does with these bytes (20 67 54 62). Are they parsed as fspec (20) and then reported as I021/161 (67 54), and again another fspec (62), and when going to decode I021/040 no bytes are left?

Can we protect somehow against this problem?

[KOGrover]

You can't protect against this other than by not having malformed data. If you give the parser bad data, unexpected results happen: how's it supposed to know to trust the REF LEN or the Indicator Bits? By the bits in your "Items Indicator" (0xfa - 0x1111_1010, your REF LEN should be at least 12. However, the SGV field can be 2 or 3 bytes. The FX is set in byte 2 - I think, this is a quick look: so it should be 3 bytes. Thus you REF LEN should be 13, but there's really only 12 bytes there. What is the parser supposed do in this case? If you give the parser bad data, unexpected results happen: how's it supposed to know to trust the REF LEN or the Indicator Bits? The parser tries to parse the remaining data as another messages because the early ASTERIX spec (part 1) allowed blocking: another CAT + FSPEC + FIELDS, until all the bytes in this 'message' (the length from the ASTERIX header) are used. It's one of the downsides to the ASTERIX format: unless you can exactly parse (or know the size of each field to discard, only if you know the revision the message was coded to, so you know the UAP) all fields, you cannot continue. Even with perfectly formed data, there is no guarantee you can parse it correctly - there is no indicator in the message what version of the ASTERIX (for the CATEGORY) spec is being used. You must know that (from external sources) so you know the exact UAP, so you can parse all of the fields (and know their sizes). IMO the Category version number should be in the ASTERIX header to directly specify which version of that category message you are processing. It would have also been better to have the format used tagged blocks (like TIFF, PNG, PCAPNG, etc) each with a length, so that any block you do not understand (or care about) could be skipped. This would also mean that the order of the tags (fields in ASTERIX terms) is not so important, and if new tags are added it does not break existing readers.

…

On Tue, Mar 17, 2026 at 8:24 AM diego torres ***@***.***> wrote: *ifsnop* created an issue (CroatiaControlLtd/asterix#252) <#252> Reported length of the datablock is 5D = 93 bytes. There are 93 bytes on the datablock. ***@***.***# echo "15 00 5D F3 1B 7B 6B D3 A7 04 14 D8 01 01 00 00 8A 01 0E E6 37 53 01 13 CA 6C 39 64 F2 54 62 13 54 62 1E 17 18 31 F3 15 B0 12 0F A0 05 C8 40 00 14 08 E9 7D 51 54 62 51 10 12 31 C3 0C E0 15 00 5A F3 1B 73 6B D3 A7 04 14 D8 01 01 00 00 08 01 0F 08 FA 94 01 D8 65 36 4D 20 67 54 62" | xxd -r -p | asterix_d -d asterix.ini [...] ERROR: Mar 17 15:03:15 2026 (asterixformat.cxx) DataItem::parse needed length=2 , and there is only 0 : [ ] ERROR: Mar 17 15:03:15 2026 (asterixformat.cxx) Wrong length in DataItem format for CAT021/I040 ERROR: Mar 17 15:03:15 2026 (asterixformat.cxx) Data Record bytes: [ 62 ] ERROR: Mar 17 15:03:15 2026 (asterixformat.cxx) Wrong data length in Explicit. Needed=11 and there is 7 bytes. Everything is ok until the decoding of the Reserved Expansion: 08 FA 94 01 D8 65 36 4D 20 67 54 62 08 bytes of length reported, but 12 bytes in the content. So decoder aborts in the 8th byte, and feeds the remaining bytes (20 67 54 62) to the decoder. Then, I honestly don't know what the decoder process does with these bytes (20 67 54 62). Are they parsed as fspec (20) and then reported as I021/161 (67 54), and again another fspec (62), and when going to decode I021/040 no bytes are left? Can we protect somehow against this problem? — Reply to this email directly, view it on GitHub <#252>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABDQPYD45VDASN6WQW7U7VD4RFU3RAVCNFSM6AAAAACWUYKBD6VHI2DSMVQWIX3LMV43ASLTON2WKOZUGA4DSMRZGEZTQOA> . You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>

[ifsnop]

Maybe using FinalFormat (as it has a header with datablock length) may help against this kind of issues?