short bit-analysis of a STANAG-4538 HDLn transfer (BW2 bursts)

 

Each HDLn transfer consists of n TX Frames (n = 24, 12, 6, or 3) each consisting of n data packets; each data packet consists of 233-byte data segment plus a 17-bit Sequence Number. Each TX Frame is sent using burst waveform BW2.

During the construction of BW2, a 32-bit Cyclic Redundancy Check (CRC) value is computed across the 1881 payload data bits of each data packet (233 bytes of user data, plus the 17-bit of Sequence Number) and is then appended to the data packet. Then, seven encoder flush bits with values of zero (Flush) are appended to produce an Extended Data packet of 1920 bit length,ie: 233 data bytes + 7 overhead bytes (17-bit sequence number + 32-bit CRC + 7-bit flush). See this post about the formation of a BW2 burst.

Fig. 1 - BW2 formation

That said, we can go back to the original datagram by inspecting the last 56 bits of the Extended Data packets in the two BW2 bursts (Fig. 2a).

The value of the Packet Number fields varies from 0 to 5, this means that it's a 2 x HDL6 transfer; looking carefully at CRC fields we note that the two HDL6 bursts carry exactly the same datagram: maybe the destination station requested a retransmission (the first BW1 ACK burst) or the datagram is sent twice so to improve the reliability of the transfer (Fig. 2a):

Fig. 2a

That said, we can focus on the Extended Data packets (below termed only "packets") of a single BW2 burst (Fig. 2b).
The values of the six Packet Number fields are: 0,1,2,3,0,1. If TxDatagram buffer is not completely emptied the remaining packet positions are filled with repetitions of packets already residing in other positions of TxFrame buffer. The HDL transmitter is at liberty to select packets from the current datagram to repeat as it pleases (the HDL receiver shall inspect the sequence number of each packet received without errors,and use this information to discard duplicate packets).
The original datagram, in this sample, is then composed of the packets #0 #1 #2 #3.

Fig. 2b

Looking at the Packet Byte Count fields in the first four packets (Fig. 2c) we see that the firts three packets carry 233 bytes of user data (as expected, since HDL) and the last packet carries 224 bytes of user data (the remaining 9 bytes are filled with "0" value bytes).
Thus, the length of the original datagram is (233 x 3 ) + 224 = 923 bytes. 

Fig. 2c

Back to the whole bitstream, once structured in a 1920-bit period (the Extended Data packet length), the original datagram can be extracted by isolating the firts 4 rows and removing the overhead bytes: the resulting is an HARRIS "Citadel" encrypted file of 923 bytes length (Fig. 3).

Fig. 3

The duplicated HDL6 burst (A,B), the retransmitted packets (C,D) and the "0" value bytes filling a single packet can be noted looking at the whole bitstream in Figure 4. 

Please note as the bitstream is misleading: at a superficial glance, one could think to four "Citadel" encrypted files!

Fig. 4

housing of data packets in a HDLn Tx Frame

This recording is nothing special but rather a good example in order to study and verify the way the data packets are housed in an HDLn Tx frame and the mechanisms used to fill the unused room when the length of the outgoing datagram is less than the length of the Tx frame of the used HDLn version (n = 3,6,12,24 packets each of 233 bytes). Particularly, the recording concerns a 1889-byte lenght 'Citadel' encrypted datagram which is sent in one HDL12 PDU, ie using 233 x 12 = 2796 bytes: the protocol shall use the extra space of 907 bytes adding 0-value bytes and reinserting one or more data packets (Fig. 1).

Fig. 1

For what concerns the on-air signal (Fig. 2), the HDL12 PDU is trasmitted within a BW2 waveform across an already-established HF link (FLSU/BW5) and is folowed by three EOM PDUs (BW1) sent by the transmitter station meaning that entire datagram has been successfully delivered. The HDL12 PDU is acknownledged by an ACK PDU (BW1) transmitted by the receiver station. The logical link will be terminated by a couple of FLSU exchanged  between the two stations. The transmission was copied on 8327.0 KHz/USB.

Fig. 2

For a better understanding of the way HDL protocol arranges the extra 917 bytes, let's see how the HDL PDU and BW2 are formed (quoting Annex C to NATO STANAG-4538).

For each of the first data packet positions in HDL TxFrame buffer (12 positions in this case), if the data packet position is empty, construct a new outgoing data packet in this position:

1. Get the next data segment (the next 233 consecutive data bytes to be transmitted) from
TxDatagram buffer; place these in the payload field of the data packet. If fewer than 233 data bytes remain to be transmitted, place these bytes at the beginning of the payload field; fill the remainder of the field with zero bytes.

2. Construct a sequence number value and write this value into the packet’s Sequence Number field.

 3. If TxDatagram buffer is completely emptied of payload data before all packet positions in TxFrame buffer have been filled with new packets, fill the remaining packet positions with repetitions of packets already residing in other positions of TxFrame buffer. The HDL transmitter is at liberty to select packets from the current datagram to repeat as it pleases; the HDL receiver must inspect the sequence number of each packet received without errors, and use this information to discard duplicate packets. 

4. Send an HDL PDU containing the tx frame in TxFrame buffer, using BW2 waveform.

Fig. 3 - formation of the HDL12 Tx fame for this example

The 0-value bytes and the packet repetions are well visible in Fig. 1.

During the construction of BW2 waveform structure, a 32-bit Cyclic Redundancy Check (CRC) value is computed across the 1881 payload data bits in each data packet (1864 bits of user data, plus a 17-bit Sequence Number added by the protocol), and is then appended to the data packet. The seven encoder flush bits with values of zero (Flush) are then appended to produce an Extended Data packet of 1920 bit length that will be used for FEC, frame formation, symbol formation, and so on. 

From theory to practice, the way the data packets are housed in TxFrame buffer can be verified by inspecting the  Sequence Number fields in the last 56 bits of the first nine BW2 extended data packets of the copied transmission (Fig. 4).

Fig. 4 - the last 56 bits (Sequence Number + CRC + Flush) of the copied BW2 Extended Packets

As expected, EOM and SOM fields (bits 16,15) show that the positions 0 and 9 of the TxFrame buffer are both filled with the first packet of the datagram, while position 8 of the TxFrame buffer is filled with the last packet of the datagram. Consequently, the Packet Number (bits 5-0) starts from 0 to 8: ie, the useful payload consists of 9 packets (note also that positions 0 and 9 have the same Packet Number).
Consider now the Packet Byte Count (bits 14-6): this field show the number of user bytes in packet –1. The first eight packets, positions 0-7, contain 233 bytes (011101000 = 232) and the last packet, in position 8, contains 35 bytes (000100010 = 34). The outgoing datagram was then (8 x 233) + 35 = 1899 bytes length.

One could say "since the datagram length is 1899 bytes and fills 9 data packets, why don't use 1 HDL6 PDU + 1 HDL3 PDU?". It would be impossible.
During the link setup phase, the session manager process determines the number of data packets per HDL PDU (3, 6, 12, or 24) shortening the HDL PDU whenever the entire datagram is short enough to fit within the shortened PDU. Once it is determined, the number of data packets per HDL PDU for the current datagram delivery remains unchanged, ie every HDL PDU contains the same number of data packets until the entire datagram has been delivered (so: 3 x HDL3 PDUs, or 2 x HDL6 PDUs, or just 1 HDL12 PDU... btw more PDUs means the more turnaround time).

https://yadi.sk/d/wcvQBJfr3H75uc
https://yadi.sk/i/j0nqeWiC3H75x9 

xDL PDUs, BW2 & BW3 waveforms, and Harris 'Citadel' encryption

I spent some time to understand where and when the 'Citadel' encryption is applied to a message in the STANAG-4538 xDL implementation by Harris, eg in the RF-5800H transceiver, regardless of whether the message came from STANAG-4406 P-MUL or HMTP/CFTP and at least (!) in my copies.

HDL and LDL protocols exist in different variants, and a number n (eg xDLn) specifies the size of one forward transmission. For HDL the number n (24, 12, 6, or 3) should be multiplied by 233 bytes plus a 17-bit sequence number added by the protocol to give the total number of bytes in one forward Tx frame.  If the data segment is of length less than 233 bytes,  a sequence of null data bytes (of value zero) is appended to the data segment so as to extend it to length 233 in constructing the data packet.
In case the datagram is emptied and not all data packet positions have been filled with new packets, the remaining data packet positions are filled with repetitions of packets already residing in other positions. The HDL transmitter is at liberty to select packets from the current datagram to repeat as it pleases; the HDL receiver must inspect the sequence number of each packet received without errors, and use this information to discard duplicate packets.  Note: Whenever a packet is retransmitted, it is always placed in the same packet position within the Tx frame that it occupied in the previous transmission.
Fig. 1 shows the formation of two HDL3 forward transmissions.

Fig. 1 (not in scale)

For LDL, the number n (32, 64, 96,…,512) gives the number of bytes explicitly in a tx frame plus 25 bits due to LDL overhead, ie 17-bit sequence number plus 8-bit data reserved for future use. As for HDL, the tx frame length of the LDL protocol has discrete values, so that a data segment will be padded with 0 value bits if ist length is less than the packet length established for the current LDL transfer (32, 64, 96,…,512). Fig. 2 shows the formation of two LDL512 forward transmissions.

Fig. 2 (not in scale)

Now assume that a STANAG-4406 message shall be encrypted and transferred. P-MUL protocol segments a message into UDP/IP packets with a maximum length of the IP packet of 1500 bytes. If the length of the message to be transferred is 15 kbytes, a total of 11 IP consecutive packets are required for the message transfer. Consider now how the STANAG-4538 protocols operate when transferring one IP packet. At the arrival of the first IP packet at the sender station, a link to the destination node is set up by FLSU PDUs. Then, a 1500 byte datagram containing the first IP packet is transferred for example by using two HDL6 Data PDUs or three LDL512 Data PDUs, each related to one segment of the datagram and followed by an ACK in the reverse direction. 

The Citadel engine works on each datagram before to construct the xDL PDU, in case of the incoming datagram matches the length established for the current xDL transfer then the encryption could be seen as directly applied to the xDL PDU. Since the encryption adds overhead bits, the length of the datagram segments is chosen accordingly.

encryption in a HDL formation

The insertion of the encryption is more clear looking at xDL bitstreams from the real world. An HDL12 transfer is shown in Fig. 3: the presence of the Citadel "business card" is well visible just at the beginning of the PDU (Fig. 5).

Fig.3 - Citadel encrypted 1 x HDL12 transfer

Fig.4 - Citadel encrypted 1 x HDL12 transfer

An LDL32 transfer is shown in Figs. 5,6 : the presence of the Citadel pattern is well visible just in the first packet of the datagram.
 

Fig. 5 - Citadel encrypted 2 x LDL32 transfer

Fig. 6 - Citadel encrypted 2 x LDL32 transfer

Looking at Fig.2, the correspondence 1 LDL PDU = 1 data packet could wrongly lead to think to an encryption applied on data packet basis: the encryption insertion is more clear looking at HDLn, where 1 HDLn PDU = n data packets (Figs. 3,4).

Now look carefully at Fig. 5: each packet is indicated with its index followed by the dimension of the data segment, 32 bytes in this case (LDL32). That said, why five 8-byte rows, ie 40 bytes, are printed ? The same incongruity is verifiable in HDL packets: 240 bytes printed instead of the expected 233 (fig. 7)
 

Fig.7

LDL use burst waveform BW3 to transmit its PDUs. The number of payload bits is of the form 8n+25, where n = 32, 64, 96,..., 512 and the additional 25 bits are the overhead consisting of 17 bits of the sequence number + 8 bits for future use. A 32-bit Cyclic Redundancy Check (CRC) value is computed across the payload data bits in the data packet, and is then appended to the data packet. Then, 7 flush bits having the value 0 are appended to the (8n+57) bits of the data packet with CRC. In the example of Fig. 7 (LDL32) we obtain (8 x 32) + 57 + 7 = 320 bits that make the 40 bytes.

For what concerns HDL PDUs, these are transmitted by BW2 burst waveform. Each data packet is composed of a 1864 bits data-segment (233 bytes) + 17 bits of the sequence number. A 32-bit CRC value is computed across the 1881 payload data bits in each data packet, and is appended to the data packet. Then, seven encoder flush bits with values of zero are appended to the 1913 payload and CRC bits of each data packet to produce an Extended Data Packet (EDataPkt) of 1920 bits or just 240 bytes.

Juts another way to say that HDL is not BW2, and LDL is not BW3: xDL are protocols while BWn are waveforms.

For what concerns MDL protocol, it seems that the Citadel encryption is applied on the incoming datagram before its segmentation (Fig. 8).

Fig.8 -  MDL transfer using 3 x LDL32

Behavior of STANAG-4538 depends on vendor implementation as well as on its configurable parameters, so this post does not aim to show a sort of  fixed rule but just some cases of  transmissions that happened to hear.

 

STANAG-4538 3G-HF, HDL complete session

3G-HF HDL transfer heard on 10627.0 KHz/USB at 1258 UTC
The High-throughput Data Link protocol (HDL), defined in STANAG-4538, is a selective repeat ARQ protocol which can only be run in a point-to-point data packet connection. HDL is most efficient when large volumes of data are to be transmitted and the channel conditions are moderately good, while LDL is best suited for small data volumes and in poor quality channel
"Data transfer by HDL begins after the stations have already established the data link connection in the traffic setup phase (using FLSU BW5 waveform). In an HDL data transfer, the sending station and the receiving station alternate transmissions in the manner depicted in figure 1; the sending station transmitting HDL_DATA PDUs containing payload data packets, and the receiving station transmitting HDL_ACK PDUs containing acknowledgments of the data packets received without errors in the preceding HDL_DATA PDU. The end of a data transfer is reached when the sending station has transmitted HDL_DATA PDUs containing all of the payload data in the delivered datagram, and the receiving station has received these data without errors and has acknowledged their successful delivery. When the sending station receives an HDL_ACK PDU indicating that the entire contents of the datagram have been delivered successfully, it sends an HDL_EOM PDU repeated as many times as possible within the duration of an HDL_DATA PDU, starting at the time at which it would have otherwise transmitted the next HDL_DATA PDU." [1]

fig. 1a

Fig. 1b

 
As in STANAG-4538 Table 13-1, HDL protocol use the burst waveforms BW2 for data forward and BW1 for ACK and EOM/Term signal, all the burst waveforms use the basic PSK-8 modulation at 2400 baud centered at 1800hz also used in the MIL-STD 188-110A serial tone modem waveform (fig. 2) and can be identified by measuring their duration (fig. 3)

fig. 2

fig. 3

BW2 consists of 100 msec TLC section and a short (26,67 msec) preamble followed by a number of fixed-size data packets (3, 6, 12, or 24): the number of packets is negotiated before the HDL protocol starts and remains unchanged until the end of the data transfer. Each packet consists of 20 frames, each of which contains 32 unknown symbols (data) followed by 16 known symbols acting as probe (fig. 5). 
HDL protocol is packet-oriented, in contrast with LDL protocol which is byte-oriented, and it can be designated by a number as HDL<n> where n - as said above - is the negotiated  number of packets which are transmitted in one forward frame: for example, in the recorderd transfer we see HDL3-type frames.

fig. 5

Burst waveform 1 (BW1) is a general-purpose waveform used to carry short messages for many of the 3G protocols: traffic management, link maintenance, and data acknowledgments for the HDL protocol. It consists of 576 PSK-8 symbols preamble followed by 2304 PSK-8 symbols of data which are coded using 16-ary Walsh seqences (fig. 6).

fig. 6

It's worth noting that all the six burst waveforms specified by STANAG-4538 begin with a TLC/AGC guard sequence, why? Existing HF radios were generally not designed with burst waveforms in mind. For example, MIL-STD-188-141 military radios are allowed 25 ms to reach full transmit power after keying. While the transmitter radio frequency stages are ramping up, the input audio signal level is adjusted by a transmit level control (TLC) loop so that it fully modulates the transmit power. At the receiver, an automatic gain control (AGC) loop must also adjust to a new receive signal. To accommodate these characteristics of existing radios, the 3G burst waveforms just begin with a TLC section of “throwaway” 8-ary PSK symbols that are passed through the system while the transmitter’s and receiver’s level control loops stabilize.

[1] from: "Third-Generation and Wideband HF Radio Communications" 
 

https://yadi.sk/d/kwnmIEE_y3xNb