STANAG-4285 & HMTP-66 (HF Mail Transport Protocol) example

Other than (H)BFTP and CFTP, e-mails can be sent over HF using the protocol HMTP (HF Mail Transport Protocol) defined in STANAG-5066 Annex F and  MIL 188-141B Appendix E.  The HMTP protocol is used by a mail client to submit a mail-object to a mail server using the allowed HF transport waveforms, or alternatively, it may be used as the protocol to transfer a mail object over the HF subnetwork from one mail server to another. The HMTP protocol is NOT used to send and receive Formal or High Grade Military Messages but rather for informal interpersonal e-mail only.

The transmission has been heard on 5054.0 KHz/USB at 0804 UTC on 26 September, the used HF waveform is STANAG-4285: ACF and frame structure are shown in fig. 1. 

fig. 1 - ACF and frame structure of the head signal (STANAG-4285)

 

A 1776 bit period, often met in STANAG-5066, pops up after removal the STANAG-4285 overhead (fig. 2)

 

fig. 2 - 1776 bit period

 

The "payload" (email message) is visible in figure 3 once removed the STANAG-5066 headers:

fig. 3 - HMTP mail message

Since the data link protocol used in this transmission is STANAG-5066 we expect HMTP-66 sequences, and this is pecisely what you get (fig. 5):

fig. 5

 

Note that the sequences in-air are "splitted" in PDUs so we see sequences in seven files that will be re-assembled at S-5066 receive peer.

MIL 188-110C App.D: BW6 KHz, SR4800 Bd, Walsh

yet another 188-110C App.D signal (WBHF,  Wide Band High Frequency) spotted around 1850 UTC on 5407.0 KHz/USB by my friend Karapuz: it worth noting the bandwidth of the signal, 6000Hz, and consequently the sampling frequency adopted for the recording, 24000 Hz, which allows a good signal resolution and accommodation. Since the presence of an annoying fading and the signal strength,  the block #2 is the most suitable for a good analysis.

The basic parameter of the waveform are shown in figs 2,3

fig. 2 - baudrate line

fig. 3 - PSK-8 constellation

The cited value of 5407.0 KHz is the tuning frequency used to mantain the signal at the center of the band and thus it isn't the real dial frequency: indeed, you may note that the carrier frequency is almost the double of the expected 3300Hz.

preamble section

From the 188-110C App.D documentation, the orthogonal Walsh modulation is used in the Synchronization Section of the preamble and the length of each super-frame is 18 channel-symbols, ie:

9 (fixed) + 4 (downcount symbols) + 5 (waveform identification symbols) 

Since in 6KHz bandwidth waveforms the preamble channel-symbol is 64 symbol length, the length of each repeated superframe is: 18 (channel-symbols) x 64 (length of one channel-symbol) = 3456 bit. 

fig. 4 - repeated superframes in the Sync Section of the preamble

The lenght of the Sync Section superframes generates the 3456 bit period which is apparent in the bitstream of the preamble after its demodulation (fig. 5).

fig. 5 - 3456 bit period in the preamble due to the superframes length

data section

The data section exhibits ~426ms ACF spikes (fig. 6) that make a 6144-bit length period(!), corresponding to 2048 tribit symbols. The period does not have the Known/Unknown data structure, so mini-probes are not sent but rather the data symbols are sent continuously after the initial preamble: this means that the block #2 is the wavfeorm Id 0 and Walsh Orthogonal Modulation is used.

fig. 6 - ACF value measured in the data section

from D.5.1.2.3  (MIL 188-110C Appendix D):

Waveform ID 0 utilizes a different modulation technique, Walsh Orthogonal Modulation. For each pair of coded and interleaved data bits, the method produces a 32 symbol repeated Walsh sequence. The Walsh Orthogonal Modulation is accomplished by taking each pair of bits, or di-bit, and selecting a corresponding Walsh Sequence. The selected four element Walsh sequence is repeated 8 times to yield a 32 element Walsh sequence. For example, if the di-bit is 01, the sequence 0404 is repeated to generate the 32 symbol sequence:

0, 4, 0, 4, 0, 4, 0, 4, 0, 4, 0, 4, 0, 4, 0, 4, 0, 4, 0, 4, 0, 4, 0, 4, 0, 4, 0, 4, 0, 4, 0, 4

Processing the bitstream of the data section, we get a value of 6144 bit (fig. 7) that matches the ACF value obtained in fig 6:

fig. 7 - 6144 bit period of the data section

Why this 6144 bit? 

For the Walsh Orthogonal Modes (waveform id 0) the data scrambling implementation generates 256 x 8 = 2048 values and the scrambling sequences are continuously wrapped around the 2048 symbol boundary: ie just 2048 x 3 = 6144 bit and then the ACF of the data section is due to the scrambler lenght.

Athough data are modulated using Walsh ortogonal modulation, they are scrambled to appear on-air as the PSK-8 constellation seen in fig. 3.

PACTOR-IV ("Dragon" modem) BPSK/QPSK

PacTOR-IV transmission using BPSK and QPSK modulation and speed of 1800 symbols/sec (in both the cases), without the characteristic PacTOR-IV spreading factor used at lower data rates. The presence of the spreading can be detected running the 'AM detector' module of SA and looking at the phase vector of the signal: in fig. 2 the AM detector exhibits a single baudrate line (so no spread factor) and the phase vector has a normal trend.

fig. 2

Instead, the AM detector in fig.3 exhibits multiple baudrate lines and the phase vector assumes the characteristic sawtooth ramp. Note that the base speed is about 112.5 Baud (the lower baudrate line and the frequency in the phase vector) while the speed detected by tha analyzer is 1800 Baud: this is a spread-16 since 112.5Bd * 16 just results 1800Bd.

fig. 3

Modulation vary according to the channel condition (PACTOR-IV is an adaptive mode) and the resulting constellations are visible in fig. 4, the related phase vectors are shown in fig. 5

fig. 4 - constellations

fig.5 - phase vectors for BPSK and QPSK blocks

A distinguishing characteristic, other than the 3300ms block length, is the 239 symbols lenght of the BPSK and QPSK frame structures:
239 symbols = 239 bit for BPSK (two modulation symbols, 1 binary digit)
239 symbols = 478 bits for QPSK (four modulation symbols, 2 binary digits)

Bitstreams after demodulations and frame structures are shown in figs 6,7. It may happen that the bistreams have the period length which  is the double of the expected one: most likely this is due to the  un-initialized state of the (generic) psk demodulator of SA.

fig. 6 - PACTOR-IV BPSK frame structure

fig. 7 - PACTOR-IV QPSK frame structure

HF networks mapping from on-air signals

The aim of the post is to illustrate an "hand" method to draw HF networks from STANAG-5066 Protocol Data Units (PDUs) that are exchanged over HF radio between peer HF nodes: mainly, the DATA-ONLY type 0 D_PDUs (Simplex data transfer) will be used here.
According to the STANAG-5066 terminology, a node (or HF node), is an implementation of the profile described in the main body and mandatory annexes to the S-5066 profile. The node is generally assumed to include the HF (modem and radio) and cryptographic equipment required for communications. A subnetwork is a collection of nodes. As a whole, a subnetwork provides a reliable networked data-transport service for external users or clients (client applications). 
It may happen that in certain deployments the client applications (clients) reside inside the same physical machine that run the node as well as deployments in which the clients reside in a LAN(s), behind the HF node.

To map a certain STANAG-5066 HF network we need to know the IP adresses of the nodes which belong to that network: it can be done by recording the transmissions from that network and analyzing the S-5066 D_PDUs which compose the bitstream obtained after the removal of the overhead added by the used HF waveform (S4285, MS188-110A/B,...). If the heard transmissions are in plain-text (not encrypted) then source and destination IP addresses will be found in the headers of the D_PDUs (fig. 1). These on-air addresses will be associated to the routing indicators in the HF address table of the 5066 software.

fig. 1

The D_PDU headers can be highlighted by synchronizing the bitstream on the 16-bit Maury-Styles sequence 0xEB90 since all D_PDUs, regardless of type as seen in previous posts in this blog, begin with that same sync. Sometimes, change the polarity may be necessary (fig. 2). The sync sequence 0xEB90 causes a marked 1776 bit ACF, that makes a 222 bytes period, in case of DATA-ONLY type D_PDUs.

fig. 2

As specified in Annex C of S-5066, the Size-of-Address Field  specify the number of bytes in which the source and destination address are encoded. The address field may be from one (1) to seven (7) bytes in length, with the source and destination address of equal length.
Since the D_PDU header must be made up of an integer number of bytes, addresses are available in 4-bit increments of size: 4 bits (or 0.5 bytes), 1 byte, 1.5 bytes, 2 bytes, 2.5 bytes, 3 bytes, and 3.5 bytes.
In the headers of figure 2 the size of address field is the binary 111 or decimal 7, so half of the bits, 3.5 bytes, are assigned to the source and the other half to the destination. By dividing the addresses field we get the 0.0.0.203 as the source IP, and 0.0.0.90 as the destination IP of the peers (figs. 3a,3b).

fig. 3a

fig. 3b

The "Group Address" can be likely found in D_PDU headers: as above, these addresses are obtained by looking at size of address field as in figs 4,5.

fig. 4

fig. 5

Transmissions that exchange data are usually preceeded by MS 188-141A handshakes which allow to build and populate a sort of basic "HF pairings table" [1] for each organization you want monitor, consisting of the ALE calls and their (on-air) IP addresses. With a little luck you can receive HF emails and henance your pairings-tables by adding the e-mail addresses (fig. 6).

fig. 6

Below some simple mappings of little(!) pieces of real HF networks: although the IP addresses come from real-world, some node names are fictional. Countries and  Mil/Gov/Diplo authorities and organizations are intentionally omitted.

As said, we focused on the DATA-ONLY type 0 D_PDUs but source and destination IP addresses are available also from the other D_PDU types (up to 15). By the way, since type 0 D_PDU send segmented C_PDUs, they are used in conjunction with a basic selective automatic repeat request type of protocol such as the ACK-ONLY type 1 D_PDU (fig. 7).
 

fig. 7

What has said above is an hand-method to obtain the addresses, a great help comes from protocol analyzers which can do the work for you.  Anyway, comparisons between results and bitstreams is always a good practice:  exotical IP addresses may come from corrupted frames (fig.8) which are ignored by the parser.

fig.8

As a final note, I want to advise that this post is not an hostile attempt or an encouragement to grab reserved informations since the heard transmissions are in clear-text and a rich documentation of STANAG-5066 D_PDUs is widely known and publicly downloadable from NATO and other web sites.
I have only shown the "boxes"  and not their contents: contents are encrypted and anyway I am not interested in them but only in the technical aspects of such communications. Likely, further posts on this topic will follow.


[1] the so-called "HF Domain Map" and "HF Address Table" as they appear in some 5066 system packages: it's worth noting the similarity with the "HF Pairing table". Thanks to my friend Marco for these screenshots!

example of e-mail over HF using STANAG-5066 and STANAG-4285 as HF waveform

Looking at some my recent logs to which this post refers:

05838.0 ABC7: Unid  0734 USB MIL 188-141 2G-ALE ABS5 handshake, voice auth-id
then STANAG-5066 HMTP over STANAG-4285 1200bps/S (06Sep16)
05838.0 ABC7: Unid  0738 USB MIL 188-141 2G-ALE ABD1 handshake, voice auth-IDs exchange then STANAG-5066 HMTP over STANAG-4285 1200bps/S (06Sep16)
05838.0 ABC7: Unid  0747 USB MIL 188-141 2G-ALE ABG6 handshake, voice auth-IDs exchange then STANAG-5066 data over STANAG-4285 1200bps/S (06Sep16)
05838.0 ABC7: Unid 0804 USB MIL 188-141 2G-ALE ABF2 handshake, voice radio-check, auth-IDs exchange then STANAG-5066 data over STANAG-4285 1200bps/S (08Sep16)
05838.0 ABC7: Unid  0818 USB MIL 188-141 2G-ALE ABk4 handshake, voice auth-IDs exchange then STANAG-5066 data over STANAG-4285 1200bps/S (06Sep16)

"ABC7", likely the main station, establishes a link with each outstation ("ABK4","ABG6",..) using 188-141 2G-ALE and then forwards e-mails using STANAG-5066 HMTP at data link layer and STANAG-4285 as HF waveform (fig. 1).

fig. 1

I already posted about sending e-mail over HF, browse the Data Link tag, but worth publish this example for three good reasons:
1) the used HF wavefom is STANAG-4285 rather than MS188-110A/B STANAG-4539;
2) the "messaging" software is an HMTP system (although email is sent using zipped files)
3) the email client (and likely the STANAG-5066 controller/gateway too) is Linux rather than MS Outlook/Windows.

The operational configuration of a 2G HF system based on 188-141A, STANAG-5066 and STANAG-4285 could sound a bit strange since in BRASS scenarios we are used to see ship/shore links put in place using a pool of known frequencies or MRL circuits, and, in land scenarios, 188-141A almost always followed by 188-110A/B STANAG-4539 waveforms.

I want to mean that, although many fleets make use of 188-141 technology (Venezuelan Navy,  Chilean Navy, Iraqi Navy, Maltese Navy,... Italian GdF Corp too), its use in conjunction with STANAG-4285 is a bit unusual, at least in Europe.

fig. 2 - S4285 frame obtained from the analysis of the received signals

The bistream after S4285 decoding exhibits a 1776 bit length period, or 222 bytes (fig. 4) that matches the length of the data only D_PDU (see later)

fig. 3 - receiving data from modem on COM3 (note the synchronous settings for port COM3)

fig. 4 - S5066 1176-bit period (222 bytes)

Data only D_PDUs may transport plain text or GZIPped files such as .eml, .txt and others: in this case I extract an email whose study offers interesting insights (fig. 5)
 

fig. 5

1.  email domains are not routable ones ("asdf.123" and "jklp.123");
2.  use of generic user names user1@asdf.123,user1@jklp.123 (maybe tests?). And about the users, voice comms are in a "formal" English, sometimes with a wrong pronunciation (i.e. "five" pronunced as "faìv" with accent on "i");
3. email headers say that a Linux SuSe system is used: "Received: from linux.site ([127.0.0.1]) by linux.site with ESMTP (CROZ ESMTP/HMTP Gateway) Gecko/20111101 SUSE/3.1.16 Thunderbird/3.1.16". Note the use of HMTP protocol.
4. the attachment name is in Slavic language: "image/jpeg  name="PRILOG 5 - PRILOG ZA SLANJE E-MAILA.jpg".


As said, the HMTP (HF Message Transfer Protocol) is used as messaging protocol rather than the FTP-oriented protocols such as CFTP and (H)BFTP. Shortly, HMTP performs IP over HF and defines a mode of operation very similar to standard SMTP pipe-lining that minimizes number of turnarounds (to two for a small message). It also fixes options to maximize interoperability without the need for service negotiation. HMTP is defined in Annex F of STANAG-5066.

As a final note, protocol analyzers (and not protocol 'classifiers') are very powerful tools that allow to go back to the structure in which the data bits are arranged. Indeed,  in addition to extract and uncompress the .eml file, the analyzer also splits the data blocks in files and retuns back a text file (OutInfo.txt) containing a complete and detailed report of the STANAG-5066 D_PDUs that are involved in the transfer:

This way the SIGINT enthusiasts have the chance to study protocols looking at examples from practice and not only from books, as shown in the following figure just related to the signal in the matter:

STANAG-5066 D_PDUs belonging to frames 19 and 20 (out000.dat file)

(to be continued in a next post)

BRASS/MRL file transfer with STANAG-5066 and STANAG-4285

fig. 1a - STANAG-4285 in half-duplex mode

fig. 1b - STANAG-4285 in full-duplex mode

I recently had the chance to analyze some interesting recordings from a two nodes BRASS-MRL test (MRL, or Maritime Rear Link, is a special Ship/Shore link using a separately allocated HF frequency and with messages flowing in both directions).  

Both the link-ends are equipped with Thales HDM-03 HR modems  and  BID-950 as crypto devices. The used HF waveform, as usual in current BRASS deployments, is STANAG-4285, although in some publications it's also reported as "S-4285C" where the ending "C" stands for  "coded", hence the use of FEC and interleavig (Annex E of S-4528). At the data-link layer, the STANAG-5066 interfaces and functionalities are provided by the Selex MDH package.
The recordings are two tracks "stereo" and  pertain half-duplex and full-duplex transmissions [1][2], encrypted and plain-text mode. In the latter, BID-950 are electrically disconnected from the red/black circuits so are not traversed by the data flow: I focused only on these recordings. 

I started by separating the full-duplex stereo track into two mono channels (thanks to the sw package "Audacity") and then analyzed one of these (the track related to the "slave" transmissions): the 106.66ms ACF value and the 256 symbol frame structure meet the expected values for the S-4285 waveform (figs 2,3).

fig. 2

fig. 3

Since the lack of the auto-baud functionality, messages about data rate exchange must be transmitted and in this case the DRA mode (Data Rate Adaptive) must be also activated to manage the underlaying (adaptive) modems. In this case DRA is not activated and the right settings for decoding the S-5066 ARQ transmissions are 4285 75bps/Short for the ARQ messages and 4285 2400bps/Short for the file transfer (fig. 4)

fig. 4

Protocol S-5066 has been detected by processing the demodulated bitstream with the protocol analyzer module (fig. 5)

fig.5

According to fig. 1a, the file is sent as two "chunks": indeed, after S-5066 removal, we get two extracted files 000_out and 001_out (will be reassembled by the S-5066 package running at the receive side).
 

fig.6

Looking at the ASCII view of these files (fig. 7) are visible some interesting clues as the "ARQ1 ARQ2 TESTER" label, the file name, and the indication "S5066 GUI" that is placed at the start and the end of the file. 

As said, the package SELEX MDH is used in these tests and indeed S5066 GUI is just the name of one of its functions (fig. 8). This could also explain why the contents of the files are not intellegible: according to their public documentation (see links) it seems that a proprietary packet format is used, with CRC and other solutions to make the on-air transfer more robust.

fig.7

fig.8

Summarizing, S-5066 is a pure datalink protocol and it doesn't define ALE (Automatic Link Establishment) as S-4538 does. Either, a fixed frequency must be agreed upon, or it is possible to run S-5066 in conjunction with an external 2G-ALE system. As seen, in these test recordings there are no ALE sessions and once the frequency has been agreed by the peers (or known from CARBs broadcasts) the master initiates a link request and after the acknowledgment sent by the slave, the file transfer/exchange may start.

I want to thanks my friend Marco for the help with professional modem and for the screenshot of the "MDH Control Centre" sw package.

[1] half-duplex recordings 
A system anomaly is clearly evident in fig. 1a, after all ...these are just test transmissions.

You may see that the master start to transmit the second frame before to receive the first ACK sent by the slave. This should never happen since in the absence of ACK, and after a certain timeout, the master should close the link ("breaking" state).
Moreover, a collision happens since the two peers transmit at the same time (half-duplex violation). In this case the Data Carrier Detect (DCD) should be activated so that when it goes "high" it means that the channel is busy and, in this case, the slave must be silent just to avoid collisions.

[2] full-duplex recordings
Looking at the full-duplex tracks, after the peers linked they go in full-duplex split-frequency operation and then work in two different (formerly agreed) frequencies: this is not evident in fig. 1b since software Audacity represents the tracks in the time domain.

Full-duplex operations are described in the Annex C of STANAG-5066 profile (see the links above).

 
Useful Links:
BID-950 
SELEX MDH
Implementing Full Duplex with Data Rate Adaption in STANAG-5066  
STANAG-5066 Annex C