27 October 2018

STANAG-5066 Annex G, the ancestor to US MIL STD 188-110B and STANAG-4539

The other day I was still trying to figure out something from the S4539 8-ary burst transmissions, posted here, using Harris RF-5710A HF modem. Turning between the various operating modes provided I came across what at first glance seemed a strange waveform: i.e. 5066-G. As far as I know, STANAG-5066 is a protocol standard that does not define waveforms, so I went into detail to see what the hell it is.
STANAG-5066 Annex G presents guidelines and requirements for the use of the STANAG 5066 protocol profile with modems and waveforms at rates above 2400bps. Early draft versions of STANAG 5066 Annex G included a detailed specification for high-speed single-tone waveform and convolutional forward-error-correction coding with data rates up to 9600 bps, and this formed the basis for more than one vendor implementation of a commercial product.
Quoting Edition 3 #G.3.0 Implementation Guidance for STANAG 5066 Operation at Higher Rates : "It is clear that higher throughput will be available for the HF long-haul channel in near future (i.e, post 2000). What is not clear is the final form of the waveform standard or standards that will provide these data rates". Notice that the Edition 3 was promulgated on 30 march 2015 and the Annex G is still "information only", ie it is still not mandatory for the purposes of the communication minimum requirements.

Now look at the words "...in near future (i.e, post 2000)": clearly, Annex G has remained unchanged from the first editions of STANAG-5066 (dated before year 2000). Most likely the NATO groups responsible for the standardization preferred a separate STANAG to define the new waveforms since 5066 is a protocol standard, so Annex G is "frozen" and stands like a kind of ancestor to MIL 188-110B and STANAG-4539 (3200 to 12800 bps): these new waveforms used constellations and much of the waveform structure developed for Annex G and added further enhancements. 
Harris developed its implementation of 5066-G, you may find it among the operational modes of their RF-5710A HF modem:

The STANAG-5066 Annex G waveforms provide the highest possible data rates over conventional 3KHz HF channels. A single 1800 Hz sub-carrier is modulated at a constant rate of 2400 symbols per second. the type of modulation varies from QAM-64 to PSK-4 according to the data rate selected. Known data symbols are periodically inserted in the transmitted signal to allow fro adaptive channel equalization at the receiver. Convolutional coding FEC and Viterbi decoding are combine with interleaving to enhance the performance of the receive modem on fading HF channels. Data rates from 3200 bps to 9600 bps are supported together with long, double long, short, and zero interleaving options. An additional 12800 bps uncoded waveform is supported for line-of-sight applications. Automatic detection of the data rate and interleaver setting are provided in the receive mode.

By the way, I tried to demodulate the S4539 8-ary bursts using the 5066-G mode , despite obsolete. Surprisingly, the modem is responsive and distinguishes four different waveforms: 4800L (PSK-8), 8000L bps (QAM-32), 9600L and 12800 bps uncoded (QAM-64), the last two are detected less frequently (Fig. 1). It should be noted that when the same transmission is demodulated using 4539 or 110B you always get the same waveform 12800U.

Fig. 1

Annex G: Use of Waveforms at Data Rates Above 2400 bps (V1.2) 

22 October 2018

BPol vessels tracking using RSX.25 and GM2X00 waveforms (R&S PostMan II)

Quoting R&S PostMan II brochure "[...] another feature is the capability to track mobile stations by using the global positioning system (GPS). If a station is equipped with a GPS receiver, position data can be transmitted in addition to the actual data after analysis of the National Marine Electronics Association (NMEA) 0183 protocol [1]. The current position of and the route covered by each mobile station can thus be tracked from a command center".  The standard package includes an interface with scanned maps as below: in the left side a screenshot from PostMan and in the right side the correspondent map available from the web [2].

Fig. 1
After analyzing some recordings, I think this system is the one used by BPol (Bundespolizei See) for tracking their patrol vessels: positions are transmitted on HF to the ashore command center using RSX.25 protocol over GM-2X00 waveforms as discussed here.
After removing the GM-2X00 waveform and decoding the RSX.25 bitstream, it is possible to see the UUCP commands that are responsible for transferring a Bzipped file from BPOLBP23 (patrol vessel "Bad Düben", now out of service) to BPOLBPLEZSEE_HF (Operations Center HQ Cuxhaven).

Fig. 2 - RSX.25 bitstream

login: UBPOLBPLEZSEE_HF <MPL>2016-11-28 15:49:52+01,BPOLBP23;2016-11- BPOLBPLEZSEE_HF 15:49:52+01,BPOLBP23; 2016-11-28 15:47:45+01,BPOLBP26</MPL> Connected
Shere=BPOLBP23_HF SBPOLBPLEZSEE_HF -pz -vgrade=z -R -N07 ROKN07 Pyie
Uy  ¸ HN = #E D.05PK D.BPOLBP2C05PK uucp -C D.05PK 0666 "" 0x66
ƒ!Z)vL@¬“4m˜ßÐ{ f ã BZh91 ...

The UUCP session has been described in detail in this post, here it's more interesting to dwell on the piped commands uucp file | rsgpsadd

D.05PK D.BPOLBP2C05PK uucp -C D.05PK 0666 "" 0x66 
D.05PK = file to send
D.BPOLBP2C05PK = file name on slave (should be D.BPOLBP2505PK)
uucp = application requesting the file transfer
-C = file has been copied to the spool directory
0666 = mode of file on master for outgoing files
0x66 = 102 Bytes file size

rsgpsadd = command to be executed after the transfer, most likely stands for "R&S GPS Add". This program  will process the sent file D.05PK, probably unpacking it and store the lines into a database for the further graphic elaboration (map display).

here starts the D.05PK file which is sent in Bzip compressed way:
BZ = Signature (0x425A magic number)
h = Bzip2 (h is for Huffman coding)
9 =  increments of 100 kB block-size uncompressed

The compressed file, once unpacked, consists of two lines, each consisting of four values separated by commas "timestamp, vessel, latitude, longitude" as in the transmission heard on August 10 at 0806 in which the vessel Bayreuth (BPOLBP25), comunicates its position at times 0737 and 0757 (CEST):

2017-08-10 07:37:48 +02, BPOLBP25, 54.023927, 10.800935
2017-08-10 07:57:47 +02, BPOLBP25, 54.023125, 10.800987 

Some comments about these files.
GPS Devices can create a file saving the route they are taken and these files are usually stored in a format called NMEA 0183. Now, as can be seen from the mentioned brochure ("... after analysis of the National Marine Electronics Association (NMEA) 0183 protocol") I believe the 0183 format files cannot be read directly by PostMan so they must be converted into a new file format. Well, the format of the files piped to rsgpsadd command looks like the Waypoint file format. In the waypoint format each line contains a triplet of values separated by commas, these values are a text string (optional), a latitude value and a longitude value.  In the rsgpsadd files it seems that each waypoint line, consisting of the triplet "vessel, latitude, longitude", is preceded by a timestamp string that could be a header value added during the 0183-waypoint conversion. By the way, notice that R&S already developed the "GPS Route Converter" program (NMEA 0183 to waypoint files) to be used by the R&S SMU200A Vector Signal Generator [3]: maybe parts and knowledges from that software is used/embedded in this PostMan application, who knows?

Fig. 3

I tried to use some rsgpsadd commands, recordered on 26 August (2017), to trace the patrol vessel BPOL21 by using a .kml file and google Earth application [4]: these rsgpsadd commands update the position with a resolution of 10 minutes

2017-08-26 09:02:08 +02, BPOLBP21, 54.4225, 12.271833
2017-08-26 09:12:09 +02, BPOLBP21, 54.414833, 12.252
2017-08-26 09:22:09 +02, BPOLBP21, 54.4075, 12.2325
2017-08-26 09:32:09 +02, BPOLBP21, 54.4, 12.212833
2017-08-26 09:42:09 +02, BPOLBP21, 54.3925, 12.193167
2017-08-26 09:52:09 +02, BPOLBP21, 54.385, 12.173

below the map I obtained, the vessel was moving from right to left


[1]  https://www.nmea.org/.../nmea_0183_v_410.asp 
[2]  http://fishing-app.gpsnauticalcharts.com
[3]  https://cdn.rohde-schwarz.com/...GPS_route_converter.pdf
[4]  http://www.gpsvisualizer.com/map_input?form=googleearth

19 October 2018

use of the radio address format "RSPeer" for direct delivering of messages (R&S PostMan II)

This post can be considered as a continuation of the previous one in which the use of Rohde & Schwarz PostMan was shown: in that post, rsmail sending "classic" email over HF, was analyzed. In order to find other interesting RSX.25 sessions I started to look for my files and old recordings and finally I recovered an interesting transmission of the Italian Coast Guard (Guardia Costiera, say It-GC) on 12270.5 KHz/usb dated January 2016 consisting of the combined RSX.25 and GM-2X00 HF waveform.
Fig. 1
I probably went late on that transmission and so I probably lost the initial RSX.25 exchanges but after the demodulation of the signal - although weak - an interesting session came up that uses the Rohde & Schwarz address format "RSPeer" to send a file between two radios. The received signal does not have a good SNR but its interpretation is nevertheless possible, below the file I got after the removal of the GM2X00 HF waveform and RSX.25 demodulation:

c„%Ì, TNF8c19f176-c647-11e5-ab22-000c2940a7d1.tmp v˜ xŸ>"©  ' E €  1 1
CP940 RSPEER:CP940,Administrator   N €  à  1 O   € !  +*V    ,ØCµá ªG )@§Ñd€
+ ¤¾£ n Ý T  ROMA RSPEER ROMA,ROMA   0  'ROMA'  0  0   *)  ì  #  &)

file being sent (most likely, see below)

RSPeer address format addresses
CP940, Administrator = patrol vassel "DATTILO" CP-940, callsign IGUB [1]
ROMA, ROMA = Guardia Costiera HQ Roma, callsign ICI [2]

Quoting the paper ADP010679 [3] "The R&S address format RSPeer ensures the direct delivery of the message to the computer of the addressee, i.e. the message is physically available on the hard disk of the recipient, the usual detour to the SMTP server being avoided. This delivery procedure excludes any misuse of and unauthorized access to the mail traffic of a network and ensures that one's own information is secure. This type of addressing also minimizes the data exchange on the frequencies available and so eases the traffic load of the radio network."

Further and more interesting details emerge if the HDLC protocol decoding is used at data-link layer, i.e. just before the RSX.25 demodulation(!): in particular, the strings related to MicroSoft Exchange  are highlighted

940a7d1.tmp v˜   xŸ>"©      •þÿÿÿþÿû(F°€œøÞÿÿÿ6(F°àš›Þÿÿÿ
ÿ(F¿ 2   Ì, TNF8c19f176-c647-11e5-ab22-000c2940a7d1.tmp v˜  

xŸ>"©    tÑ(F°   ä    è € IPM.Microsoft Mail.Note 1 € €    
PIM 291000A GEN 16 ô €    à 6 (F± ' E €  1 1 CP940 RSPEER:CP940,Administrator
O € ! h›(F² 74F1198C47C6E511AB22000C2940A7D1 ý  þ    ÿ Z µ;ÂÀ,w ¡¼ 
 Úµ(F³ +*V    ,ØCµá ªG )@§Ñd€  Ñd€ + ¤¾£ n%Ý T ROMA RSPEER:ROMA,ROMA 'ROMA'

I believe the CP940 user is sending an outlook message in rich text format (RTF) encapsulated into Microsoft "Transport Neutral Encapsulation Format"(TNEF) [4] instead of sending an SMTP (HTML or Plain Text mode) email. The original filename is probably 940a7d1.tmp while the filename TNF8c19f176-c647-11e5-ab22-000c2940a7d1.tmp and the 32-char lenghth string 74F1198C47C6E511AB22000C2940A7D1 are something related to MS-Exchange TNEF encapsulation (I believe the TNF converted file and the key/identifier of the message).
For what concerns PIM 291000A GEN 16, it is probably the time when the message was prepared, expressed in the format: DayHourMinuteAnte(Post)Meridiem Month Year (something like PIM = Product Information Management ???), i.e. 29 January 2016 10.00 AM; indeed it matches the time I recorded the transmission, i.e. 29 Jan 2016 at 10.25 or sent 25 mins after its preparation (Fig. 2).

Fig. 2
That said, PostMan performs an "address gateway" to email networks (SMTP, X.400, MS-Mail) with different address formats (Fig. 3).


So far, I never heard 188-141 ALE messages form It-GC ships and ashore stations but I have a guess about it. Several times I copied on 12270.5 and 8196.5 KHz (frequencies operated by It-GC) R&S-ARQ 228.6Bd/170 "ALIS" selcalls such as:

Called address: 40
Pool size: 8
ALIS 2000: No
Ack: true
Followon type: External modem
Spectral diversity: Adaptive
Data rate: Fast
Data encryption: No (clear)
Rephase: false
Sending counter: 1

Assuming that 40 is the ALIS address of CP940, it could be that they use the ALIS selcall instead of the 188-141 mode, but it is only a speculation of mine without any confirmation.

https://yadi.sk/d/liJWgfG8jZGqXQ (OHDLC-layer.bin)

[1]  http://www.guardiacostiera.gov.it/.../scheda-dati-nave-dattilo-cp-940
[2]  http://www.mediasuk.org/archive/ici.html
[3]  http://www.dtic.mil/dtic/tr/fulltext/u2/p010679.pdf
[4]  http://www.fiction.net/blong/programs/tnef2txt/apptnef.txt 

15 October 2018

email over HF using RSX.25 and GM2X00 waveforms (R&S PostMan II)

I had already met the "combined" Rohde & Schwarz RSX.25 ARQ protocol + GM2X00 waveforms some time ago, in that case UUCP sessions were exchanged between German BPOL patrol boats and the ashore station server in order to transmit and store the positions of the vessels: the analysis is posted here
This time I was lucky to spot transmissions where RSX.25 ARQ protocol and GM2X00 waveforms are used by the R&S message handling system "PostMan II" to send emails between Italian GdF patrol boats and between patrol boats and ashore commands. Such transmissions, along with 188-141 2G-ALE messages exchanges, can be heard by monitoring some known frequencies such as 6450 (noised by close S4285 transmissions), 8190 (sometimes with co-channel interference from Saudi-AF and Israeli-Ny), and 12431 all usb. Unfortunately it could be a long and fruitless monitoring since emails transmissions are not frequent.

R&S PostMan II is a combined hardware and software product, the hardware platform is a communication server running on the Linux operating system and also controlling the connected radios. Notice that PostManII uses the advanced waveforms provided by GM2X00 HF modems since STANAG/MIL-STD waveforms cannot be used together with the RSX.25 protocol; if interoperability is needed the transmission method defined in STANAG-5066 protocol and 110A/4539 HF waveforms can be applied.

Fig. 1 - R&S PostMan II email gateway

Commands and associated files coming from PostMan, who sits at application layer, are segmented and encapsulated into RSX.25 frames which in turn are transported on air by GM2X000 HF waveforms; therefore in order to get the PostMan bitstream you need to demodulate the received signal, extract RSX.25 frames, remove their encapsulation and finally reassembly the segments into a single file (here termed "infoData.tmp"): what I got is shown in Figure 2.

Fig. 2 - infodata.tmp file, PostMan commands togheter with the (Bzip2 compressed) email file
Now let's have a close look to the first bytes of the Hex/ASCII coded infoData.tmp file, i.e. to the PostMan commands (Fig. 3).


OC0008 pm2mrs -CR D.000t 8 0666 simo@oltramonti.gdf.it 0x3da rs*ail -v2 -f simo@oltramonti.gdf.it simo@cagliari.gdf.it # Ú#º¦BZh94AY&SYèså  ...

R&S PostMan II commands
pm2mrs = PostMan II messenger(?) R&S
-CR  = (?)
rs*ail -v2 -f = rsmail -v2 (version2) -f (?)

email addresses
simo@oltramonti.gdf.it = sender, patrol vessel "Oltramonti"
simo@cagliari.gdf.it = recipient, ashore station Cagliari

Bzip2 4 bytes header
BZ = Signature (0x425A magic number)
h = Bzip2 (h is for Huffman coding)
9 =  increments of 100 kB block-size uncompressed

... compressed email file bytes follow (I did not unpack it!)

Some comments:
1) Since PostMan offers  e-mail, fax and file transfer, I think that the additional command rsmail (most likely R&S mail) that follows the pm2mrs invocation just specifies the email service.
2) In order to reduce the transmission time, both the emails text body and attached files always undergo data compression.
3) For what concerns the email addresses simo@<ALE-address>.gdf.it, it seems that each radio (with its own name, ALE address and PostMan address) belongs to one station so that each station has one unique e-mail domain name where the ALE-address is also the hostname. All the email addresses have the same local-part "simo": I do not know if it's the PostMan default username (just like "user" for Harris RF-6750 mail gateway) or if it's the special GdF entity/role/staff with delegation to the message handling system.
4) The uncoloured strings are not constant in the PostMan sessions I heard, they could be related to the underlying Linux OS layer or to some other parameters (eg, 0x3da could be a file length). Below the strings of another captured (and a bit distorted) PostMan session:
C004 pm2mrs   -CR Dû;004 06D5   simo@cappelletti.gdf.it 0x565 rsmail -v2 -f simo@cappelle tti.gdf.it simo@roma.gdf.it

As said above, PostMan command and files are segmented and encapsulated into RSX.25 frames which in turn are transported on air by GM2X000 HF waveforms. 
The RSX.25 protocol is the R&S adaptation of wired X.25 protocol to the HF radio channel. RSX.25 organizes the data to be transmitted in packets, which are successively transferred to the data modem. The packets contain a variable number  of  frames depending on radio-link quality and being adapted at regular intervals. RSX.25 has a typical 8-bit period (Fig. 4) with recognizable patterns and is visible once removed the overhead due the GM2X00 "advanced" HF serial waveform.

Fig. 4 - typical RSX.25 bitstream
GM2X00 HF waveforms are based on a PSK-8 constellation modulated at a symbol-rate of 2400Bd. The frame structure consists of an initial 192 symbol sequence followed by a data block consisting of 64-symbols frames each composed of 48 unknown (data) symbols + 16 known symbols (probe). The postamble, terminating the data block, has a structure which is basically the same as the one of the data frames but it contains a stop-code sequence instead of information data.

By the way, these are the R&S HF equipments used in the two stations (*):
P.V. 5 "Oltramonti" patrol boat [1]
- XK2500 500W, antenna dipole Whip 8 mt mod. STA80 + tuner FK855C3
- XK2100L 150W, antenna dipole + tuner HX002M1
"Cagliari" Aeronaval Group, Operational Control Room
- XK2900 1 KW, antenna HX002
- XK859C1 1 KW, antenna HX002

(*) these informations are publicly available from the GdF website:
[1] http://www.naviecapitani.it/.../GDF/G%205%20Oltramonti.htm

10 October 2018

NILE/Link-22 waveform #2 (S4539 Annex D)

STANAG-4539 HF Fixed Frequency TDMA transmission, used by NILE/Link-22, spotted on 7606.0 KHz/usb and followed from 0905z up to 0950z (October, 9). Modem for HFFF TDMA operations are described in Annex D of STANAG-4539.
The 270 symbols of the Media Code Frame in this samples are arranged according to the QPSK Traffic Waveform #2 (WF2) consisting of 8 sections with 18 symbols data blocks and 15/16 symbols mini-probes.

Other NILE/Link-22 waveforms, all characterized by having the same 112.5ms ACF value, are discussed here.

Fig. 1
Fig. 2

As in the table below, there are six sections consisting of 18+16=34 dibit symbols (ie 68 bits) and two sections consisting of 18+15=33 dibit symbols (ie 66 bits). Using the time shift cursor of the tool in Fig. 2 you may get the same order of the sections which is shown in the table.


9 October 2018

S5066 data transfer, scaling from 3200bps (QPSK/PSK8 on-air) up to 9600bps (QAM64)

8th October update: using "Wireshark" software to detect IP packets (IP over HF)

13378.0 KHz/USB, 0848z: S5066 data transfer using HF waveforms 110A & S4539 which scale from 3200bps (QPSK/PSK-8 on air) up to 9600bps (QAM-64); unid user/location. Notice in Figure 1 that QAM-32 constellation use multiple PSK rings to maintain good peak-to-average ratios, and the QAM-64 constellation is a variation of the standard square QAM constellation, which has been modified to improve the peak-to-average ratio.

Fig. 1 . HF waveforms constellations
The typical 1776 bits structure of S5066 is obtained after the removal of the HF waveforms overhead, in Figure 2 the bitstream is synchronized on the sync sequence 0x90EB. 

Fig. 2 - STANAG-5066 bitstream synched on 0X90EB
Looking at the 16-byte headers of the Data Transfer Protocol Data Units (D_PDU), we see that #0 (Data only)  is the used D_PDU type: this type is used for simplex data transfer of segmented C_PDUs with a Selective Repeat-Request (SRQ) service protocol. The peers in the link have the S5066 addresses: and

0x90EB D_PDU sync sequence
04 D_PDU type

10 50 56 91 01 01 65 source & destination address

90 EB 04 F6 3C E9 10 50 56 91 01 01 65 48 BA 85 ...
90 EB 04 F6 3B E9 10 50 56 91 01 01 65 08 C8 87 ...
90 EB 04 F6 3A E9 10 50 56 91 01 01 65 08 C8 88 ...
90 EB 04 F6 39 E9 10 50 56 91 01 01 65 08 C8 89 ...
90 EB 04 F6 39 E9 10 50 56 91 01 01 65 08 C8 8A ...
90 EB 04 F6 38 E9 10 50 56 91 01 01 65 08 C8 8B ...
90 EB 04 F6 37 E9 10 50 56 91 01 01 65 48 BA 8C ...
90 EB 04 F6 36 E9 10 50 56 91 01 01 65 08 C8 8E ...
90 EB 04 F6 35 E9 10 50 56 91 01 01 65 08 C8 8F ...
90 EB 04 F6 34 E9 10 50 56 91 01 01 65 08 C8 90 ...

The  D_PDUs payloads originate a group of files which have the same 38 bytes length initial structure consisting of a 33-byte pattern followed by a progressive 0xnn number and four 0x00 bytes:

00 07 99 42 EF 44 45 00 05 64 FF FF 40 00 FE 32 E6 B6 C0 A8 01 30 C0 A8 0E 30 00 00 04 89 00 00 00 18 00 00 00 00 4F ...
00 07 99 42 EF 44 45 00 05 64 FF FF 40 00 FE 32 E6 B6 C0 A8 01 30 C0 A8 0E 30 00 00 04 89 00 00 00 19 00 00 00 00 5A ...
00 07 99 42 EF 44 45 00 05 64 FF FF 40 00 FE 32 E6 B6 C0 A8 01 30 C0 A8 0E 30 00 00 04 89 00 00 00 1A 00 00 00 00 CC ...
00 07 99 42 EF 44 45 00 05 64 FF FF 40 00 FE 32 E6 B6 C0 A8 01 30 C0 A8 0E 30 00 00 04 89 00 00 00 1B 00 00 00 00 53  ...

In my opinion the first six bytes are the headers of C (Channel Access Sublayer) and S (Subnetwork Interface Sublayer) Protocol Data Units: 

00 07 99 42 EF 44

00 C_PDU type (0 = data) 
07 S_PDU type (0 = data)
99 S_PDU source & destination SAP IDs (1001 & 1001)

42 EF 44 S_PDU control and TDD fields

and the remaining 32 bytes, from 0x45 to 0x4F, could be the headers of the User Protocol data Units (U_PDUs) incoming from the client/application upper layer (Figure 3). Since the presence of a progressive number (0x18, 0x19,0x1A,...) it could be that the client message has been segmented into smaller U_PDUs before the subnet interface, but it's only a my guess.

Fig.3 - sublayers within STANAG-5066

SAP_ID stands for Service Access Point Identifier, it's a number in the range 0-15 and is equivalent to the “port” of the TCP protocol. In this case - according to S5066, the used Service Access Point should be the IP port (1001).

My friend and colleague j. sent me an email with his comments about this signal "I also analysed the last S5066 signal you posted on your page. It finally contains IP data packets (local addresses are -> in the direction -> The used protocol is ESP (IPSec). The other direction confirms the data using RCOP"
Well, it's possible to prepare an hex dump file by removing the  C & S headers (0x 00 07 99 42 EF 44) :

00 07 99 42 EF 44 45 00 05 64 FF FF 40 00 FE 32 E6 B6 C0 A8 01 30 C0 A8 0E 30 00 00 04 89 00 00 00 18 00 00 00 00 4F ...

and then process the obtained file using "wireshark" software. The results show the IP packet originally submitted to S5066 and thus to the HF network, i.e. IP over HF [1]

[1] https://www.isode.com/.../ip-over-stanag-5066.html