install, run and test GNU Octave against T207 and CIS-11 streams

updated

This is just a quick how-to for Octave since some friends asked me for some tips about the installation and use of Octave with some analysis tools such as the ones for T207 and CIS-11 streams (...and hopely soon for the STANAG-5066 dissector).
GNU Octave is a high-level interpreted language, primarily intended for numerical computations. It provides capabilities for the numerical solution of linear and nonlinear problems, and for performing other numerical experiments. It also provides extensive graphics capabilities for data visualization and manipulation.
The GNU Octave language is largely compatible to Matlab so that most programs are easily portable. In addition, functions known from the C standard library and from UNIX system calls and functions are supported. C/C++ and Fortran code can be called from Octave.

The easiest way to install GNU Octave on Microsoft Windows is by using MXE builds. For the current release, both 32-bit and 64-bit installers and zip archived packages (.zip and .7z formats) can be found here under the Windows tab:
https://www.gnu.org/software/octave/download.html 
For executable (.exe) installers (the better way) the user can simply run the downloaded file and follow the on-screen installation prompts. It is recommended that the installation path does not include spaces or non-ASCII characters. Shortcuts to the program will be created automatically on the desktop: CLI for the command line and GUI for the graphic user interface, the latter (GUI) is the one that is normally used:

 The installation steps are shown below:

After the installation is finished you will have to create the main working directory, usually c/users/<your_name>/octave; this will be the the right place where to create the sub-directories to store the Octave .m scripts. For example, create the sub-directory /signal_analysis and download the file t207-test.txt to it, then create the sub-directory /m (within /octave/signal_analysis) and download the script T207_detect.m into the /m sub-directory (T207_detect.m is a tool to verify the presence of the so-called T207 "format"). Now double click on the GNU Octave GUI icon and Octave will start in its Command Window

Change the Current Directory from  /octave to /octave/signal_analysis by using the arrows of the file browser, and then run the two commands:

addpath ("./m")
T207_detect ("t207-test.txt",32,14)

The first command (addpath) tells Octave where to look for the invoked .m scripts, i.e. the su-b-directory /m where you previously downloaded the script T207_detect. The second command launches the T207_detect.m script which will process the file t207-test.txt: an ASCII-bit file containing a demodulated FSK bitstream, i.e. the file to analyze that you previously downloaded to the /signal_analysis sub-directory (the other parameter is the desired frame size, 14).

after few seconds (depending on the speed of your pc) you will see the result of the T207 detection:

After close the Figure 1 window, you may also see the results shown in the textual form. Now Octave is ready to accept a new command:

You can switch to the Editor Window in order to open and edit the scripts in the /m sub-directory

Well, in the same way as above, now you can analyse - for example - a bitstream to check the presence of the CIS-11 format:

1) download the .m script CIS11_detect.m to the sub-directory /octave/signal_analysis/m
2) downolad the files cis11-test1.txt  and  cis11-test2.txt to the sub-directory /octave/signal_analysis
3) switch to the Command Window and run the command  CIS11_detect against the two files:

In a few words:
/octave/signals_analysis     > will contain the files to be analyzed
/octave/signals_analysis/m > will contain the various Octave scripts

Well, now you are probably wondering what you have done.

1) An interpreted language (as for example BASIC or Java) is a kind of programming language whose implementations execute instructions directly and freely, without previously compiling a program into machine-language instructions: the "interpreter" will convert the the high-level language code into the code understand by the operating system (say assembly language or machine-language). 

2) Well, as said at the beginning of the post, Octave is something like a "software environment" which consists of a high-level interpreted language, the interpreter, several software "libraries" that you may link to your Octave code, editor and various utilities all grouped inside the graphic user inteface. Thus when you click the GUI interface you do nothing but load and run the Octave environment.

3) When you wrote the "T207_detect" command in the Command Window, you simply told the interpreter to read and execute the Octave code language instructions saved in the "T207_detect.m" file, passing it some parameters such as the name of the data file to be examined (t207-test.txt).

4) Strictly, T207_detect.m is not indicated as a "program file" but rather as a "script file", ie a file containing any sequence of Octave commands (say instructions). It is read and evaluated just as if you had typed each command at the Octave Command Window prompt, and provides a convenient way to perform a sequence of instructions (you can't do that with a compiled language such as C). 

5) At the end of the day, T207_detect.m is nothing else that a sequence of Octave instructions that implement the algorithm described here in order to check if all the 14-bit sequences of a file verify the T207 checksum: even on large files the computer will take a few seconds, it would take hours, beer and coffee (lots of coffee ) if you do it by hand. 

Yes I know, that's a very basic approach to Octave but I hope it will allow you to start to work with some useful scripts, understand how it works... and maybe start to learn Octave. By the way, two important resources:
https://wiki.octave.org/Octave_for_Microsoft_Windows
https://it.wikipedia.org/wiki/GNU_Octave

CIS-14 FSK 100Bd/2000 (CIS 3x100)

Interesting 100Bd/2000 FSK signal spotted on 16059.5 KHz (CF), most likely a Russian Governative transmission. This signal is not a separate one, actually it must be considered as part of the CIS 3x100 "system" visible on the left in figure 1. Indeed, the main station operates a VFT 3x100bd/1440 signal and works simultaneously three outstations which in turn run the FSK 100Bd/2000 waveform (outstations are usually not far apart in frequency).

Fig. 1

As well as the mainstation (VFT 3x100Bd/1140), outstations too use CIS-14/T-207 (as from the analysis of the demodulated stream in figure 2); all use the same checksum mode [0312].

Fig. 2

A clearer illustration of this "system" is visible in the spectrum of figure 3, the IQ recording of which was provided by my friend KarapuZ [1] who discussed the signal here.

Fig. 3

https://yadi.sk/d/__AKCwIuMGF6gw

[1] https://yadi.sk/d/XeLE8tAX70PAKA (~100 MB) 

demodulating sync F7B mode transmissions using F7B.exe software

F7B signal, apparently a "classic" MFSK-4

My friend cryptomaster coded a program (F7B.exe) for demodulating synchronous F7B, it can be freely used and distributed for the analysis of such signals. Unfortunately, SA program parses these transmissions as classic MFSK-4 but that is not what actually is on-air: indeed,  F7B transmission mode deals with signals consisting of two independent but synchronous channels CH1 CH2 each carrying teletype signals (FSK2 on 2 independent channels).

F7B.exe can be downloaded from here:

http://www.radioscanner.ru/files/download/file21446/f7b_new.rar
https://yadi.sk/d/ZcsslwVpIwTpmg 

Warning: F7B.exe is coded to run under win/32-bit systems so you may encounter a comdlg32.ocx error (component not correctly registered) when try to run it under x64 systems, here is how to resolve comdlg32.ocx missing error:

https://www.swisssalary.ch/.../register-comdlg32.ocx-on-x64-win7

F7B.exe needs as input file the  raw MFSK-4 stream produced by the demodulator of SA and output two ASCII-Bit files CH1.txt and CH2.txt in the same directory of the input file:

Moreover, the Octave script T207_detect.m  has been used for the check of T207/CIS-14 mode [1]: it can be freely downloaded from here (you will need GNU Octave package [2] to run the script):
https://github.com/hcab14/.../T207_detect.m
the Octave script T207_detect_e.m also extracts the two world and bit interleaved channels:

https://yadi.sk/d/zsCD73C9DZpHPQ

I tested the demodulator on the two  F7B modes 100Bd/1000 and 96Bd/500, both actively used by Ukrainian Nets; by the way, two preliminary observations:

1) usually, the two F7B channels CH1 CH2 are T-207 secured according to the CIS-14 mode [1]. Since CIS-14 consists of two independent 5-bit MTK-2 channels (transmitted in 14-bit frames), it turns out that those F7B transmissions may carry up to four independent 5-bit channels (CH1a, CH1b, CH2a, CH2b);

2) both those F7B waveforms have their FSK2 counterparts.

I also tested F7B.exe on the Rockwell TE-204, a "special" time-frequency diversity FSK4 mode most commonly used by Allied Air Forces as an air-to-ground messaging system as well as in ground and naval applications.

100Bd/1000 F7B

Fig. 1 - 100Bd/1000 F7B

100Bd/1000 F7B transmissions occupies about 3000Hz bandwidth with the four tones at -1500, -500, +500, +1500 Hz respect to the center frequency. As said above, 100Bd/1000 F7B transmissions may carry up to four independent 5-bit channels. In this regard, it's worth noting in figure 2 that - at least in this sample - the two F7B channels adopt different CIS-14 checksums: mode 20 (0312) in channel CH1 and mode 3 (3120) in channel CH2: who knows, maybe the two F7B channels carry the same data but with different checksums just to increase the redundancy of the system. Note that CIS-14 100Bd/1000  (the FSK2 counterpart) uses the checksum mode 20.

Fig. 2 - two 100Bd/1000 F7B channels, each transporting two CIS-14 channels (CH1a, CH1b, CH2a, CH2b)

96Bd/500 F7B

Fig. 3 - 96Bd/500 F7B

96Bd/500 F7B transmissions occupies about 1500Hz bandwidth with the four tones at -750, -250, +250, + 750 Hz respect to the center frequency. The same earlier conclusion applies: that is since CIS-14 consists of two independent channels,  96Bd/500 F7B transmissions too may carry up to four independent 5-bit channels.

I don't know if it's a mere coincidence, anyway it's interesting to note in figures 4,5 that only one channel transports data, as well as the CIS-14 96Bd/500 (the FSK2 counterpart) does: i.e., either 96Bd/500 F7B either 96Bd/500 FSK2 use only one of their two "available" channels.  Both the waveforms use the CIS-14 checksum mode 20.

Fig. 4 - two 96Bd/500 F7B channels, only one of the two available CIS-14 channels is used

Fig. 5 - CIS-14 96Bd/500 FSK2 channels, only one of the two available 5-bit channels is used

In this transmission the two channels use different EOT signaling sequences (figure 6)

Fig. 5

Rockwell TE-204 FSK4
Although TE-204 does not definitely use the F7B mode, I decided to check it since regards two FSK2 channels. Indeed, TE-204 transmits the "mark" on 935 Hz for 6.67 msec period followed by a replicated 6.67 msec "mark" at 1815 Hz. Similarly, the "space" is transmitted at 1375 Hz for 6.67 msec followed by a replicated 6.67 msec "space" at 2255 Hz (figure 6). This "mode" provides an in-band frequency  and time diversity function  (thus the speed is the half of the measured one). As for above, from the perspective of the  data-transfer, the modem works as a 75Bd/880Hz FSK2 modem.

Fig. 6

Well, decoding separately the two FSK2 channels (935-1375, 1815-2255) we get obviously two time-shifted similar streams (figure 7)

Fig. 7 - the two TE-204 demodulated FSK2 streams

Demodulating it as if it were an F7B signal we get "01" sequences in channel CH1 and  (expected) duplicated data in channel CH2 (figure 8)

Fig. 8

https://yadi.sk/d/f0U80pRg3F58iA  (F7B signals)

[1] https://i56578-swl.blogspot.com/.../redefining-t-207-checksums.html
[2] https://www.gnu.org/software/octave/

CIS-14 FSK 100Bd/1000 (prob. Russian AF)

FSK transmission heard today on 13994.8 KHz (cf) at 0930z, contents are T-207 encrypted. Possibly from Russian AF.

https://yadi.sk/d/ri_lmpC0lMkQlg

CIS-14 FSK 100Bd/500

Yet another CIS-14 format 100Bd/500 FSK signal spotted on 9075.0 CF around 0810z (11 Sep) with good SNR. User is likely from Kaliningrad Oblast, contents are encrypted with T-207 system.

https://yadi.sk/d/Y29ujJhDt-EG9g
https://yadi.sk/d/xYkIjgOZXOYErQ

CIS-14 FSK 96Bd/500

Although the name FSK 96Bd/500, the used shift of this sytem is 480 Hz. Speed is 96Bd and the two 5-bit data words are bit-interleaved according to the CIS-14 format; first two bits are the system bits: t_a and t_b: 0 = traffic, 1 = idle (or no traffic). As you can see in the demodulate stream, only channel b transports data (figure 1). It's probably a test transmission since data are in clear-text mode and consist of a "classic" Russian test sequence "GA VIL BY CITRUS? DA,NO FALX"  (figure 2), something like "the brown fox jumps over the lazy dog".

Fig. 1

Fig. 2

Another example of a CIS-14 96Bd/500 bit-interleaved is shown in figure 3

Fig. 3

https://yadi.sk/d/4LlYlL_cIb8Myw

https://yadi.sk/d/jmBxGrfSAJjpJA

redefining T-207 (CIS-14) checksums

T-207 (T-207, T-207 2M "VIKTORIA" - Soviet teletype encryption device) is a multiplexed two channels "system" that is used in several CIS waveforms. Since the lack of official documentation it's difficult to say much more about the T207: guys from radioscanner talk about "equipment" as a in-line ciphering device while ex DDR STASI archives refer to T207 as an "encryption algorithm".
CIS-14 (also known as as TORG 14) is a designation of a transmission mode: a full duplex system using FSK at several speeds (42.1Bd, 47.5Bd, 48Bd, 50Bd, 70.5Bd, 72Bd, 83.3Bd, 84.21Bd, 94.11Bd, 96Bd, 100Bd, 144Bd, 192Bd, 200Bd, 288Bd, ...) and shifts. Data of two independent data channels can be processed; they are in MTK-2 alphabet (Russian [Cyrillic] Third-shift ITA-2, sometimes also called "ITA-2 Cyrillic M2") thus have 5 bits per character, but are transmitted in 14-bit frames, each containing two characters.
As shown in figure 1, the data code words (A in the figure) of the two channels are amended with two leading "channel state" bits and then either word-interleaved (case B) or bit-interleaved (case C). Two parity bits are calculated over the complete 12-bit frame generated and expand it to the final 14-bit frame. The two bits indicating the channel state signify whether the channel contains traffic(bit = 0) or idle (bit = 1) sequences at the moment.

Fig. 1 - 14 bit frame (from R&S Manual of transmitting methods)

Additionally, a variant of CIS-14 has been observed using frames of 28 bits. As can be seen in figure 2, after having established the 14-bit frame(s) (B) form the datawords (A) as explained above, two of these frames are bit-interleaved (C) to the new28-bit frame.

Fig. 2 - 28 bit frame (from R&S Manual of transmitting methods)

Note that although T207 is "hardware" while CIS-14 is a transmission mode, I use T207 in this blog as an implicit reference to CIS-14.

software tools (download)
- The Octave script T207_detect.m  has been used for the check of T207/CIS-14 mode:
T207_detect.m
- The Octave script T207_detect_e.m also extracts the two world and bit interleaved channels:
https://yadi.sk/d/zsCD73C9DZpHPQ
(the two Octave scripts are coded by me and Christoph, you will need GNU Octave package [1] to run them)
- The software CIS14-C.exe (coded by cryptomaster) can be used to etract the two 5-bit channels from a 10-bit stream C-interleaved:
https://yadi.sk/d/IfdhHvf3mMcZXQ

As said in a previous posts, T207 detection had to be manually spotted by processing the demodulated bitstream and checking if it matches the criteria described in this post in radioscanner forum: the Octave scripts are now improved and detects the presence of T207 checksums in a given bit stream and for each permutation of the checksum bits.  T207_detect scripts are very useful since encrypted CIS-14 messages have ACF=0 and anonymous demodulated streams, clear-text messages instead may be recognized as CIS-14 by the "solid" columns of the channel state bits.

I run the script against several waveforms and the results are very interesting.So far, I found two checksum modes termed "3" or [3120] and "20" or [0312]:

  

T207/CIS-14 verified waveforms (so far)
(note that some waveforms  can be coded with both the two checksums)

checksum mode 3 [3120]:
VFT 3x100Bd/1440, VFT 6x100Bd/120
FSK 50Bd/1000, FSK 100Bd/500 
F7B 100Bd/1000 (on one channel)

checksum mode 20 [0312]:
VFT 3x100Bd/1440, VFT 6x100Bd/120
FSK 50Bd/1000, FSK 96Bd/500, FSK 96Bd/1000
FSK 100Bd/500, FSK 100Bd/1000, FSK 100Bd/2000
F7B 96Bd/500 (on one channel), F7B 100Bd/1000 (on one channel) 

T-207 2M "VIKTORIA"

[1] https://www.gnu.org/software/octave/

CIS-14 FSK 96Bd/1000

The characteristic of CIS-14 FSK-2 is the 14-bit length frame, in this recording shift is 1000Hz and speed 96 Bd. This signal has been spotted on 8009.8 KHz (cf) on 0624 UTC.

fig. 1

fig. 2

The transmission is encrypted by the T-207 on-line ciphering device, as shown in fig. 3

fig. 3

T-207 is quite old and it is used by CIS Mil as well as by Ukrainian Mil or other Ukrainian Governative users such as Diplomatic and/or Intelligence services: allegedly as a Soviet communication legacy.

https://yadi.sk/d/yNCL9o7quBWZN 

T-207 recognition in CIS VFT systems

Some days ago I heard some CIS VFT systems and in particular one of them, a six  100Bd/120Hz channels, caught my attention. I already logged it but sent its main parameters as speed, modulation and shift, to my friend Karapuz asking if he knew the real name of that signal or the name of the modem. He told me that in last January he had just the same receptions and pointing an interesting discussion in radioscanner.ru about the encryption/coding used in such signals: T-207.  Although radioscanner is entirely in Russian, reading the opinion of the expert analyzers from this forum was interesting and I could figure out how detect the T-207 presence. In this post I describe the way I sought its "signature" in some CIS VFT signals as:

a) 3 x 100Bd/1440Hz VFT system
b) 6 x 100Bd/120Hz VFT system

replicating the experiences seen in radioscanner.ru and getting the expected results. 
By the way, these VFT systems are easy to receive (with good strength, at least here in JN52) on 13-16 MHz USB bands, mainly during the morning  and seldom during weekends.
 
T-207 detection has to be manually spotted by processing the demodulated bitstream and checking if it matches the criteria described in the cited post. We have first to choose a 14 bit period for the bistream and then focus on the first 12 positions and count the amount of "1" symbols:

- if the amount counts 2 or 6 or 10: the last two symbols (13th and 14th bits) must be 10

- if 3 or 7: 00

- if 4 or 8: 11

- if 5 or 9: 01

In case the sum is 0, 1, 11 and 12, it can be assumed that the last two symbols will be 11, 01, 00 and 11, respectively. These rules are shown in tab. 1.

Tab.1 - T-207 criteria

Since the above rules act presumably as a synchronization mechanism, the signal will be decoded and decrypted once removed the columns 13 and 14.

 a) 3 x 100Bd/1440Hz VFT

fig. 1 - 3 x 100Bd/1440Hz VFT

In this signal we have three channels modulated at 100Bd and a pilot tone at ~3300 Hz (characteristic feature of Russian systems). Every channel has a 1440 Hz shift and 100 Baud speed, channels are separated by 480Hz steps and interleaved as in figure1.

In my test I used the lower channel (fig. 2).
 

fig. 2

The obtained bitstream must be processed using the right/left shift (one bit at time) and sometimes the negative polarity:  criteria of Tab. 1 must be checked in all the rows at each shift-step, in case of fails we go on shifting. Unless possible interferences and demodulator errors, I confirmd the T-207 signature (fig. 3).
 

fig. 3

https://yadi.sk/d/SlYz9x6JllZhDg

b) 6 x 100Bd/120Hz VFT

Fig.4

the 6 x 100Bd/120 system (a variant of the 3 x 100Bd/1140 system) allows six independent channels, each of them exhibits 440 Hz shift and 120 Baud speed: in this sample the one-of-six mode is used. T-207 signature was found after processing the demodulated bitstream in the usual way (figs 5,6).

fig.5

fig.6

https://yadi.sk/d/wqahxt8RDXEBoA