a difficult signal

Sometimes it may happens to come across signals for which - unless of know it a priori - it is difficult to correctly define the used modulation;  it's the case of the so-called "semi-modes", where some FSK modulations at certain conditions have the shape of phase manipulation just because such signals possessing both PSK and FSK issues. Such "dualism" is spread enough and concerns tightly connected modulations as (G)MSK and OQPSK as well as CPFSK and SDPSK. A signal sent me by a friend of mine just falls into this category.
Let's get some parameters of the signal such as bandwidth (Bw), baud rate (Br), and shift (Sh): as from Figure 1

Bw = 19000 Hz
Br = 16000 Bd
Sh =  8000 Hz

Fig. 1 - main parameters of the signal

Looking at Figure 2, from "Signals Analyzer - radioscanner.ru" [1], MSK modulation has a bandwidth of about 1.5*Br, GMSK Bw is lesser than this value (in it’s limit is very close to theoretical Br), and SDPSK Bw is more than 1.5*Br. Well, this signal has the value of Sh exactly = Br/2 while Bw is < 1.5*Br: so, judging by these results, it could be a GMSK signal.

Fig. 2 - difference between MSK and GMSK

Another specific feature of the so-called "semi-modes" is the spectrum of their second harmonic: looking at Figure 3, the second harmonic has two very clear and defined lines and the spacing between these lines is equal to Br. As from [1], this is the necessary condition for their identification, but not the sufficient one: indeed, also both SDPSK and OQPSK modes exhibit two spectral lines in the second harmonic. Please note that the carrier in the fourth degree is very weakly expressed, sometimes it is practically invisible at all.

Fig. 3

That said, as shown in Figure 4, some equalization/autocorrelation is necessary to bring out the carrier (a) so that the SA demodulator PLL can lock onto it: this way you have a clearer view of phase plane and constellations. The 4-ary constellation (b) and its transitions pattern (c) rule out the OQPSK mode (and GMSK too) since it should show an 8-ary like constellation but w/out zero-crossing transitions. The relative/differential view (Diff=1) show instead a two-state mode (d,e).
The above considerations suggest SDPSK (Simmetrical Diferential PSK) modulation, just like the one used for Orbcomm series sats [2]. Moreover, note the "Offset mode detected" warning that means a relative phase shift keying, aka offset keying! Indeed SDPSK is equivalent to π/2 DBPSK or PSK2 with phase rotation: ie, as shown by the transitions in absolute mode (c), SDPSK assumes that the phase is rotated by +π/2 for bit “0” and by -π/2 for bit “1” thus there is not a 180° turn.

Fig. 4 - phase plane and constellation of the signal being analyzed

So, while the mathematical relations among Bw/Br/Sh point to a GMSK modulation, phase plane and constellations seem to point to a SDPSK (or even CPFSK) modulation: the relative phase planes and constellations are shown in Figure 5 (the SDPSK and CPFSK signals are synthesized).

Fig. 5 - phase planes and constellations of  (synthesized) SDPSK and CPFSK signals

However, the comparison between the phase detector results shows a behavior more similar to a GMSK signal (Figure 6).

Fig. 6 - phase detector results (CPFSK, SDPSK, our signal)

Since such kinds of signals can be demodulated also as FSK, I tried both the SA universal PSK and  FSK demodulaors: the resulting bitstreams are shown in Figure 8, as you can see they are the same (the 15-bit period is due to the initial preamble).

Fig. 7 - SA universal PSK and MFSK demodulators

Fig. 8 - bistreams after PSK and FSK demodulations

 

To conclude, "There is a lot of information that proves that semi-modes are practically the same from a mathematical point of view but at the receiver side there is no a reliable and easy method to discern the exact type of modulation. However, if the signal has a good quality, there are some clues that can help tip the balance one way or another... even if they could be not the conclusive" my friend AngazU says.

https://disk.yandex.com/d/WwBwL6tD_CTwLw  (.wav signals and bistreams)

[1] http://signals.radioscanner.ru/info/item68/
[2] http://signals.radioscanner.ru/base/signal16/