RF Signal Models & Visualization

PODCAST: SCYTHE Radio Frequency (RF) Signal Intelligence system, focusing on machine learning models for signal analysis and visualization tools. Specifically, the various neural network architectures, including Convolutional Neural Networks (CNNs), Long Short-Term Memory (LSTM) networks, Residual Networks (ResNet), and Transformer models, designed for classifying different types of RF signals. It also presents a hierarchical classification approach that uses a general model followed by specialized models for more precise identification. Furthermore, the source code details a LatentAggregator for combining spectral and packet metadata, and a comprehensive SignalVisualizer class for plotting RF spectrums, waterfall displays, modulation characteristics, and signal features.

The sources describe several machine learning (ML) models designed for analyzing and classifying RF signals, each employing a distinct architecture tailored to different aspects of signal data.

Here’s how each model specifically analyzes and classifies RF signals:

• SpectralCNN (Flexible CNN model for classifying spectral images of RF signals):
  • Input: This model is designed to classify spectral images of RF signals. The input x to its forward method is processed by 1D convolutional layers.
  • Analysis Process:
    1. The signal data first passes through three sequential blocks, each consisting of a 1D Convolutional layer (conv1, conv2, conv3), followed by Batch Normalization (bn1, bn2, bn3), a ReLU activation function, and finally a Max Pooling layer (pool). This process progressively extracts features and reduces the dimensionality of the spectral data.
    2. After these convolutional and pooling operations, the data is flattened (x.view(x.size(0), -1)) to prepare it for the fully connected layers.
    3. The flattened features then go through a fully connected layer (fc1) with a ReLU activation, followed by a Dropout layer (dropout) for regularization.
  • Classification: The final output is generated by a second fully connected layer (fc2), which projects the processed features onto the num_classes dimensions, yielding classification scores.
• SignalLSTM (Flexible LSTM model for classifying time-series RF signal patterns):
  • Input: This model is built for classifying time-series RF signal patterns. The input x is expected to have a shape of [batch_size, seq_len, input_size], where input_size defaults to 2 (likely representing I/Q components).
  • Analysis Process:
    1. The time-series input is fed into an LSTM layer (lstm), which processes the sequence data, capturing temporal dependencies and patterns.
    2. The output of the LSTM then passes through an Attention mechanism (attention). This mechanism computes attention_weights using a softmax function, allowing the model to focus on the most relevant parts of the time series.
    3. An attention_output is generated by taking a weighted sum of the LSTM’s outputs based on these attention weights. This effectively aggregates information from the sequence, emphasizing important time steps.
    4. This aggregated output then goes through a fully connected layer (fc1) with ReLU activation and a Dropout layer (dropout).
  • Classification: A final fully connected layer (fc2) outputs the classification scores across the num_classes.
• TemporalCNN (Temporal 1D CNN for classifying time-domain RF signals):
  • Input: This model is designed for time-domain RF signals. It expects input x to be in the [batch, channels, seq_len] format, but it can automatically transpose the input if it’s provided as [batch, seq_len, channels]. input_channels typically defaults to 2 (e.g., I/Q data).
  • Analysis Process:
    1. The (potentially transposed) input x undergoes a series of three convolutional blocks, similar to SpectralCNN. Each block consists of a 1D Convolutional layer (conv1, conv2, conv3), Batch Normalization (bn1, bn2, bn3), a ReLU activation function, and a Max Pooling layer (pool). This extracts features from the temporal signal data.
    2. After feature extraction, the output is flattened to a 1D vector (x.view(x.size(0), -1)).
    3. This flattened representation is then fed into a fully connected layer (fc1) with ReLU activation, followed by a Dropout layer (dropout).
  • Classification: The final output, representing the classification scores, is produced by a second fully connected layer (fc2).
• ResNetRF (ResNet-style model for RF signal classification):
  • Input: This model is generally for RF signal classification, implying it processes 1D signal data (e.g., spectral or temporal) with an input channel of 1.
  • Analysis Process:
    1. The input first goes through an initial convolutional layer (conv1), Batch Normalization (bn1), ReLU activation, and a Max Pooling layer (maxpool).
    2. The core of the ResNetRF model lies in its Residual Blocks (_make_residual_block function creates these layers). Each ResidualBlock contains two 1D convolutional layers, each followed by batch normalization and ReLU activation, but crucially includes a shortcut connection (identity) that adds the input of the block directly to its output. This structure helps in learning deeper features by alleviating the vanishing gradient problem in deep networks.
    3. After passing through multiple layers of these residual blocks (layer1, layer2, layer3), the features are aggregated using Adaptive Average Pooling (avgpool), which reduces each feature map to a single value.
    4. The output is then flattened.
  • Classification: The final classification scores are generated by a fully connected layer (fc).
• SignalTransformer (Transformer model for RF signal classification):
  • Input: This model uses a Transformer architecture for RF signal classification, taking input x with shape [batch, seq_len, features]. The input_dim (number of features per time step/token) defaults to 258.
  • Analysis Process:
    1. The input features are first transformed into a higher-dimensional representation by an embedding layer (embedding).
    2. Positional Encoding (pos_encoder) is then added to these embeddings. This is critical for Transformer models as it injects information about the relative or absolute position of the features within the sequence, which is otherwise lost due to the permutation-invariant nature of the attention mechanism.
    3. The enriched embeddings are processed by a Transformer Encoder (transformer_encoder), which consists of multiple TransformerEncoderLayer instances. These layers utilize multi-head self-attention mechanisms to weigh the importance of different parts of the input sequence when processing each element, capturing long-range dependencies effectively.
    4. After the Transformer encoder, Global Pooling (mean over the sequence dimension) is applied to aggregate the information from all tokens in the sequence into a fixed-size representation.
  • Classification: A final fully connected layer (fc) then maps this aggregated representation to the num_classes to produce the classification scores.

In all these models, once the final layer produces raw scores (logits), these are typically converted into probabilities using a Softmax function. The class with the highest probability is then chosen as the final classification, and its probability represents the confidence of that classification. The HierarchicalMLClassifier extends this by initially classifying with a general model and then, if confidence is high, applying a more specialized model for refinement.

The primary methods for visualizing and interpreting complex RF signal data, as described in the sources, involve generating various types of plots and extracting key characteristics, as well as employing specialized models for anomaly detection.

Here are the key methods:

• Spectrum Plotting:
  • What it visualizes: This method plots a spectrum with frequencies against power in dB. It can include optional peak markers and annotations for specific frequency bands. Frequencies are often converted to MHz for readability.
  • How it aids interpretation: It helps in identifying the power distribution across different frequencies. Users can see peaks, which often indicate the presence of a signal, and interpret their power levels. Annotations can highlight common frequency bands (e.g., FM Radio, VHF Amateur, GPS) to provide context for the observed signals.
• Waterfall Displays:
  • What it visualizes: A waterfall display shows a series of spectrum lines over time, with the latest spectrum typically at the top. It uses color intensity to represent signal power, often with custom colormaps like “rf_quantum” or “thermal”.
  • How it aids interpretation: This visualization is crucial for observing how signals change over time. It allows for the identification of intermittent signals, frequency hopping, or signal fading, providing a dynamic view of the RF environment. The update_waterfall method continuously adds new spectrum lines to build this temporal representation.
• Modulation-Specific Visualizations:
  • What it visualizes: These plots are tailored to reveal characteristics of the signal’s modulation type. They extract and display I (in-phase) and Q (quadrature) components, amplitude, and phase information from complex IQ data.
  • How it aids interpretation:
    • For FM/PM (Frequency/Phase Modulation), plots include IQ plots, unwrapped phase, and instantaneous frequency, alongside amplitude. This helps in understanding phase and frequency deviations.
    • For AM (Amplitude Modulation), the focus is on amplitude envelope, I, and Q components over time, in addition to the IQ plot. This highlights amplitude variations.
    • For PSK/QAM (Phase/Quadrature Amplitude Modulation), visualizations include IQ constellations, phase histograms, and eye diagrams for I and Q components. These are vital for analyzing digital signal quality, symbol timing, and symbol detection.
    • For unknown modulation types, a generic display of IQ plot, I vs. time, Q vs. time, and amplitude vs. time is provided.
• Signal Characteristic Plots:
  • What it visualizes: This method generates plots based on extracted signal features (e.g., bandwidth, peak power, mean power, spectral flatness, crest factor, modulation confidence, skewness, kurtosis, variance, AM/FM index, IQ correlation).
  • How it aids interpretation: It provides a statistical summary of the signal, allowing for quantitative interpretation of its properties. For example, bar charts can show signal statistics, modulation confidence for a detected modulation type, and higher-order statistics (like skewness and kurtosis), which are important for understanding the shape and distribution of signal components.
• Latent Aggregation and Anomaly Detection:
  • What it interprets: While not a direct visual method, the LatentAggregator interprets RF signal data by combining FFT (spectrum), Ghost Imaging, and packet metadata. It uses a model to compute a reconstruction error score for spectral data.
  • How it aids interpretation: This process is used for anomaly detection. A high reconstruction error score indicates a “ghost anomaly,” suggesting an unusual or potentially malicious signal. It also performs orbital mimic detection, analyzing if a signal’s fingerprint matches known orbital patterns. This provides alerts about unusual or impersonating RF signals based on computed confidence scores.

In summary, the models and tools described provide a comprehensive suite for analyzing RF signals, ranging from raw spectral and temporal data visualization to high-level feature extraction and anomaly detection, crucial for understanding complex RF environments.

Based on the sources and our conversation history, the main visualization types for complex RF signal data are:

• Spectrum Plotting: This method involves plotting a spectrum with frequencies against power in dB. It can include optional peak markers and annotations for specific frequency bands, with frequencies often converted to MHz for better readability. This visualization helps in identifying the power distribution across different frequencies and observing signal peaks and their power levels. Custom annotations can be added to highlight frequency bands, such as FM Radio, VHF Amateur, Weather Radio, UHF Amateur, ISM 915, GSM 900, ADS-B, and GPS L1.
• Waterfall Displays: This visualization shows a series of spectrum lines over time, where the latest spectrum is typically at the top. Signal power is represented by color intensity, often using custom colormaps like “rf_quantum” or “thermal”. Waterfall displays are crucial for observing how signals change over time, allowing for the identification of intermittent signals, frequency hopping, or signal fading, thus providing a dynamic view of the RF environment.
• Modulation-Specific Visualizations: These plots are tailored to reveal characteristics based on the signal’s modulation type. They extract and display I (in-phase) and Q (quadrature) components, amplitude, and phase information from complex IQ data.
  • For FM/PM (Frequency/Phase Modulation), plots include IQ plots, unwrapped phase, instantaneous frequency, and amplitude over time.
  • For AM (Amplitude Modulation), the focus is on the amplitude envelope, I, and Q components over time, in addition to the IQ plot.
  • For PSK/QAM (Phase/Quadrature Amplitude Modulation), visualizations include IQ constellations, phase histograms, and eye diagrams for I and Q components, which are vital for analyzing digital signal quality and timing.
  • For unknown modulation types, a generic display includes IQ plot, I vs. time, Q vs. time, and amplitude vs. time.
• Signal Characteristic Plots: This method generates plots based on extracted numerical signal features, such as bandwidth, peak power, mean power, spectral flatness, crest factor, modulation confidence, skewness, kurtosis, variance, AM index, FM index, and IQ correlation. These plots, often presented as bar charts, provide a statistical summary and quantitative interpretation of the signal’s properties.

Spectrum data is primarily visualized through two main methods: Spectrum Plotting and Waterfall Displays. These visualizations allow for the interpretation of power distribution across frequencies and the dynamic behavior of signals over time.

Here’s how each method visualizes spectrum data:

• Spectrum Plotting:
  • What it visualizes: This method plots a spectrum with frequencies against power in dB. Frequencies are often converted to megahertz (MHz) for better readability.
  • Key elements and interpretation:
    • It typically shows the main spectrum line, representing the signal’s power at different frequencies.
    • Optional peak markers can be added to highlight strong signal presences. These markers can also be annotated with the frequency of the peak.
    • Annotations are crucial for providing context. These can include:
      • Specific frequencies marked with vertical lines and labels.
      • Frequency bands (e.g., FM Radio, VHF Amateur Radio, NOAA Weather, GSM 900, GPS L1, ADS-B) highlighted with colored spans, indicating common uses or regulatory allocations.
    • This visualization helps in identifying the power distribution across different frequencies, allowing users to easily discern signal peaks and their power levels, and understand which frequency bands they occupy.
• Waterfall Displays:
  • What it visualizes: A waterfall display shows a series of spectrum lines stacked over time, typically with the most recent spectrum appearing at the top. Color intensity is used to represent signal power, often employing custom colormaps such as “rf_quantum” or “thermal”.
  • Key elements and interpretation:
    • Each horizontal line in the display represents a spectrum captured at a specific moment in time.
    • The evolution of the spectrum over time is shown vertically, providing a dynamic view of the RF environment.
    • This visualization is crucial for observing how signals change over time. It allows for the identification of intermittent signals, frequency hopping, or signal fading, which might not be apparent from a static spectrum plot.
    • The update_waterfall method continuously adds new spectrum lines to build this temporal representation.

The SignalTransformer is a Transformer model designed for RF signal classification. It is one of several machine learning models, including SpectralCNN, SignalLSTM, TemporalCNN, and ResNetRF, that can be created to classify RF signals based on their characteristics.

The SignalTransformer is initialized with parameters such as input_dim, num_classes, nhead, and num_layers, indicating its role in categorizing signals into a defined number of classes, typically 11 by default. Its forward method processes input data through an embedding layer, positional encoding, and a Transformer encoder, finally passing the output through a classification layer. This architecture is designed to identify and categorize different types of RF signals.

The SpectralCNN is a flexible CNN model designed for classifying spectral images of RF signals.

Specifically:

• It is a type of machine learning model intended for RF signal classification.
• It is initialized with a num_classes parameter, defaulting to 11, which represents the number of different signal categories it can classify into.
• The SpectralCNN processes input data, typically spectral images, through convolutional layers, pooling layers, batch normalization, and fully connected layers to perform its classification task.
• It can be used as a general model for initial classification within a hierarchical classification system. When loaded, it can adapt to different numbers of classes than it was originally trained with.

The TemporalCNN is designed to classify time-domain RF signals.

It is explicitly described as a “Temporal 1D CNN for classifying time-domain RF signals”. Similar to other classification models discussed, such as SpectralCNN and SignalTransformer, the TemporalCNN is initialized with a num_classes parameter, which defaults to 11, indicating it categorizes signals into a predefined number of classes. Its architecture includes convolutional layers, pooling layers, batch normalization, and fully connected layers to process time-domain input data for classification. The forward method specifically handles input data in the shape of [batch, channels, seq_len].

The SignalLSTM is designed to classify time-series RF signal patterns.

Similar to other machine learning models discussed, such as SpectralCNN, SignalTransformer, TemporalCNN, and ResNetRF, the SignalLSTM is used for RF signal classification. It is initialized with a num_classes parameter, which defaults to 11, indicating its role in categorizing signals into a predefined number of classes. The model processes input data, which is expected to be in the shape [batch_size, seq_len, input_size], through LSTM layers, an attention mechanism, and fully connected layers to perform its classification task.

The LatentAggregator‘s primary function is to combine Fast Fourier Transform (FFT) data, Ghost Imaging results, and Packet Metadata into a single latent fusion layer. It serves as an integration point for various types of RF signal analysis, particularly for anomaly detection and correlation.

Here are its key functionalities:

• Data Aggregation: It explicitly combines fft_bins (spectral data) from “signal_spectrum” messages and packet_info from “packet_metadata” messages, storing them in an internal buffer associated with a signal_id. It can then publish a “latent_summary” containing these aggregated features.
• Spectral Analysis via Ghost Imaging:
  • It uses a pre-configured CompiledGhostDetectorSingleton model to process incoming spectrum data (fft_bins).
  • This model performs a reconstruction of the spectrum (recon) and calculates a reconstruction_error_score (anomaly score) based on the input spectrum and its reconstruction.
  • These ghost analysis results, including the ghost_recon_spectrum and reconstruction_error_score, are stored in its buffer.
• Orbital Mimic Detection:
  • If enable_orbital_detection is true, it initializes and utilizes an OrbitalMimicDetector.
  • This detector performs specialized orbital_analysis on the signal’s spectral data.
  • If an orbital mimic is detected, it publishes an “orbital_mimic_alert” which includes details like signal_id, alert_type, matched_fingerprint, match_confidence, ghost_score, and timestamp.
• Anomaly Alerting:
  • It defines an anomaly_threshold.
  • If the calculated reconstruction_error_score for a signal exceeds this threshold, it publishes a “signal_alert” with an alert_type of “ghost_anomaly”. This alert also includes the confidence (score), timestamp, and whether an orbital_mimic was detected.
  • PyTorch is a prerequisite for the LatentAggregator to function.

Waterfall colors refer to the custom colormaps used to represent signal power in waterfall displays. These colors are crucial for visualizing how signal strength changes over time in a dynamic RF environment.

The SignalVisualizer module defines specific colormaps for this purpose:

• “rf_quantum”: This colormap is defined by specific red, green, and blue color transitions at different data values.
  • Red component: Starts at 0.0, rises to 0.5 at 0.4, then to 1.0 at 0.6, and stays at 1.0.
  • Green component: Starts at 0.0, rises to 0.5 at 0.6, then to 1.0 at 0.8, and stays at 1.0.
  • Blue component: Starts at 0.0, rises to 0.5 at 0.2, then to 1.0 at 0.4, and then declines to 0.0 at 0.8 and stays at 0.0.
• “thermal”: This colormap also has defined red, green, and blue transitions, designed to evoke a thermal imaging effect.
  • Red component: Starts at 0.0, then jumps to 1.0 at 0.66, and stays at 1.0.
  • Green component: Starts at 0.0, rises to 0.5 at 0.33, then to 1.0 at 0.66, and then drops to 0.0 at 1.0.
  • Blue component: Starts at 0.0, rises to 1.0 at 0.33, and then drops to 0.0 at 0.66 and stays at 0.0.

These custom colormaps are registered with Matplotlib and can be selected via the colormap configuration parameter when initializing the SignalVisualizer, with “rf_quantum” being the default. When plotting a waterfall display, the imshow function uses the chosen colormap to render the waterfall_data, where signal power is represented by color intensity.