Skip to content

Fed-SB for Communication Efficient Federated LoRA Fine-Tuning

This module presents a Signal Classifier designed to identify RF signal types using a neural network. It leverages Federated LoRA-SB (Fed-SB) for efficient, distributed training across various edge devices, ensuring private fine-tuning with Differential Privacy SGD (DP-SGD). The system enhances its classification capabilities by integrating a vision Large Language Model (LLM) for in-depth spectrogram analysis, extracting visual features like bandwidth and peak counts. Essential for naval and Starship applications, the classifier is built with GPU acceleration support and facilitates gRPC communication for central aggregation of LoRA-SB parameters, offering a comprehensive solution for real-time RF signal intelligence.

The integration of Fed-SB (federated LoRA-SB) and Vision LLMs significantly enhances RF signal classification, particularly across distributed edge devices, by combining efficient distributed learning with rich visual analysis of signals.

Fed-SB (Federated LoRA-SB) for Distributed Training

  • Distributed Training: Fed-SB facilitates distributed training of the neural network classifier across multiple edge devices. This means that RF signal classification models can be trained collaboratively without centralizing all raw data, which is crucial for privacy and data locality.
  • Parameter-Efficient Fine-Tuning (LoRA-SB Layers): The core of Fed-SB is the LoRA-SB layer, which stands for Low-Rank Adaptation with a Shared Bias matrix.
    • These layers are designed for parameter-efficient fine-tuning, meaning only a small set of parameters needs to be updated and exchanged during training, rather than the entire model.
    • Specifically, LoRA-SB layers introduce a trainable R matrix (self.R) in addition to non-trainable A and B matrices.
    • During local training on an edge device, the model (which includes LoRA-SB layers) is updated.
    • After local training, the R matrices from each client’s LoRA-SB layers are sent to a central aggregation server (via gRPC).
    • The aggregation server then averages these R matrices to create an aggregated R matrix, which is subsequently distributed back to the clients. This aggregation process allows the global model to learn from the diverse data of many edge devices without direct data sharing.
  • Private Fine-Tuning: The module supports private fine-tuning with DP-SGD (Differential Privacy Stochastic Gradient Descent), which further enhances privacy during distributed training by adding noise to gradients.
  • Scalability and Efficiency: By only aggregating the small R matrices, Fed-SB significantly reduces the communication overhead and computational burden compared to traditional federated learning where entire model weights might be exchanged, making it more feasible for resource-constrained edge devices.

Vision LLM Integration for Spectrogram Analysis

  • Enhanced Feature Extraction: A Vision LLM (Large Language Model for vision) is integrated to provide a deeper, more nuanced analysis of RF spectrograms.
    • The system generates an RF spectrogram image from frequency and amplitude data.
    • This spectrogram image is then sent to a locally hosted Vision LLM.
    • The Vision LLM analyzes the spectrogram based on a detailed prompt, extracting various visual features and insights that go beyond traditional numerical signal processing. These include:
      • Frequency markers (e.g., MHz labels).
      • Number of signal peaks.
      • Bandwidth of the main signal.
      • Symmetry of sidebands (e.g., symmetric or asymmetric).
      • Modulation pattern (e.g., sinc-like, dual peaks, single sideband, narrow spike, symmetric sidebands, wide uniform).
      • Anomalies (e.g., interference, scintillation).
  • Augmented Classifier Input: The extracted visual features from the Vision LLM (e.g., visual_bandwidth, visual_peak_count, visual_symmetry, anomalies, visual_modulation) are then combined with traditional signal features (e.g., bandwidth, center frequency, peak power, variance, skewness, kurtosis) to form a comprehensive input vector for the neural network classifier. This provides the model with a richer understanding of the RF signal.
  • LLM Validation and Confidence Adjustment: The Vision LLM’s output can be used to validate or adjust the confidence of the neural network’s prediction. For instance, if the visual modulation pattern identified by the LLM (e.g., “sinc-like” suggesting PSK) contradicts the neural network’s initial prediction, the confidence of the prediction may be adjusted (e.g., reduced by 20%). This acts as an intelligent cross-check, potentially improving the robustness and reliability of the classification.

Overall Enhancement

By integrating Fed-SB and Vision LLMs, the system offers:

  • Robust Classification: The combination of traditional signal features with visually-derived insights from spectrograms allows the model to learn more complex and subtle characteristics of RF signals, leading to more accurate and robust classification.
  • Privacy-Preserving and Scalable Deployment: Fed-SB enables the deployment and continuous improvement of these sophisticated models directly on distributed edge devices, such as those used in naval and Starship applications, without compromising data privacy or requiring massive data transfers to a central server.
  • Adaptive Learning: The federated approach means that the model can adapt and improve over time by learning from new data generated at various edge locations, ensuring it remains effective in dynamic RF environments.

The following modulation types are defined and used within the RF signal classification system:

  • AM (Amplitude Modulation)
  • FM (Frequency Modulation)
  • SSB (Single Sideband)
  • CW (Continuous Wave)
  • PSK (Phase-Shift Keying)
  • FSK (Frequency-Shift Keying)
  • NOISE
  • UNKNOWN

These modulation types are mapped to numerical indices (0 through 7, respectively) for use in the neural network classifier. The system also uses these labels to generate synthetic training data, excluding ‘UNKNOWN’ from the generation process. Furthermore, the Vision LLM integration uses specific visual patterns (e.g., ‘sinc-like’, ‘dual peaks’, ‘single sideband’, ‘narrow spike’, ‘symmetric sidebands’, ‘wide uniform’) to suggest and potentially validate predictions for types like PSK, FSK, SSB, CW, AM, and FM.

"""Predict the modulation type of a signal with LLM validation."""
if not spectrogram_path:
spectrogram_path = self.generate_spectrogram_image(freqs, amplitudes)
features = self.extract_features(freqs, amplitudes, threshold, spectrogram_path)
X = self.features_to_vector(features)
if self.gpu['enabled']:
X = self.xp.asnumpy(X.cpu().numpy())
X = torch.tensor(X, dtype=torch.float32, device=self.device)
self.model.eval()
with torch.no_grad():
output = self.model(X)
probabilities = torch.softmax(output, dim=1)
confidence, pred_idx = torch.max(probabilities, dim=1)
modulation = MODULATION_LABELS[pred_idx.item()]
visual_modulation = features.get('visual_modulation', '')
if visual_modulation:
expected_modulation = {
'sinc-like': 'PSK',
'dual peaks': 'FSK',
'single sideband': 'SSB',
'narrow spike': 'CW',
'symmetric sidebands': 'AM',
'wide uniform': 'FM'
}.get(visual_modulation.lower(), modulation)
if expected_modulation != modulation:
confidence *= 0.8
return {
'modulation': modulation,
'confidence': float(confidence),
'features': features,
'anomalies': features.get('anomalies', []),
'visual_modulation': visual_modulation
}
def generate_training_data(self, num_samples=1000):

The RF SCYTHE signal classifier module requires several types of data for its operation, encompassing both real-time signal analysis and distributed model training.

Here are the key data requirements:

  • RF Signal Data (Frequencies and Amplitudes): This is the fundamental input for the system.
    • For classification and prediction, the system takes freqs (frequencies) and amplitudes of an RF signal.
    • This data is used to generate RF spectrogram images.
    • It is also used to generate synthetic training data, where frequencies and amplitudes are simulated for various modulation types like AM, FM, SSB, CW, PSK, FSK, and NOISE.
  • Spectrogram Images: These images are derived from the RF signal’s frequency and amplitude data.
    • Spectrograms are crucial for the Vision LLM integration, as they are sent to a locally hosted Vision LLM for analysis.
    • The Vision LLM analyzes these images based on a prompt to extract visual features and insights.
  • Extracted Features: The system extracts a comprehensive set of features from the RF signals, which serve as input for the neural network classifier. These include:
    • Traditional signal features: Such as bandwidth, center_freq, peak_power, mean_power, variance, skewness, kurtosis, crest_factor, spectral_flatness, and spectral_rolloff.
    • Vision LLM-derived visual features: These are extracted from the spectrogram images and include visual_bandwidth, visual_peak_count, visual_symmetry, anomalies, and visual_modulation.
    • These features are combined into a feature vector to be fed into the neural network.
  • Modulation Labels: These are the target outputs for the classification model.
    • The system defines specific modulation types: AM, FM, SSB, CW, PSK, FSK, NOISE, and UNKNOWN.
    • For training purposes, numerical labels corresponding to these modulation types (y) are paired with the extracted feature vectors (X).
  • LoRA-SB R Matrices: During the distributed training process using Fed-SB, specific parameters of the LoRASBLayers are exchanged.
    • Clients send their locally updated R matrices (trainable R matrices within the LoRA-SB layers) to a central aggregation server.
    • The clients then receive an aggregated R matrix back from the server to update their models.

Why Most Ku/Ka-band Datasets Are for Remote Sensing, Radar, or Channel Studies

  1. Remote Sensing (Earth Observation, Weather, Climate):

Agencies like NASA, ESA, and JAXA use Ku/Ka-band radars and radiometers on satellites and aircraft to measure ocean surface winds, rainfall, snow, soil moisture, and ice.
Datasets typically contain radar backscatter, brightness temperature, or processed geophysical parameters—not raw IQ or wideband comms signals.
Example: NASA’s GPM (Global Precipitation Measurement) uses Ku/Ka-band radar for rainfall mapping.

  1. Radar (Imaging, Altimetry, Target Detection):

Ku/Ka-band is popular for high-resolution radar imaging (SAR), altimetry (measuring surface height), and military target detection.
Datasets are often in the form of processed radar images, range-Doppler maps, or channel impulse responses, not general communication signals.
Example: Sentinel-6 (altimetry), TerraSAR-X (imaging).

  1. Satellite Channel Studies (Propagation, Fading, Link Quality):

Research in satellite communications often focuses on how Ku/Ka-band signals propagate through the atmosphere, especially rain fade, scintillation, and multipath.
Datasets include channel impulse responses, SNR time series, or link outage statistics, not modulated comms waveforms.
Example: ITU-R channel models, rain fade measurement campaigns.
Why This Is Different from General Communications Datasets
General comms datasets (like those used for modulation classification, as in your signal_classifier_fed_sb.py) require raw IQ or spectrum data with labeled modulation types (AM, FM, PSK, FSK, etc.).
Remote sensing/radar/channel datasets are usually not labeled by modulation, and often do not contain modulated comms signals at all—they focus on physical/environmental measurements or radar returns.
Public SDRs rarely cover Ku/Ka-band due to hardware cost and regulatory issues, so there’s little open comms data in these bands.
For Your Use Case (signal_classifier_fed_sb.py)
Your classifier expects labeled comms signals (modulation types, features, possibly spectrograms).
Most Ku/Ka-band datasets you’ll find are not suitable for direct use—they lack comms signals, labels, or are not in IQ/spectrum format.
If you want to train or test on Ku/Ka-band comms, you’ll likely need to:
Simulate signals (using GNU Radio, MATLAB, or your own synthetic data generator).
Collect your own data with suitable hardware (LNB/downconverter + SDR).
Adapt remote sensing/radar datasets only if you want to classify radar returns or propagation effects, not comms modulations.