We present a real-time RF-to-speech pipeline that
decodes inner speech from RF-inferred neural surrogates using
a word-state HMM with GPT-style priors. Starting from 1.5
ms FFT triage (0.754 AUROC), we map spectral confidence
to link quality q ∈ [0, 1], which predicts command success
(73.9% → 100%) and p95 latency (2.6 s → 375 ms). A
Bayesian decoder with language priors reduces WER from 2.8%
to 1.1% at 10 dB SNR (60.7% relative reduction), with posterior
concentration on correct word spans. The system integrates with
tactical control systems for hands-free command execution. Full
end-to-end reproducibility: make all generates IQ → WER.
Liberty Hamilton – Determining the functional organization of the speech cortex
To process speech, the brain must transform low-level acoustic inputs to higher order linguistic categories, such as phonemes, words, and narrative meaning. This involves being able to encode acoustic features that happen at both brief and long timescales. We applied unsupervised methods to neural recordings of people listening to naturally spoken sentences, and uncovered an organization of the auditory cortex and surrounding areas into two spatially and functionally distinct modules. They are now applying similar methods to look at changes in functional organization during brain development in children with epilepsy. They also apply computational models to analyze which particular sound features are represented in the brain, and how areas functionally interact during natural speech perception and production.
Some Inspiration from:
https://www.cincinnatichildrens.org/research/divisions/b/biostatistics
bgilbert@neurosphere:~/paper_Bayesian_Decoding_of_Inner_Speech_from_RF-Inferred_Activity$ cd /home/bgilbert/paper_Bayesian_Decoding_of_Inner_Speech_from_RF-Inferred_Activity && grep -n "atl\|ATL" code/core.py
95: # ATL/TWPA design-aware processing
96: self.atl_design = None
105: # Load ATL/TWPA design configuration
106: self._load_atl_design()
142: def _load_atl_design(self):
143: """Load optional ATL/TWPA design facts from Arxiv 2510.24753v1."""
145: path = "config/atl_design.json"
150: self.atl_design = {
160: logger.info("ATL/TWPA design loaded from config/atl_design.json")
162: logger.warning(f"Failed to load ATL design: {e}")
164: def _label_atl_band(self, f_hz: float, tol_hz: float = 0.01e9):
165: """Return band label based on design facts from ATL synthesis."""
166: if not self.atl_design:
168: d = self.atl_design
190: if not self.atl_design or not self.atl_design.get("pump_hz"):
192: fp = self.atl_design["pump_hz"]
202: if self.atl_design.get("mixing_mode", "4WM").upper() == "4WM":
223: def annotate_signal_with_atl(self, signal: "RFSignal"):
224: """Attach ATL/TWPA labels to signal.metadata (no-op if no design)."""
231: band_label, band_info = self._label_atl_band(signal.frequency)
234: signal.metadata.setdefault("atl", {})
235: signal.metadata["atl"].update({
241: # Log important ATL events
243: logger.info(f"ATL event detected - Signal {signal.id}: {band_label}, near_3fp: {mix_info.get('near_3fp', False)}")
246: logger.debug(f"ATL annotate failed: {e}")
248: def process_atl_alerts(self, signal: "RFSignal"):
249: """Process ATL-related alerts and update classifications as needed."""
250: if not self.atl_design or "atl" not in signal.metadata:
253: atl_data = signal.metadata["atl"]
255: # Check for important ATL events that warrant classification updates
258: if atl_data.get("near_3fp"):
261: if atl_data.get("band_label") == "stopband":
264: if atl_data.get("near_rpm_notch"):
267: if atl_data.get("idlers"):
268: alert_conditions.append(f"parametric_mixing_detected({len(atl_data['idlers'])})")
272: new_classification = f"ATL_Event: {', '.join(alert_conditions)}"
277: update_info={"atl": atl_data, "alert_conditions": alert_conditions}
396: # Re-annotate with ATL if this is an ATL-related update
397: if update_info and "atl" in update_info:
398: self.annotate_signal_with_atl(signal)
470: # Add ATL band labeling if design is present
471: if self.atl_design:
473: label, _ = self._label_atl_band(center, tol_hz=0.02e9)
474: band_dict["atl_band_label"] = label