Neural MIMO Beam Steering for Non-Invasive Neuromodulation

Overall Impression

Your paper presents an intriguing and timely application of reinforcement learning (RL) to MIMO beam steering for non-invasive neuromodulation, emphasizing real-time adaptation and safety constraints. The camera-in-the-loop approach is a novel hook that bridges simulation gaps in electromagnetic field targeting, potentially advancing personalized therapies. The focus on exploration-exploitation dynamics via entropy and divergence metrics adds depth to the RL analysis, which is often underexplored in engineering papers. However, the manuscript feels underdeveloped for a full conference or journal submission—it’s concise (3 pages) but lacks substantive results, quantitative validation, and methodological rigor. This makes it read more like a position paper or extended abstract than a complete study. With expansion, it could be compelling, but currently, it prioritizes conceptual framing over empirical evidence.

Strengths

• Novelty and Relevance: The integration of camera-based feedback for RL training in neuromodulation is innovative, addressing key challenges like anatomical variability and SAR (Specific Absorption Rate) limits. Listing contributions bullet-style in the Introduction is effective and reader-friendly.
• Safety Emphasis: Incorporating SAR proxies into rewards and monitoring via camera is a strong ethical angle, aligning with growing concerns in bioelectromagnetics.
• Visualization Choices: The θ–f heatmaps and divergence plots (Figs. 1–4) sound useful for illustrating policy evolution, though they’re not fully described here.
• Discussion Structure: The limitations and future work subsections are candid and forward-looking, showing self-awareness (e.g., free-space vs. tissue modeling).

Weaknesses and Suggestions

I’ll break this down by section, highlighting issues with clarity, completeness, and scientific soundness.

Abstract

• Issues: It’s overly dense and jargon-heavy (“θ–f heatmaps for learned beams using lightweight scripts wired to make”), which might confuse non-experts. It mentions logging reward curves but doesn’t quantify outcomes (e.g., convergence speed or performance gains). The phrase “wired to make” feels incomplete or typo-ridden—perhaps “wired to a Makefile”?
• Suggestions: Expand to 150–200 words for better flow. Add a teaser result, e.g., “Policies converge in <200 epochs with 20% improved targeting precision.” Ensure acronyms (e.g., MIMO, RL, SAR) are defined on first use.

Introduction

• Issues: The motivation is solid but generic—claims like “precise spatial targeting” need a citation to prior work (e.g., compare to static beamforming in TMS studies). “Neural MIMO” in the title and intro is ambiguous; does “neural” refer to neuromodulation or neural networks? Clarify.
• Suggestions: Cite 2–3 benchmarks (e.g., traditional phased-array limits in [ref]). Strengthen contributions by quantifying where possible (e.g., “reduces side lobes by X dB”).

Methods

• Issues:
• Array Configuration: The ULA setup and phase-only beamforming equation (1) are clear, but why 8 Tx/4 Rx at 2.4 GHz? Justify frequency choice (e.g., penetration depth for neuromodulation) and spacing (λ/2 is standard, but link to safety).
• Camera-in-the-Loop: High-level description is good, but lacks specifics: What camera (e.g., resolution, frame rate)? How is intensity mapped to angles? No mention of calibration errors or noise handling.
• RL Framework: Promising contrast between epsilon-greedy and PPO, but superficial. For PPO, what are the action spaces (e.g., discretization levels for θ, f)? No hyperparameters (e.g., learning rate, clip ratio), environment details (state: camera image? Reward: exact formula?), or episode structure. “Factorized categorical action heads” is advanced but unexplained—how does it handle multi-action coupling?
• Metrics: Good selection (e.g., JS divergence for convergence), but definitions are missing (e.g., what’s the “SAR proxy”? Peak intensity?).
• Suggestions: Add subsections for reproducibility: pseudocode for reward function, simulation params (e.g., Gym-like env). Include a system diagram figure. Aim for 1–2 pages to flesh this out—current brevity risks irreproducibility.

Results

• Issues: This is the weakest section—it’s fragmented and figure-heavy without narrative. Subheadings (A–D) are placeholders with no text; Figs. 2–3 describe KL/JS divergences, but what do they mean practically? Fig. 1 shows entropy dropping (good for exploitation), but no baselines or error bars. Critically, no core outcomes: Where are the beam patterns, main lobe gains, or SAR values? “Visitation–Policy” metrics imply action analysis, but without data tables or stats (e.g., p-values), it’s opaque. The section ends abruptly before Discussion.
• Suggestions: Expand to show quantitative results, e.g., a table comparing epsilon-greedy vs. PPO:

Metric	Epsilon-Greedy (200 epochs)	PPO (200 epochs)	Baseline (Static)
Main Lobe Gain (dB)	15.2 ± 1.1	18.4 ± 0.8	12.5
Side Lobe Ratio (dB)	-20.1	-25.3	-15.2
SAR Proxy (W/kg)	0.8	0.7	1.2
Convergence Epochs	150	120	N/A

Include actual θ–f heatmaps as promised. Discuss figure trends: e.g., “KL divergence stabilizes post-100 epochs, indicating policy robustness.”

Discussion

• Issues: Strong on advantages (e.g., real-time feedback beats simulations), but interpretations are qualitative. Policy convergence claim (“after ~200 epochs”) cites JS but ignores entropy-return scatter (Fig. 4)—does low entropy correlate with high returns? Safety discussion is vague: How is SAR enforced (hard constraint or soft penalty)? Limitations are honest but brief; e.g., no phase measurement limits interference patterns—quantify impact.
• Suggestions: Tie back to results explicitly (e.g., “Fig. 4’s negative entropy-return slope validates exploration benefits”). Add a paragraph on clinical translation (e.g., FDA SAR limits). Balance with a “Broader Impacts” subsection.

Conclusion

• Issues: Repetitive of abstract; no new synthesis.
• Suggestions: End with a call-to-action, e.g., “This framework paves the way for RL-driven TMS devices.”

Figures and General Presentation

• Issues: Figures are referenced but not embedded in the provided text (assuming PDF has them). Captions are descriptive but lack scales/units (e.g., y-axis “Entropy (bits)”—what’s the max possible?). Typos abound: “VisitationPolicy” (likely “Visitation-Policy”), “learns conditions” (probably “learns optimal”), garbled chars in PDF extraction. “No Collaborators” is quirky—remove or explain.
• Suggestions: Ensure figures are high-res and self-contained. Use consistent notation (e.g., θ_0 vs. θ). Proofread for LaTeX artifacts.

Final Recommendations

This has strong potential as a workshop paper (e.g., NeurIPS ML4H) but needs ~50% more content for broader venues: prioritize results with data, methods with details, and citations (aim for 15–20 refs). Run ablation studies (e.g., w/o camera feedback) to bolster claims. Total score: 6/10—innovative idea, but execution lags. Revise iteratively, perhaps sharing drafts on arXiv for feedback. Great start—keep pushing the neural-EM intersection!

Potential Synergies Between TTA for Quantized NNs and Neural MIMO Beam Steering

Your Neural MIMO beam steering paper (from the prior critique) focuses on RL-driven adaptation for precise, safe electromagnetic targeting in neuromodulation, using a camera-in-the-loop setup with PPO and epsilon-greedy methods. It’s innovative for handling dynamic anatomy but, as noted, lacks depth in results, efficiency for real-time hardware, and handling of quantization-induced errors—common in edge-deployed systems like wearable neuromod devices. The new paper on Test-Time Model Adaptation for Quantized Neural Networks (TTA for QNNs) introduces Zeroth-Order Adaptation (ZOA), a forward-pass-only framework for adapting low-bit models (e.g., W6A6 ViT) to domain shifts without backpropagation. This is highly relevant, as neuromodulation hardware often quantizes models for power/latency constraints (e.g., on FPGAs or MCUs), amplifying sensitivity to shifts like tissue variations or interference.

Here’s how this TTA work could help strengthen your paper, structured by key areas: conceptual integration, methodological enhancements, and empirical extensions. These suggestions address prior weaknesses (e.g., irreproducibility, limited results) while boosting novelty for venues like NeurIPS or EMBC.

1. Addressing Quantization Sensitivity in Dynamic Environments

• Relevance: Your paper notes free-space limitations and calls for tissue phantoms in future work. The TTA paper’s Proposition 1 theoretically proves QNNs suffer exponential loss degradation under OOD perturbations (ΔL ∝ 1/2^{2n}), empirically shown in Fig. 1 (e.g., 20%+ accuracy drop for W3A3 ViT on ImageNet-C). This mirrors your MIMO challenges: quantized beamforming weights could amplify errors from anatomical shifts, worsening SAR violations or targeting precision.
• How it Helps:
  • Incorporate Theoretical Motivation: Add a subsection in your Sec. III (Results/Discussion) adapting their Prop. 1 to beam steering. E.g., model quantization noise in phase weights (Eq. 1) as Δw ∝ 1/2^n, showing how it exacerbates off-target radiation. This substantiates your safety-aware rewards empirically (e.g., via simulated OOD fields).
  • Practical Boost: Quantize your ULA weights (e.g., to 4-8 bits) and demonstrate TTA-like adaptation reduces side-lobe ratios by 10-15% on perturbed datasets (e.g., noisy camera feeds simulating tissue scatter).
• Impact on Your Paper: Elevates it from descriptive RL to a robustness-focused study, with citations to [42] (FOA, a baseline they beat). Cite arXiv:2508.02180 for the theoretical hook.

2. Efficient, Gradient-Free Adaptation for Real-Time Constraints

• Relevance: Your PPO uses policy gradients, which vanish in quantized nets (as TTA notes), and requires many epochs (Figs. 1-3 show ~200 for convergence). PPO’s factorized heads are clever but compute-heavy for edge neuromod (e.g., no BP on low-power arrays). ZOA uses zeroth-order optimization (ZO) with two forward passes per sample—one for inference, one for perturbation-based gradient estimation—via continual domain knowledge learning. It reuses historical adaptations with low memory (domain management scheme), cutting interference in long-term streams.
• How it Helps:
  • Replace/ Augment RL Backend: Swap PPO’s gradients for ZOA’s two-sided ZO estimator (their Sec. 4). For your bandit/PPO hybrid, treat steering angle θ_0 and phase offsets as low-dim actions; perturb them forward-only to minimize a TTA objective like entropy on field intensities (from camera feedback). This enables single-sample updates, ideal for real-time (e.g., <10ms per beam adjustment).
  • Domain Knowledge Reuse: Adapt their management scheme to store “domain snapshots” (e.g., θ-f heatmaps per anatomy type). Use learnable coefficients to blend them, reducing your policy entropy drops (Fig. 1) and enabling continual learning across sessions—addressing your exploration-exploitation analysis.
  • Implementation Tip: Their GitHub (https://github.com/DengZeshuai/ZOA) has lightweight ZO scripts; integrate with your “lightweight scripts wired to make” for θ-f viz. Test on quantized PPO heads to show 2x faster convergence vs. epsilon-greedy.
• Impact on Your Paper: Fixes efficiency critiques—e.g., add ablation in expanded Results: ZOA vs. PPO on 8-bit weights shows 3x fewer passes, 5% better main-lobe gain. Positions your work as “ZO-RL for quantized neuromod,” novel for bio-EM.

3. Enhancing Safety and Generalization Metrics

• Relevance: Both emphasize safety (your SAR penalties; their implicit via robust adaptation). TTA’s continual scheme accumulates OOD knowledge without catastrophic forgetting, using JS divergence for convergence (similar to your Fig. 3). It beats FOA by 5% on ImageNet-C for W6A6 ViT, proving ZO scales to transformers/CNNs—your MIMO could use CNN-like field mappers.
• How it Helps:
  • Safety-Aware ZO Rewards: Fuse your reward (target intensity – SAR) with TTA’s entropy min: Update via ZO on camera-derived states, monitoring SAR proxies in real-time. Their domain bank prevents overfitting to one anatomy, aligning with your limitations (e.g., phase-only intensity).
  • Metrics Expansion: Track TTA-style KL/JS on action distributions (your Figs. 2-3) post-ZO; add scatter plots like their implied return-entropy (your Fig. 4) but for SAR vs. precision. Quantify long-term: e.g., after 1000 “test samples” (simulated shifts), ZOA retains 95% ID performance vs. 80% for vanilla PPO.
  • Hardware Tie-In: For clinical translation, note ZOA’s edge-friendliness (no BP memory)—test on quantized ULA sims (e.g., via PyTorch Quantization) to show <1% SAR exceedance under shifts.
• Impact on Your Paper: Bolsters Discussion (Sec. IV): “ZOA-inspired continual learning mitigates limitations D/E, enabling hierarchical multi-target steering.” Adds a table:

Method	Forward Passes/Sample	Convergence Epochs	SAR Compliance (OOD)	Targeting Gain (dB)
Epsilon-Greedy	1	250	85%	+12.5
PPO (Baseline)	5+ (grads)	200	90%	+15.2
ZOA-Augmented	2	120	96%	+18.4

4. Broader Extensions and Future Directions

• Cross-Domain Transfer: Use TTA’s knowledge management for your future work—e.g., transfer learned beams from phantoms to live tissue via ZO on electro-optic phase data.
• Experimental Validation: Rerun your setup with QNNs (e.g., quantize policy net to W4A4); benchmark on extended ImageNet-C analogs (e.g., corrupted field maps). Their 15-page structure (full methods/results) is a model for expanding your 3-pager.
• Caveats: ZOA assumes forward-pass access (fits your camera loop) but may need tuning for high-dim actions (your masking/phases)—start with low-bit prototypes.
• Publication Angle: Frame as “ZOA-RL Hybrid for Safe, Quantized Neuromod”—submit to ICML 2026 or TMI, citing this as inspirational baseline.

Overall, this TTA paper could transform your work from a proof-of-concept to a deployable framework, emphasizing efficiency and robustness. It directly tackles your quantization gap, potentially adding 20-30% novelty. If you share code snippets or specific sections to adapt, I can refine further!