{"id":3073,"date":"2025-09-02T15:20:09","date_gmt":"2025-09-02T15:20:09","guid":{"rendered":"https:\/\/172-234-197-23.ip.linodeusercontent.com\/?p=3073"},"modified":"2025-09-02T22:48:00","modified_gmt":"2025-09-02T22:48:00","slug":"recovering-dynamics-from-energy-bands-a-sparse-regression-approach-%f0%9f%a7%a0","status":"publish","type":"post","link":"https:\/\/172-234-197-23.ip.linodeusercontent.com\/?p=3073","title":{"rendered":"Recovering Dynamics from Energy Bands: A Sparse Regression Approach \ud83e\udde0"},"content":{"rendered":"\n<p>Let&#8217;s unpack the function <code>recover_ode<\/code> for a technically curious audience, especially those working in signal processing, sparse modeling, or data-driven dynamical systems:<\/p>\n\n\n\n<hr class=\"wp-block-separator has-alpha-channel-opacity\"\/>\n\n\n\n<h1 class=\"wp-block-heading\"><\/h1>\n\n\n\n<figure class=\"wp-block-image size-full\"><img data-opt-id=1237226359  fetchpriority=\"high\" decoding=\"async\" width=\"700\" height=\"635\" src=\"https:\/\/ml6vmqguit1n.i.optimole.com\/w:auto\/h:auto\/q:mauto\/f:best\/https:\/\/172-234-197-23.ip.linodeusercontent.com\/wp-content\/uploads\/2025\/09\/&#xd83e;&#xdde0;-Recovering-Dynamics-from-Energy-Bands-A-Sparse-Regression-Approach.jpg\" alt=\"\" class=\"wp-image-3074\" srcset=\"https:\/\/ml6vmqguit1n.i.optimole.com\/w:700\/h:635\/q:mauto\/f:best\/https:\/\/172-234-197-23.ip.linodeusercontent.com\/wp-content\/uploads\/2025\/09\/\ud83e\udde0-Recovering-Dynamics-from-Energy-Bands-A-Sparse-Regression-Approach.jpg 700w, https:\/\/ml6vmqguit1n.i.optimole.com\/w:300\/h:272\/q:mauto\/f:best\/https:\/\/172-234-197-23.ip.linodeusercontent.com\/wp-content\/uploads\/2025\/09\/\ud83e\udde0-Recovering-Dynamics-from-Energy-Bands-A-Sparse-Regression-Approach.jpg 300w\" sizes=\"(max-width: 700px) 100vw, 700px\" \/><\/figure>\n\n\n\n<p>In many signal processing and physics-inspired machine learning tasks, we\u2019re often handed a time-frequency representation\u2014say, a matrix of energy across frequency bands over time\u2014and asked to infer the underlying dynamics. What drives the changes? Can we recover a differential equation that governs the system\u2019s evolution?<\/p>\n\n\n\n<p>Enter <code>recover_ode<\/code>: a compact yet powerful Python function that uses sparse regression to estimate time derivatives from band-wise energy data. Whether you&#8217;re modeling acoustic systems, RF emissions, or neural activity, this function offers a flexible way to reverse-engineer the temporal dynamics.<\/p>\n\n\n\n<hr class=\"wp-block-separator has-alpha-channel-opacity\"\/>\n\n\n\n<h2 class=\"wp-block-heading\">\ud83d\udd0d What Does <code>recover_ode<\/code> Do?<\/h2>\n\n\n\n<p>At its core, <code>recover_ode<\/code> takes a matrix of energy values <code>E<\/code> (shape: time \u00d7 bands) and tries to recover the time derivative of the signal\u2014either globally or per frequency band\u2014using sparse regression.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Two Modes of Operation:<\/h3>\n\n\n\n<ul class=\"wp-block-list\">\n<li><strong>Global Mode (<code>per_band=False<\/code>)<\/strong>:<br>Models the derivative of the total energy across all bands:<br>$$ \\frac{dY}{dt} \\quad \\text{where} \\quad Y = \\sum_b E_b $$<\/li>\n\n\n\n<li><strong>Per-Band Mode (<code>per_band=True<\/code>)<\/strong>:<br>Models the derivative of each band separately:<br>$$ \\frac{dE_b}{dt} \\quad \\text{for each band } b $$<br>This is a multitarget sparse regression problem.<\/li>\n<\/ul>\n\n\n\n<hr class=\"wp-block-separator has-alpha-channel-opacity\"\/>\n\n\n\n<h2 class=\"wp-block-heading\">\ud83e\uddea How It Works<\/h2>\n\n\n\n<h3 class=\"wp-block-heading\">Step-by-Step Breakdown:<\/h3>\n\n\n\n<ol class=\"wp-block-list\">\n<li><strong>Time Resolution Setup<\/strong><\/li>\n<\/ol>\n\n\n\n<pre class=\"wp-block-code\"><code>   dt = hop \/ fs<\/code><\/pre>\n\n\n\n<p>Converts frame hop size and sampling rate into seconds per frame.<\/p>\n\n\n\n<ol start=\"2\" class=\"wp-block-list\">\n<li><strong>Dictionary Construction<\/strong><\/li>\n<\/ol>\n\n\n\n<pre class=\"wp-block-code\"><code>   Phi = build_dictionary(E)<\/code><\/pre>\n\n\n\n<p>Builds a feature dictionary from the energy matrix. This could include polynomial expansions, temporal embeddings, or other basis functions.<\/p>\n\n\n\n<ol start=\"3\" class=\"wp-block-list\">\n<li><strong>Derivative Estimation<\/strong><br>Uses a helper <code>time_derivative()<\/code> to estimate the derivative of the signal (either total energy or per band).<\/li>\n\n\n\n<li><strong>Sparse Regression via LassoLars<\/strong><\/li>\n<\/ol>\n\n\n\n<pre class=\"wp-block-code\"><code>   model = LassoLars(alpha=alpha, fit_intercept=False, max_iter=500)\n   model.fit(Phi, dY or dEs)<\/code><\/pre>\n\n\n\n<p>LassoLars promotes sparsity\u2014ideal for discovering parsimonious models with interpretable dynamics.<\/p>\n\n\n\n<ol start=\"5\" class=\"wp-block-list\">\n<li><strong>Result Packaging<\/strong><br>Returns a dictionary with learned coefficients (<code>theta<\/code> or <code>theta_matrix<\/code>), time step <code>dt_s<\/code>, and mode metadata.<\/li>\n<\/ol>\n\n\n\n<hr class=\"wp-block-separator has-alpha-channel-opacity\"\/>\n\n\n\n<h2 class=\"wp-block-heading\">\ud83d\udcca Output Structure<\/h2>\n\n\n\n<figure class=\"wp-block-table\"><table class=\"has-fixed-layout\"><thead><tr><th>Key<\/th><th>Description<\/th><\/tr><\/thead><tbody><tr><td><code>theta<\/code><\/td><td>Coefficients for global derivative model<\/td><\/tr><tr><td><code>theta_matrix<\/code><\/td><td>Coefficients per band (shape: bands \u00d7 features)<\/td><\/tr><tr><td><code>dt_s<\/code><\/td><td>Time step in seconds<\/td><\/tr><tr><td><code>mode<\/code><\/td><td><code>\"global\"<\/code> or <code>\"per_band\"<\/code><\/td><\/tr><tr><td><code>n_bands<\/code><\/td><td>Number of bands (only in per-band mode)<\/td><\/tr><\/tbody><\/table><\/figure>\n\n\n\n<hr class=\"wp-block-separator has-alpha-channel-opacity\"\/>\n\n\n\n<h2 class=\"wp-block-heading\">\ud83e\udded Why It Matters<\/h2>\n\n\n\n<p>This function is a gateway to interpretable modeling of temporal systems. Whether you&#8217;re analyzing:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Spectral energy in RF sensing<\/li>\n\n\n\n<li>Neural oscillations across cortical bands<\/li>\n\n\n\n<li>Acoustic emissions in structural monitoring<\/li>\n<\/ul>\n\n\n\n<p>\u2026<code>recover_ode<\/code> helps you move from raw data to dynamic equations.<\/p>\n\n\n\n<p>And because it\u2019s built on sparse regression, it naturally selects the most relevant features\u2014making it ideal for scientific discovery, anomaly detection, and real-time modeling.<\/p>\n\n\n\n<hr class=\"wp-block-separator has-alpha-channel-opacity\"\/>\n\n\n\n<h2 class=\"wp-block-heading\">\ud83d\udee0\ufe0f Tips for Extension<\/h2>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Swap <code>LassoLars<\/code> with other sparse estimators (e.g., ElasticNet, Orthogonal Matching Pursuit).<\/li>\n\n\n\n<li>Customize <code>build_dictionary()<\/code> to include domain-specific features.<\/li>\n\n\n\n<li>Use cross-validation to tune <code>alpha<\/code> for optimal sparsity vs. fidelity.<\/li>\n<\/ul>\n\n\n\n<hr class=\"wp-block-separator has-alpha-channel-opacity\"\/>\n\n\n\n<pre class=\"wp-block-code\"><code>def recover_ode(E: np.ndarray, fs:int, hop:int, band_edges: np.ndarray,\n                alpha: float = 1e-3, per_band: bool = False) -&gt; Dict:\n    \"\"\"\n    If per_band=False:\n        Recover dY\/dt for Y = sum(E_b)\n    If per_band=True:\n        Recover dE_b\/dt for each band separately (multitarget sparse regression).\n    \"\"\"\n    dt = hop \/ fs  # frame step in seconds\n    T, B = E.shape\n\n    # Build dictionary\n    Phi = build_dictionary(E)\n    n = Phi.shape&#91;0]\n\n    results = {}\n\n    if not per_band:\n        # --- Global case ---\n        Y = E.sum(axis=1)\n        Y_s, dY = time_derivative(Y, dt)\n        n = min(n, len(dY))\n        Phi, dY = Phi&#91;:n], dY&#91;:n]\n\n        model = LassoLars(alpha=alpha, fit_intercept=False, max_iter=500)\n        model.fit(Phi, dY)\n        results&#91;\"theta\"] = model.coef_.tolist()\n        results&#91;\"dt_s\"] = dt\n        results&#91;\"mode\"] = \"global\"\n    else:\n        # --- Per-band case ---\n        dEs = &#91;]\n        coefs = &#91;]\n        for b in range(B):\n            Y_s, dE = time_derivative(E&#91;:, b], dt)\n            dEs.append(dE&#91;:n])\n        dEs = np.stack(dEs, axis=1)  # (n, B)\n\n        Phi = Phi&#91;:n]\n\n        model = LassoLars(alpha=alpha, fit_intercept=False, max_iter=500)\n        model.fit(Phi, dEs)\n        coefs = model.coef_  # shape (B, n_features)\n\n        results&#91;\"theta_matrix\"] = coefs.tolist()\n        results&#91;\"dt_s\"] = dt\n        results&#91;\"mode\"] = \"per_band\"\n        results&#91;\"n_bands\"] = B\n\n    return results\n<\/code><\/pre>\n\n\n\n<figure class=\"wp-block-image size-large\"><img data-opt-id=1095175187  fetchpriority=\"high\" decoding=\"async\" width=\"1024\" height=\"848\" src=\"https:\/\/ml6vmqguit1n.i.optimole.com\/w:1024\/h:848\/q:mauto\/f:best\/https:\/\/172-234-197-23.ip.linodeusercontent.com\/wp-content\/uploads\/2025\/09\/image-5.png\" alt=\"\" class=\"wp-image-3077\" srcset=\"https:\/\/ml6vmqguit1n.i.optimole.com\/w:1024\/h:848\/q:mauto\/f:best\/https:\/\/172-234-197-23.ip.linodeusercontent.com\/wp-content\/uploads\/2025\/09\/image-5.png 1024w, https:\/\/ml6vmqguit1n.i.optimole.com\/w:300\/h:249\/q:mauto\/f:best\/https:\/\/172-234-197-23.ip.linodeusercontent.com\/wp-content\/uploads\/2025\/09\/image-5.png 300w, https:\/\/ml6vmqguit1n.i.optimole.com\/w:768\/h:636\/q:mauto\/f:best\/https:\/\/172-234-197-23.ip.linodeusercontent.com\/wp-content\/uploads\/2025\/09\/image-5.png 768w, https:\/\/ml6vmqguit1n.i.optimole.com\/w:1236\/h:1024\/q:mauto\/f:best\/https:\/\/172-234-197-23.ip.linodeusercontent.com\/wp-content\/uploads\/2025\/09\/image-5.png 1236w\" sizes=\"(max-width: 1024px) 100vw, 1024px\" \/><\/figure>\n\n\n\n<pre class=\"wp-block-code\"><code>.\/test_client_fft_ode_demo.sh\nStarting API server in the background...\nServer started with PID 565343\nServer is ready\n=== Testing RF Quantum SCYTHE Client FFT ODE Demo ===\n\nRunning in global mode (default)...\nRF Quantum SCYTHE Client FFT ODE Demo - 2025-09-02 16:55:53\nAPI URL: http:\/\/localhost:8000\nPer-band mode: False\nFetching FFT plan from http:\/\/localhost:8000\/denoise\/hints...\nWarning: No FFT plan in response, using defaults\n\nGenerating synthetic RF data...\nGenerated 12288000 samples (1.0s at 12.288 MHz)\n\nComputing band energies...\nComputed 96 frames with 8 bands\n\nApplying FFT plan...\nApplied FFT plan with shift_bins=0\n\nRecovering ODE from band energies...\n\n=== ODE Recovery Results ===\nMode: global\n\nBand Influence Ranking:\nBand (MHz)  Influence\n   3.1-4.6  12.807979\n   1.5-3.1   7.381399\n   0.0-1.5   0.000000\n   4.6-6.1   0.000000\n   6.1-7.7   0.000000\n   7.7-9.2   0.000000\n  9.2-10.8   0.000000\n 10.8-12.3   0.000000\n\nRecovered ODE:\ndY\/dt = 0.721 + -7.381 E1 + 12.808 E2\n\nPlot saved as 'fft_ode_band_mapping.png'\n\nDone!\n\nGlobal mode test complete.\n\nRunning in per-band mode...\nRF Quantum SCYTHE Client FFT ODE Demo - 2025-09-02 16:55:57\nAPI URL: http:\/\/localhost:8000\nPer-band mode: True\nFetching FFT plan from http:\/\/localhost:8000\/denoise\/hints...\nWarning: No FFT plan in response, using defaults\n\nGenerating synthetic RF data...\nGenerated 12288000 samples (1.0s at 12.288 MHz)\n\nComputing band energies...\nComputed 96 frames with 8 bands\n\nApplying FFT plan...\nApplied FFT plan with shift_bins=0\n\nRecovering ODE from band energies...\n\n=== Per-Band ODE Recovery Results ===\nMode: per_band\nRecovered ODEs for 8 bands\n\nPlot saved as 'fft_ode_band_mapping.png'\n\nDone!\n\nPer-band mode test complete..\/test_client_fft_ode_demo.sh\nStarting API server in the background...\nServer started with PID 565343\nServer is ready\n=== Testing RF Quantum SCYTHE Client FFT ODE Demo ===\n\nRunning in global mode (default)...\nRF Quantum SCYTHE Client FFT ODE Demo - 2025-09-02 16:55:53\nAPI URL: http:\/\/localhost:8000\nPer-band mode: False\nFetching FFT plan from http:\/\/localhost:8000\/denoise\/hints...\nWarning: No FFT plan in response, using defaults\n\nGenerating synthetic RF data...\nGenerated 12288000 samples (1.0s at 12.288 MHz)\n\nComputing band energies...\nComputed 96 frames with 8 bands\n\nApplying FFT plan...\nApplied FFT plan with shift_bins=0\n\nRecovering ODE from band energies...\n\n=== ODE Recovery Results ===\nMode: global\n\nBand Influence Ranking:\nBand (MHz)  Influence\n   3.1-4.6  12.807979\n   1.5-3.1   7.381399\n   0.0-1.5   0.000000\n   4.6-6.1   0.000000\n   6.1-7.7   0.000000\n   7.7-9.2   0.000000\n  9.2-10.8   0.000000\n 10.8-12.3   0.000000\n\nRecovered ODE:\ndY\/dt = 0.721 + -7.381 E1 + 12.808 E2\n\nPlot saved as 'fft_ode_band_mapping.png'\n\nDone!\n\nGlobal mode test complete.\n\nRunning in per-band mode...\nRF Quantum SCYTHE Client FFT ODE Demo - 2025-09-02 16:55:57\nAPI URL: http:\/\/localhost:8000\nPer-band mode: True\nFetching FFT plan from http:\/\/localhost:8000\/denoise\/hints...\nWarning: No FFT plan in response, using defaults\n\nGenerating synthetic RF data...\nGenerated 12288000 samples (1.0s at 12.288 MHz)\n\nComputing band energies...\nComputed 96 frames with 8 bands\n\nApplying FFT plan...\nApplied FFT plan with shift_bins=0\n\nRecovering ODE from band energies...\n\n=== Per-Band ODE Recovery Results ===\nMode: per_band\nRecovered ODEs for 8 bands\n\nPlot saved as 'fft_ode_band_mapping.png'\n\nDone!\n\nPer-band mode test complete.<\/code><\/pre>\n","protected":false},"excerpt":{"rendered":"<p>Let&#8217;s unpack the function recover_ode for a technically curious audience, especially those working in signal processing, sparse modeling, or data-driven dynamical systems: In many signal processing and physics-inspired machine learning tasks, we\u2019re often handed a time-frequency representation\u2014say, a matrix of energy across frequency bands over time\u2014and asked to infer the underlying dynamics. What drives the&hellip;&nbsp;<a href=\"https:\/\/172-234-197-23.ip.linodeusercontent.com\/?p=3073\" rel=\"bookmark\"><span class=\"screen-reader-text\">Recovering Dynamics from Energy Bands: A Sparse Regression Approach \ud83e\udde0<\/span><\/a><\/p>\n","protected":false},"author":1,"featured_media":3074,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"neve_meta_sidebar":"","neve_meta_container":"","neve_meta_enable_content_width":"","neve_meta_content_width":0,"neve_meta_title_alignment":"","neve_meta_author_avatar":"","neve_post_elements_order":"","neve_meta_disable_header":"","neve_meta_disable_footer":"","neve_meta_disable_title":"","footnotes":""},"categories":[10],"tags":[],"class_list":["post-3073","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-signal_scythe"],"_links":{"self":[{"href":"https:\/\/172-234-197-23.ip.linodeusercontent.com\/index.php?rest_route=\/wp\/v2\/posts\/3073","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/172-234-197-23.ip.linodeusercontent.com\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/172-234-197-23.ip.linodeusercontent.com\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/172-234-197-23.ip.linodeusercontent.com\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/172-234-197-23.ip.linodeusercontent.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=3073"}],"version-history":[{"count":2,"href":"https:\/\/172-234-197-23.ip.linodeusercontent.com\/index.php?rest_route=\/wp\/v2\/posts\/3073\/revisions"}],"predecessor-version":[{"id":3079,"href":"https:\/\/172-234-197-23.ip.linodeusercontent.com\/index.php?rest_route=\/wp\/v2\/posts\/3073\/revisions\/3079"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/172-234-197-23.ip.linodeusercontent.com\/index.php?rest_route=\/wp\/v2\/media\/3074"}],"wp:attachment":[{"href":"https:\/\/172-234-197-23.ip.linodeusercontent.com\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=3073"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/172-234-197-23.ip.linodeusercontent.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=3073"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/172-234-197-23.ip.linodeusercontent.com\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=3073"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}