{"id":3063,"date":"2025-09-02T00:56:59","date_gmt":"2025-09-02T00:56:59","guid":{"rendered":"https:\/\/172-234-197-23.ip.linodeusercontent.com\/?p=3063"},"modified":"2025-09-02T00:57:00","modified_gmt":"2025-09-02T00:57:00","slug":"discovering-interpretable-ordinary-differentialequations-from-noisy-data","status":"publish","type":"post","link":"https:\/\/172-234-197-23.ip.linodeusercontent.com\/?p=3063","title":{"rendered":"DISCOVERING INTERPRETABLE ORDINARY DIFFERENTIALEQUATIONS FROM NOISY DATA"},"content":{"rendered":"\n<p><strong>\u201cDiscovering Interpretable ODEs from Noisy Data\u201d<\/strong> slots beautifully into your FFT-plan + FBS calibrator pipeline.<\/p>\n\n\n\n<p>Here\u2019s how it helps your <strong>SCYTHE denoising + TDoA residual minimization loop<\/strong>:<\/p>\n\n\n\n<hr class=\"wp-block-separator has-alpha-channel-opacity\"\/>\n\n\n\n<h3 class=\"wp-block-heading\">\ud83d\udd11 Direct Relevance<\/h3>\n\n\n\n<ol class=\"wp-block-list\">\n<li><strong>ODE discovery as spectrum dynamics<\/strong>\n<ul class=\"wp-block-list\">\n<li>Instead of treating FFT bins as unrelated amplitudes, you can model their temporal evolution as a system of coupled ODEs.<\/li>\n\n\n\n<li>Example: a drifting carrier\u2019s residuals follow damped harmonic oscillator dynamics (like the spring\u2013mass system in the paper).<\/li>\n<\/ul>\n<\/li>\n\n\n\n<li><strong>Noise-robust spline approximation<\/strong>\n<ul class=\"wp-block-list\">\n<li>Their spline-based approach gives smooth derivatives even when the raw data is noisy.<\/li>\n\n\n\n<li>That\u2019s directly useful for your case: FFT bins \u2192 splines \u2192 derivative estimates \u2192 detect which bins carry <em>true dynamics<\/em> vs <em>stochastic noise<\/em>.<\/li>\n<\/ul>\n<\/li>\n\n\n\n<li><strong>Sparse coefficient recovery<\/strong>\n<ul class=\"wp-block-list\">\n<li>Their SVD-based null space trick naturally prunes spurious dynamics.<\/li>\n\n\n\n<li>In your case: only a small set of bins or harmonics will appear in the governing ODE; the rest get zeroed. This is a principled way to build your <em>bin shift + mask widening plan<\/em>.<\/li>\n<\/ul>\n<\/li>\n<\/ol>\n\n\n\n<hr class=\"wp-block-separator has-alpha-channel-opacity\"\/>\n\n\n\n<h3 class=\"wp-block-heading\">\u26a1 Concrete Patch You Can Try<\/h3>\n\n\n\n<p>You could write a small <strong>client stub<\/strong> that:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Pulls an <code>fft_plan<\/code> from <code>\/denoise\/hints<\/code><\/li>\n\n\n\n<li>Generates synthetic spectra over time with noise<\/li>\n\n\n\n<li>Fits a spline to each band<\/li>\n\n\n\n<li>Builds the gradient matrix (derivatives of splines)<\/li>\n\n\n\n<li>Runs SVD to infer which bins follow a coherent ODE<\/li>\n\n\n\n<li>Uses the recovered ODE to:\n<ul class=\"wp-block-list\">\n<li><strong>Guide mask widening<\/strong> (bands with coherent dynamics get protected)<\/li>\n\n\n\n<li><strong>Suggest bin shift priors<\/strong> (mu\/\u03c3 from FBS calibrator can be fused into coefficient estimates)<\/li>\n<\/ul>\n<\/li>\n<\/ul>\n\n\n\n<hr class=\"wp-block-separator has-alpha-channel-opacity\"\/>\n\n\n\n<h3 class=\"wp-block-heading\">\ud83e\uddea Proof-of-Concept Stub<\/h3>\n\n\n\n<pre class=\"wp-block-code\"><code>import numpy as np\nfrom scipy.interpolate import UnivariateSpline\nfrom numpy.linalg import svd\n\ndef fit_ode_from_fft(freqs, mags, order=2, smoothing=1e-3):\n    \"\"\"\n    Given a noisy FFT magnitude trace over time,\n    recover coefficients of an approximate ODE y'' + a y' + b y = 0.\n    \"\"\"\n    t = np.arange(len(mags))\n    spline = UnivariateSpline(t, mags, s=smoothing)\n    \n    y = spline(t)\n    dy = spline.derivative(1)(t)\n    d2y = spline.derivative(2)(t)\n    \n    # Build gradient matrix G: &#91;y, dy, d2y]\n    G = np.vstack(&#91;y, dy, d2y]).T\n    \n    # SVD to find null space (coefficients)\n    U, S, Vt = svd(G)\n    coeffs = Vt&#91;-1, :]  # last row is null space vector\n    \n    return coeffs \/ coeffs&#91;-1]  # normalize\n\n# Example usage: synthetic damped oscillator\nt = np.linspace(0, 10, 500)\nsignal = np.exp(-0.1*t) * np.sin(2*np.pi*0.5*t) + 0.05*np.random.randn(len(t))\n\ncoeffs = fit_ode_from_fft(np.fft.rfftfreq(len(signal)), signal)\nprint(\"Recovered ODE coefficients:\", coeffs)\n<\/code><\/pre>\n\n\n\n<p>This let&#8217;s the <strong>FFT client stub<\/strong> go beyond just masking bins \u2014 it learns which bins evolve coherently over time and uses that to refine <code>\/fft_plan<\/code>.<\/p>\n\n\n\n<hr class=\"wp-block-separator has-alpha-channel-opacity\"\/>\n\n\n\n<h3 class=\"wp-block-heading\">\ud83d\ude80 Integration Into SCYTHE<\/h3>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Wrap as <code>ode_recovery.py<\/code><\/li>\n\n\n\n<li>Called by your <code>test_fft_plan.sh<\/code> client after pulling <code>\/denoise\/hints<\/code><\/li>\n\n\n\n<li>Export recovered coefficients via Prometheus (<code>fft_bin_ode_coeffs{band=...}<\/code>)<\/li>\n\n\n\n<li>Grafana row: overlay recovered ODE phase on your \u03bc\/\u03c3 FBS curves<\/li>\n<\/ul>\n\n\n\n<div data-wp-interactive=\"core\/file\" class=\"wp-block-file\"><object data-wp-bind--hidden=\"!state.hasPdfPreview\" hidden class=\"wp-block-file__embed\" data=\"https:\/\/172-234-197-23.ip.linodeusercontent.com\/wp-content\/uploads\/2025\/09\/Discovering-Interpretable-Ordinary-Differential-Equations-from-Noisy-Data-2507.21841v1.pdf\" type=\"application\/pdf\" style=\"width:100%;height:600px\" aria-label=\"Embed of Discovering Interpretable Ordinary Differential Equations from Noisy Data 2507.21841v1.\"><\/object><a id=\"wp-block-file--media-6f589aeb-593c-4806-a6ff-c91d921c58ad\" href=\"https:\/\/172-234-197-23.ip.linodeusercontent.com\/wp-content\/uploads\/2025\/09\/Discovering-Interpretable-Ordinary-Differential-Equations-from-Noisy-Data-2507.21841v1.pdf\">Discovering Interpretable Ordinary Differential Equations from Noisy Data 2507.21841v1<\/a><a href=\"https:\/\/172-234-197-23.ip.linodeusercontent.com\/wp-content\/uploads\/2025\/09\/Discovering-Interpretable-Ordinary-Differential-Equations-from-Noisy-Data-2507.21841v1.pdf\" class=\"wp-block-file__button wp-element-button\" download aria-describedby=\"wp-block-file--media-6f589aeb-593c-4806-a6ff-c91d921c58ad\">Download<\/a><\/div>\n","protected":false},"excerpt":{"rendered":"<p>\u201cDiscovering Interpretable ODEs from Noisy Data\u201d slots beautifully into your FFT-plan + FBS calibrator pipeline. Here\u2019s how it helps your SCYTHE denoising + TDoA residual minimization loop: \ud83d\udd11 Direct Relevance \u26a1 Concrete Patch You Can Try You could write a small client stub that: \ud83e\uddea Proof-of-Concept Stub This let&#8217;s the FFT client stub go beyond&hellip;&nbsp;<a href=\"https:\/\/172-234-197-23.ip.linodeusercontent.com\/?p=3063\" rel=\"bookmark\"><span class=\"screen-reader-text\">DISCOVERING INTERPRETABLE ORDINARY DIFFERENTIALEQUATIONS FROM NOISY DATA<\/span><\/a><\/p>\n","protected":false},"author":1,"featured_media":2534,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"neve_meta_sidebar":"","neve_meta_container":"","neve_meta_enable_content_width":"","neve_meta_content_width":0,"neve_meta_title_alignment":"","neve_meta_author_avatar":"","neve_post_elements_order":"","neve_meta_disable_header":"","neve_meta_disable_footer":"","neve_meta_disable_title":"","footnotes":""},"categories":[6,10],"tags":[],"class_list":["post-3063","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-signal-science","category-signal_scythe"],"_links":{"self":[{"href":"https:\/\/172-234-197-23.ip.linodeusercontent.com\/index.php?rest_route=\/wp\/v2\/posts\/3063","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/172-234-197-23.ip.linodeusercontent.com\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/172-234-197-23.ip.linodeusercontent.com\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/172-234-197-23.ip.linodeusercontent.com\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/172-234-197-23.ip.linodeusercontent.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=3063"}],"version-history":[{"count":1,"href":"https:\/\/172-234-197-23.ip.linodeusercontent.com\/index.php?rest_route=\/wp\/v2\/posts\/3063\/revisions"}],"predecessor-version":[{"id":3067,"href":"https:\/\/172-234-197-23.ip.linodeusercontent.com\/index.php?rest_route=\/wp\/v2\/posts\/3063\/revisions\/3067"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/172-234-197-23.ip.linodeusercontent.com\/index.php?rest_route=\/wp\/v2\/media\/2534"}],"wp:attachment":[{"href":"https:\/\/172-234-197-23.ip.linodeusercontent.com\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=3063"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/172-234-197-23.ip.linodeusercontent.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=3063"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/172-234-197-23.ip.linodeusercontent.com\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=3063"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}