{"id":5200,"date":"2026-03-23T01:20:29","date_gmt":"2026-03-23T01:20:29","guid":{"rendered":"https:\/\/172-234-197-23.ip.linodeusercontent.com\/?p=5200"},"modified":"2026-03-23T01:22:00","modified_gmt":"2026-03-23T01:22:00","slug":"scythe-intelligence-organism","status":"publish","type":"post","link":"https:\/\/172-234-197-23.ip.linodeusercontent.com\/?p=5200","title":{"rendered":"SCYTHE Intelligence Organism"},"content":{"rendered":"\n
\u276f Perfect Claude! Everything is working great. | Create a Blog Post for our Advancements please, thank you! xoxo\n\n\u25d0 The user wants me to create a blog post about the advancements made in the SCYTHE\/NerfEngine project. Let me read the plan and recent \n  checkpoints to understand the full scope of what's been built, then write an engaging blog post.\n Let me write an engaging blog post that covers the full journey of the NerfEngine \/ RF \nSCYTHE project. I'll make it technical but readable, covering:\n\n1. The overall vision\n2. The Android app journey (ATAK plugin \u2192 ScytheCommandApp)\n3. The hypergraph\/analytics engine\n4. The inference pipeline (Gemma\/EmbeddingGemma)\n5. The shadow graph\n6. The Threat Gravity Map\n7. What's next\n - Mixed tone (technical sections with punchy intros\/outros)\n  - General cybersecurity \/ infosec audience\n  - Personal blog \/ website\n  - Include code snippets\n\n  Let me write a comprehensive, engaging blog post that covers the full SCYTHE journey across all 23 checkpoints. I'll organize it into major \n  milestone phases and make it feel like a genuine development journey narrative with technical depth. >><\/code><\/pre>\n\n\n\n
#<\/strong> Building SCYTHE: From Android Plugin to Living Intelligence Organism<\/strong><\/p>\n\n\n\n
> *How we turned a tactical RF sensor tool into a self-healing, graph-cognitive threat intelligence engine \u2014 one late night at a time.*<\/p>\n\n\n\n
—<\/strong><\/p>\n\n\n\n
##<\/strong> The Origin: A Map, a Radio, and a Crazy Idea<\/strong><\/p>\n\n\n\n
It started with a simple enough premise: take [ATAK (Android Tactical Assault Kit)](https:\/\/github.com\/deptofdefense\/AndroidTacticalAssaultKit-CIV), the U.S. Department of Defense’s open-source situational awareness platform, and wire it up to a real-time RF signal intelligence backend.<\/p>\n\n\n\n
The name: **<\/strong>SCYTHE<\/strong>**<\/strong> \u2014 a system for cross-domain signals, hypergraph analytics, and threat environment mapping.<\/p>\n\n\n\n
What followed was 23 engineering checkpoints, hundreds of log lines, three complete architectural pivots, and a system that now does things we didn’t plan for when we started.<\/p>\n\n\n\n
Here’s the full story.<\/p>\n\n\n\n
—<\/strong><\/p>\n\n\n\n
##<\/strong> Phase 1 \u2014 The Plugin That Almost Wasn’t<\/strong><\/p>\n\n\n\n
The original approach was straightforward: build an ATAK plugin that streams CoT (Cursor-on-Target) entity events into a Python analytics backend. Four tabs, RF signal dots on a map, swarm detection.<\/p>\n\n\n\n
We built it. It worked.<\/p>\n\n\n\n
Then we deployed to a Pixel 7 Pro running **<\/strong>Android 16<\/strong>**<\/strong>.<\/p>\n\n\n\n
“`<\/p>\n\n\n\n
FATAL: libsqlite3.so not found<\/p>\n\n\n\n
Process: com.atakmap.app.civ, Signal: SIGABRT<\/p>\n\n\n\n
“`<\/p>\n\n\n\n
ATAK CIV 4.6.0 was compiled against pre-bionic-hardening ABI. Android 16 removed `libsqlite3.so` from the system image. The app crashed before it could load a single plugin.<\/p>\n\n\n\n
**<\/strong>The plugin worked perfectly. The host app was the problem.<\/strong>**<\/strong><\/p>\n\n\n\n
Rather than wait for TAK.gov to ship ATAK 5.x, we pivoted to a **<\/strong>standalone WebView shell<\/strong>**<\/strong> \u2014 `ScytheCommandApp` \u2014 loading the full 35K-line `command-ops-visualization.html` directly from the orchestrator. All the Cesium 3D globe, RF hypergraph visualization, and recon entity rendering reused immediately. Zero GDAL fragility.<\/p>\n\n\n\n
*The lesson: don’t fight the platform. Route around it.*<\/p>\n\n\n\n
—<\/strong><\/p>\n\n\n\n
##<\/strong> Phase 2 \u2014 The Intelligence Backend: Hypergraph Analytics Engine<\/strong><\/p>\n\n\n\n
While the Android story was playing out, the Python backend was growing into something much larger.<\/p>\n\n\n\n
The core data model is a **<\/strong>hypergraph<\/strong>**<\/strong>: nodes are entities (hosts, sessions, RF signatures, network flows), and edges encode *relationships* \u2014 not just “A connected to B” but typed, confidence-weighted assertions like `INFERRED_LATERAL_MOVEMENT`, `TRIANGULATED_FROM`, `CO_MOVED_WITH`.<\/p>\n\n\n\n
“`<\/p>\n\n\n\n
HGNode(id=”172.234.197.23″, kind=”ip_host”)<\/p>\n\n\n\n
  \u2514\u2500 INFERRED_FLOW_IN_SERVICE \u2192 HGNode(“10.0.0.44″, kind=”ip_host”)<\/p>\n\n\n\n
  \u2514\u2500 INFERRED_LATERAL_MOVEMENT \u2192 HGNode(“SESSION-911b32eb”, kind=”session”)<\/p>\n\n\n\n
  \u2514\u2500 CO_MOVED_WITH \u2192 HGNode(“RF-A4:B1:…”, kind=”rf_emitter”)<\/p>\n\n\n\n
“`<\/p>\n\n\n\n
On top of this we built:<\/p>\n\n\n\n
– **<\/strong>ScytheDuckStore<\/strong>**<\/strong> \u2014 DuckDB event store, 54ms bulk insert \/ 10ms scrub<\/p>\n\n\n\n
– **<\/strong>ParquetPipeline<\/strong>**<\/strong> \u2014 ZSTD Parquet cold storage, 3\u00d7 compression<\/p>\n\n\n\n
– **<\/strong>SpaceTimeCube<\/strong>**<\/strong> \u2014 0.01\u00b0 geo \u00d7 5s time voxels, 0.14ms radius+time queries<\/p>\n\n\n\n
– **<\/strong>ClusterSwarmEngine<\/strong>**<\/strong> \u2014 geo-bucket cluster detection \u2192 CoT swarm events<\/p>\n\n\n\n
– **<\/strong>TakSwarmEmitter<\/strong>**<\/strong> \u2014 PyTAK CoT emitter back into the ATAK ecosystem<\/p>\n\n\n\n
This ran inside an **<\/strong>AVF (Android Virtualization Framework)<\/strong>**<\/strong> Debian VM \u2014 8-core Tensor G2, 3.8GB RAM, 70GB disk \u2014 provisioned inside the Pixel itself via crosvm.<\/p>\n\n\n\n
—<\/strong><\/p>\n\n\n\n
##<\/strong> Phase 3 \u2014 The Intelligence Layer: Gemma + EmbeddingGemma<\/strong><\/p>\n\n\n\n
Here’s where things got interesting.<\/p>\n\n\n\n
SCYTHE doesn’t just store events. It *reasons about them*. We wired in two local Ollama models:<\/p>\n\n\n\n
– **<\/strong>Gemma<\/strong>**<\/strong> (inference model) \u2014 given a graph node and a set of rule prompts, produce new inferred edges<\/p>\n\n\n\n
– **<\/strong>EmbeddingGemma<\/strong>**<\/strong> \u2014 produce semantic embeddings for edge kind repair and entity clustering<\/p>\n\n\n\n
###<\/strong> The Rule Prompt Architecture<\/strong><\/p>\n\n\n\n
Each inference cycle works like this:<\/p>\n\n\n\n
1. Pick a graph node (typically a recently-active host or session)<\/p>\n\n\n\n
2. Build a structured prompt: “Here are 12 known edge kinds. Here is what we know about this entity. What edges should exist?”<\/p>\n\n\n\n
3. Gemma returns structured JSON \u2014 a list of `{ rule_id, should_fire, inferred_edges: […] }`<\/p>\n\n\n\n
4. Validator checks each edge against the schema<\/p>\n\n\n\n
5. Valid edges \u2192 committed to hypergraph via `WriteBus`<\/p>\n\n\n\n
6. Invalid edges \u2192 shadow graph (more on this later)<\/p>\n\n\n\n
###<\/strong> The Silent Killers We Fixed<\/strong><\/p>\n\n\n\n
After running this under real load, we discovered four concurrent failure modes that were destroying pipeline throughput silently:<\/p>\n\n\n\n
**<\/strong>1. Model Hallucination as Edge Kinds<\/strong>**<\/strong><\/p>\n\n\n\n
Gemma was emitting things like:<\/p>\n\n\n\n
“`<\/p>\n\n\n\n
kind: “session_observed_SESSION-8abf8b865c15fce3”<\/p>\n\n\n\n
“`<\/p>\n\n\n\n
The model was hallucinating *session IDs as edge schema types*. The validator was correctly rejecting these, but then wasting compute cycles trying to semantically repair them.<\/p>\n\n\n\n
Fix: added a regex pattern match in `normalize_edge_kind` \u2014 anything matching `SESSION_OBSERVED_<hex>` gets a schema-policy drop (empty string return, logged at DEBUG, no repair attempt):<\/p>\n\n\n\n
“`python<\/p>\n\n\n\n
if re.match(r’^SESSION_OBSERVED_[0-9a-f]{8,}’, kind_upper):<\/p>\n\n\n\n
    return “”  # schema-policy drop \u2014 model hallucinated a session ID as a kind<\/p>\n\n\n\n
“`<\/p>\n\n\n\n
**<\/strong>2. JSON-String-Wrapped Dicts<\/strong>**<\/strong><\/p>\n\n\n\n
Sometimes Gemma would return a valid JSON structure, but one of the items in the list would itself be a *JSON-encoded string* rather than a dict:<\/p>\n\n\n\n
“`python<\/p>\n\n\n\n
# What we got:<\/p>\n\n\n\n
[“{\\”rule_id\\”: \\”R-0001\\”, \\”inferred_edges\\”: […]}”, {…}, …]<\/p>\n\n\n\n
# What we expected:<\/p>\n\n\n\n
[{“rule_id”: “R-0001”, …}, …]<\/p>\n\n\n\n
“`<\/p>\n\n\n\n
Fix: coerce in `_validate_gemma_output_inner`:<\/p>\n\n\n\n
“`python<\/p>\n\n\n\n
if isinstance(item, str):<\/p>\n\n\n\n
    try:<\/p>\n\n\n\n
        item = json.loads(item)<\/p>\n\n\n\n
    except json.JSONDecodeError:<\/p>\n\n\n\n
        continue  # drop malformed string<\/p>\n\n\n\n
“`<\/p>\n\n\n\n
**<\/strong>3. Missing <\/strong>`<\/strong>rule_id<\/strong>`<\/strong> \/ <\/strong>`<\/strong>should_fire<\/strong>`<\/strong>**<\/strong><\/p>\n\n\n\n
Timeout-corrupted responses would sometimes have `inferred_edges` but no `rule_id`. The validator discarded the entire item.<\/p>\n\n\n\n
Fix: synthesize a recovery stub if edges are present:<\/p>\n\n\n\n
“`python<\/p>\n\n\n\n
if “rule_id” not in item and “inferred_edges” in item:<\/p>\n\n\n\n
    item[“rule_id”] = “R-RECOVERED”<\/p>\n\n\n\n
    item[“should_fire”] = True<\/p>\n\n\n\n
“`<\/p>\n\n\n\n
**<\/strong>4. The 15-Minute Gemma Blackout<\/strong>**<\/strong><\/p>\n\n\n\n
The worst one. With the GPU in a degraded state:<\/p>\n\n\n\n
– Timeout: 300s per attempt<\/p>\n\n\n\n
– Retries: 3<\/p>\n\n\n\n
– **<\/strong>Worst case: 900 seconds (15 minutes) of blocked inference per node<\/strong>**<\/strong><\/p>\n\n\n\n
The entire pipeline would stall. No logs, no errors \u2014 just silence.<\/p>\n\n\n\n
**<\/strong>Fix: Circuit Breaker<\/strong>**<\/strong><\/p>\n\n\n\n
“`python<\/p>\n\n\n\n
class GemmaCircuitBreaker:<\/p>\n\n\n\n
    “””Opens after 3 consecutive failures. Cooldown: 60s. Half-open probe on recovery.”””<\/p>\n\n\n\n
    def __init__(self, threshold=3, cooldown=60):<\/p>\n\n\n\n
        self.threshold = threshold<\/p>\n\n\n\n
        self.cooldown = cooldown<\/p>\n\n\n\n
        self._failures = 0<\/p>\n\n\n\n
        self._opened_at = None<\/p>\n\n\n\n
        self._state = “closed”  # closed | open | half_open<\/p>\n\n\n\n
    def allow_request(self) -> bool:<\/p>\n\n\n\n
        if self._state == “closed”:<\/p>\n\n\n\n
            return True<\/p>\n\n\n\n
        if self._state == “open”:<\/p>\n\n\n\n
            if time.time() – self._opened_at > self.cooldown:<\/p>\n\n\n\n
                self._state = “half_open”<\/p>\n\n\n\n
                return True  # probe<\/p>\n\n\n\n
            return False<\/p>\n\n\n\n
        return True  # half_open: allow probe<\/p>\n\n\n\n
    def record_success(self):<\/p>\n\n\n\n
        self._failures = 0<\/p>\n\n\n\n
        self._state = “closed”<\/p>\n\n\n\n
    def record_failure(self):<\/p>\n\n\n\n
        self._failures += 1<\/p>\n\n\n\n
        if self._failures >= self.threshold:<\/p>\n\n\n\n
            self._state = “open”<\/p>\n\n\n\n
            self._opened_at = time.time()<\/p>\n\n\n\n
“`<\/p>\n\n\n\n
New worst case: 45s \u00d7 2 retries = 90 seconds, then the circuit opens for 60s. The pipeline continues processing other nodes.<\/p>\n\n\n\n
—<\/strong><\/p>\n\n\n\n
##<\/strong> Phase 4 \u2014 The Shadow Graph: Pre-Reality Buffer<\/strong><\/p>\n\n\n\n
One of the most interesting architectural pieces we built is the **<\/strong>Shadow Graph<\/strong>**<\/strong>.<\/p>\n\n\n\n
The problem: the validator was too strict. Edges with `unknown_src`, `unknown_dst`, or `invalid_kind` were simply dropped. But many of these were *actually valid* \u2014 the referenced nodes just hadn’t been materialized into the graph yet.<\/p>\n\n\n\n
We were deleting real intelligence because the graph wasn’t ready to receive it.<\/p>\n\n\n\n
**<\/strong>The Shadow Graph holds these rejected edges in a probabilistic pre-reality buffer:<\/strong>**<\/strong><\/p>\n\n\n\n
“`python<\/p>\n\n\n\n
@dataclass<\/p>\n\n\n\n
class ShadowEdge:<\/p>\n\n\n\n
    src: str<\/p>\n\n\n\n
    dst: str<\/p>\n\n\n\n
    kind: str<\/p>\n\n\n\n
    confidence: float<\/p>\n\n\n\n
    rejection_reason: str       # “unknown_src” | “unknown_dst” | “invalid_kind”<\/p>\n\n\n\n
    context_node_id: str<\/p>\n\n\n\n
    created_at: float<\/p>\n\n\n\n
    ttl: float = 300.0          # 5 minutes<\/p>\n\n\n\n
    attempt_count: int = 0<\/p>\n\n\n\n
“`<\/p>\n\n\n\n
After every materialization pass (when new stub nodes are added to the graph), the shadow graph calls `re_evaluate(known_node_ids)` \u2014 any edge whose `src` AND `dst` are now in the graph gets **<\/strong>promoted<\/strong>**<\/strong>:<\/p>\n\n\n\n
“`<\/p>\n\n\n\n
[shadow] PROMOTED session\u2192host edge kind=flow_observed<\/p>\n\n\n\n
  after 1 attempts, age=2.3s<\/p>\n\n\n\n
“`<\/p>\n\n\n\n
When you see this in the logs, the system isn’t just processing data.<\/p>\n\n\n\n
**<\/strong>It’s beginning to believe its own hypotheses.<\/strong>**<\/strong><\/p>\n\n\n\n
The shadow graph exposes `\/api\/shadow\/summary` and `\/api\/shadow\/edges` so the UI can visualize what the system *suspects but hasn’t confirmed yet*.<\/p>\n\n\n\n
—<\/strong><\/p>\n\n\n\n
##<\/strong> Phase 5 \u2014 Semantic Edge Repair<\/strong><\/p>\n\n\n\n
Not every invalid edge kind is a hallucination. Sometimes the model produces a semantically correct edge in a non-canonical form.<\/p>\n\n\n\n
`FLOW_OBSERVED` \u2192 should be `INFERRED_FLOW_IN_SERVICE`  <\/p>\n\n\n\n
`HOST_OBSERVED` \u2192 should be `INFERRED_HOST_ROLE`  <\/p>\n\n\n\n
`OBSERVED` \u2192 should be `INFERRED_NODE_EXISTS`<\/p>\n\n\n\n
We handle two layers of repair:<\/p>\n\n\n\n
**<\/strong>Layer 1: Static Aliases<\/strong>**<\/strong> (`rule_prompt.py`)<\/p>\n\n\n\n
“`python<\/p>\n\n\n\n
EDGE_KIND_ALIASES = {<\/p>\n\n\n\n
    “FLOW_OBSERVED”:          “INFERRED_FLOW_IN_SERVICE”,<\/p>\n\n\n\n
    “FLOW_OBSERVED_PORT”:     “INFERRED_FLOW_IN_SERVICE”,<\/p>\n\n\n\n
    “HOST_OBSERVED”:          “INFERRED_HOST_ROLE”,<\/p>\n\n\n\n
    “HOST_OBSERVED_SERVICE”:  “INFERRED_HOST_OFFERS_SERVICE”,<\/p>\n\n\n\n
    “OBSERVED”:               “INFERRED_NODE_EXISTS”,<\/p>\n\n\n\n
    # … ~40 total aliases<\/p>\n\n\n\n
}<\/p>\n\n\n\n
“`<\/p>\n\n\n\n
**<\/strong>Layer 2: Semantic Repair<\/strong>**<\/strong> (`semantic_edge_repair.py`)<\/p>\n\n\n\n
For anything that doesn’t hit a static alias, we compute cosine similarity against EmbeddingGemma embeddings of all known canonical kinds. If similarity \u2265 0.65 (tuned down from 0.82 after live log analysis showed consistent ~0.65 valid repairs being rejected), the kind is repaired:<\/p>\n\n\n\n
“`<\/p>\n\n\n\n
SemanticEdgeRepair: FLOW_OBSERVED_SERVICE \u2192 INFERRED_HOST_OFFERS_SERVICE<\/p>\n\n\n\n
  cosine_similarity=0.71, threshold=0.65 \u2713<\/p>\n\n\n\n
“`<\/p>\n\n\n\n
—<\/strong><\/p>\n\n\n\n
##<\/strong> Phase 6 \u2014 Multi-Instance Orchestrator<\/strong><\/p>\n\n\n\n
A single SCYTHE instance processes one investigation context. We extended this into a full **<\/strong>multi-instance orchestrator<\/strong>**<\/strong>:<\/p>\n\n\n\n
“`<\/p>\n\n\n\n
ScytheOrchestrator (port 5001)<\/p>\n\n\n\n
    \u251c\u2500\u2500 scythe-a43e657c (port 45574) \u2014 Investigation A<\/p>\n\n\n\n
    \u2502       \u2514\u2500\u2500 HypergraphEngine (own sqlite, own Gemma queue)<\/p>\n\n\n\n
    \u251c\u2500\u2500 scythe-b7f21e90 (port 51283) \u2014 Investigation B  <\/p>\n\n\n\n
    \u2502       \u2514\u2500\u2500 HypergraphEngine (own sqlite, own Gemma queue)<\/p>\n\n\n\n
    \u2514\u2500\u2500 …<\/p>\n\n\n\n
“`<\/p>\n\n\n\n
*”Each instance: one sovereign hypergraph. No shared memory. No shared state. No accidental cross-contamination.”*<\/p>\n\n\n\n
Each instance registers with the orchestrator, gets assigned a random high port, and maintains its own:<\/p>\n\n\n\n
– SQLite hypergraph store<\/p>\n\n\n\n
– Gemma inference queue<\/p>\n\n\n\n
– Stream relay WebSocket connection<\/p>\n\n\n\n
– PCAP\/FTP ingest pipeline<\/p>\n\n\n\n
The orchestrator provides `\/api\/scythe\/instances`, `\/api\/scythe\/instances\/new`, and health endpoints. The UI shows all active instances and allows spawning new ones directly from the command panel.<\/p>\n\n\n\n
—<\/strong><\/p>\n\n\n\n
##<\/strong> Phase 7 \u2014 The Geolocation Fusion Engine<\/strong><\/p>\n\n\n\n
By checkpoint 19 we had enough RTT and traceroute data flowing to attempt real **<\/strong>timing-based geolocation<\/strong>**<\/strong>:<\/p>\n\n\n\n
– **<\/strong>`<\/strong>\/api\/timing\/probe<\/strong>`<\/strong>**<\/strong> \u2014 server-side ICMP ping RTT \u2192 distance estimate (62.5 km\/ms, speed-of-light in fiber)<\/p>\n\n\n\n
– **<\/strong>`<\/strong>\/api\/timing\/traceroute<\/strong>`<\/strong>**<\/strong> \u2014 real traceroute with per-hop RTT + cumulative km<\/p>\n\n\n\n
– **<\/strong>`<\/strong>\/api\/timing\/tdoa<\/strong>`<\/strong>**<\/strong> \u2014 multi-observer TDoA (Time Difference of Arrival) trilateration via gradient descent least-squares<\/p>\n\n\n\n
“`python<\/p>\n\n\n\n
def tdoa_trilateration(observers: list[dict]) -> dict:<\/p>\n\n\n\n
    “””<\/p>\n\n\n\n
    Gradient descent least-squares TDoA fix.<\/p>\n\n\n\n
    observers: [{“lat”: …, “lon”: …, “rtt_ms”: …}, …]<\/p>\n\n\n\n
    returns: {“lat”: est_lat, “lon”: est_lon, “confidence”: 0-1}<\/p>\n\n\n\n
    “””<\/p>\n\n\n\n
“`<\/p>\n\n\n\n
The IPv6 audit that preceded this revealed 302 entities with IPv6 addresses that weren’t being resolved correctly \u2014 their geolocation was defaulting to null islands in the Atlantic. Fixed by extending the geolocation fusion pipeline to handle `::ffff:`-prefixed IPv4-mapped IPv6 addresses and dual-stack resolution.<\/p>\n\n\n\n
—<\/strong><\/p>\n\n\n\n
##<\/strong> Phase 8 \u2014 MIMO Hop Classifier<\/strong><\/p>\n\n\n\n
One of the more technically specific modules: a **<\/strong>MIMO (Multiple-Input Multiple-Output) hop classifier<\/strong>**<\/strong> for analyzing RF signal paths through intermediate relay nodes.<\/p>\n\n\n\n
Real RF environments have multi-hop paths. A signal doesn’t go directly from emitter to receiver \u2014 it bounces off terrain, refracts through layers, or routes through intentional relay infrastructure. The MIMO classifier:<\/p>\n\n\n\n
1. Takes a set of observed signal characteristics (timing, frequency offset, phase coherence)<\/p>\n\n\n\n
2. Classifies the likely hop count and relay topology<\/p>\n\n\n\n
3. Produces `INFERRED_RF_RELAY_HOP` edges in the hypergraph<\/p>\n\n\n\n
This feeds directly into the **<\/strong>geo-stability score<\/strong>**<\/strong> \u2014 if an entity’s IP is shifting but its RF fingerprint (latency profile + frequency drift + phase coherence) is stable, the *physical location is anchored* even if the identity layer is churning.<\/p>\n\n\n\n
—<\/strong><\/p>\n\n\n\n
##<\/strong> Phase 9 \u2014 The Threat Gravity Map \ud83c\udf0c<\/strong><\/p>\n\n\n\n
The most visually dramatic piece we’ve built so far.<\/p>\n\n\n\n
The problem with graph visualization at scale: 1,000+ nodes is unreadable as a traditional force-directed graph. We needed a physics metaphor that *reveals structure* rather than obscuring it.<\/p>\n\n\n\n
**<\/strong>The insight: treat threat level as mass.<\/strong>**<\/strong><\/p>\n\n\n\n
“`python<\/p>\n\n\n\n
def compute_mass(node):<\/p>\n\n\n\n
    return (<\/p>\n\n\n\n
        0.30 * log(node.degree + 1) +           # connectivity<\/p>\n\n\n\n
        0.25 * log(node.flow_count + 1) +        # traffic volume<\/p>\n\n\n\n
        0.20 * node.persistence_score +          # temporal stability<\/p>\n\n\n\n
        0.15 * node.anomaly_score +              # behavioral anomaly<\/p>\n\n\n\n
        0.10 * node.shadow_promotion_count       # early-indicator weight<\/p>\n\n\n\n
    )<\/p>\n\n\n\n
“`<\/p>\n\n\n\n
High-mass (high-threat) nodes *pull nearby nodes toward them*. Clusters form naturally. Infrastructure that coordinates \u2014 botnets, C2 beacons, proxy chains \u2014 compresses into visible clumps without any rules.<\/p>\n\n\n\n
**<\/strong>Frontend: Canvas-based force simulation<\/strong>**<\/strong><\/p>\n\n\n\n
No D3, no WebGL \u2014 pure canvas 2D with a custom Verlet integration loop:<\/p>\n\n\n\n
“`javascript<\/p>\n\n\n\n
function applyThreatGravity(alpha) {<\/p>\n\n\n\n
    for (let i = 0; i < nodes.length; i++) {<\/p>\n\n\n\n
        for (let j = i + 1; j < nodes.length; j++) {<\/p>\n\n\n\n
            const a = nodes[i], b = nodes[j];<\/p>\n\n\n\n
            if (a.mass < 1 || b.mass < 1) continue;<\/p>\n\n\n\n
            const dx = b.x – a.x, dy = b.y – a.y;<\/p>\n\n\n\n
            const dist = Math.sqrt(dx*dx + dy*dy) + 0.1;<\/p>\n\n\n\n
            const force = (a.mass * b.mass) \/ (dist * dist) * alpha * 0.3;<\/p>\n\n\n\n
            a.vx += force * dx \/ dist;<\/p>\n\n\n\n
            a.vy += force * dy \/ dist;<\/p>\n\n\n\n
            b.vx -= force * dx \/ dist;<\/p>\n\n\n\n
            b.vy -= force * dy \/ dist;<\/p>\n\n\n\n
        }<\/p>\n\n\n\n
    }<\/p>\n\n\n\n
}<\/p>\n\n\n\n
“`<\/p>\n\n\n\n
**<\/strong>Visual encoding:<\/strong>**<\/strong><\/p>\n\n\n\n
| Property | Visual |<\/p>\n\n\n\n
|———-|——–|<\/p>\n\n\n\n
| Node size | Mass (threat weight) |<\/p>\n\n\n\n
| Glow intensity | Anomaly score |<\/p>\n\n\n\n
| Pulse rate | Temporal activity \/ recency |<\/p>\n\n\n\n
| Purple ring | Shadow promotions pending |<\/p>\n\n\n\n
| Color: blue | Low threat (benign) |<\/p>\n\n\n\n
| Color: yellow | Uncertain (in shadow graph) |<\/p>\n\n\n\n
| Color: red | High-confidence threat |<\/p>\n\n\n\n
The panel auto-refreshes every 10 seconds, reheats the simulation on new data, and supports drag, pan (mouse drag on background), and zoom (scroll wheel). Hover tooltip shows ID, kind, mass, anomaly score, and pending shadow promotions.<\/p>\n\n\n\n
###<\/strong> The Circular Reference Crash<\/strong><\/p>\n\n\n\n
Before the gravity map was live, we hit a fun one:<\/p>\n\n\n\n
“`<\/p>\n\n\n\n
ValueError: Circular reference detected<\/p>\n\n\n\n
  File “rf_scythe_api_server.py”, line 10299, in pcap_session_subgraph<\/p>\n\n\n\n
    return jsonify({…})<\/p>\n\n\n\n
“`<\/p>\n\n\n\n
The hypergraph’s `HGNode` and `HGEdge` objects had back-references to each other \u2014 `node.edges \u2192 edge.nodes \u2192 node…`. Python’s `json.dumps` detected the cycle and threw.<\/p>\n\n\n\n
The fix: replace `_as_dict` with a cycle-safe serializer:<\/p>\n\n\n\n
“`python<\/p>\n\n\n\n
def _safe_serial(obj, _seen=None):<\/p>\n\n\n\n
    if _seen is None:<\/p>\n\n\n\n
        _seen = set()<\/p>\n\n\n\n
    oid = id(obj)<\/p>\n\n\n\n
    if oid in _seen:<\/p>\n\n\n\n
        return “__circular_ref__”<\/p>\n\n\n\n
    _seen.add(oid)<\/p>\n\n\n\n
    try:<\/p>\n\n\n\n
        if isinstance(obj, dict):<\/p>\n\n\n\n
            return {k: _safe_serial(v, _seen) for k, v in obj.items()}<\/p>\n\n\n\n
        elif isinstance(obj, list):<\/p>\n\n\n\n
            return [_safe_serial(v, _seen) for v in obj]<\/p>\n\n\n\n
        elif hasattr(obj, ‘__dict__’):<\/p>\n\n\n\n
            return _safe_serial(vars(obj), _seen)<\/p>\n\n\n\n
        else:<\/p>\n\n\n\n
            return obj<\/p>\n\n\n\n
    finally:<\/p>\n\n\n\n
        _seen.discard(oid)  # allow same object in different branches<\/p>\n\n\n\n
“`<\/p>\n\n\n\n
The key insight: `_seen.discard(oid)` in `finally` \u2014 the same object can appear in *different branches* of the tree legitimately. Only true cycles (where we’d loop forever) get the `__circular_ref__` marker. The UI receives valid JSON with a string sentinel at cycle points.<\/p>\n\n\n\n
—<\/strong><\/p>\n\n\n\n
##<\/strong> Where We Are Now<\/strong><\/p>\n\n\n\n
After 23 checkpoints, here’s the current stack:<\/p>\n\n\n\n
“`<\/p>\n\n\n\n
Pixel 7 Pro<\/p>\n\n\n\n
    \u2514\u2500\u2500 ScytheCommandApp (WebView shell)<\/p>\n\n\n\n
            \u2514\u2500\u2500 http:\/\/192.168.1.185:5001\/command-ops-visualization.html<\/p>\n\n\n\n
                    \u251c\u2500\u2500 Cesium 3D globe (live entity markers)<\/p>\n\n\n\n
                    \u251c\u2500\u2500 RF Hypergraph visualization<\/p>\n\n\n\n
                    \u251c\u2500\u2500 \ud83c\udf0c Threat Gravity Map (force simulation)<\/p>\n\n\n\n
                    \u251c\u2500\u2500 Recon Panel (1,185+ entities, IPv6-aware)<\/p>\n\n\n\n
                    \u251c\u2500\u2500 MIMO Hop Classifier panel<\/p>\n\n\n\n
                    \u251c\u2500\u2500 GraphOps LLM chat (Gemma-backed)<\/p>\n\n\n\n
                    \u2514\u2500\u2500 COMMAND CONSOLE, OPERATORS, and more<\/p>\n\n\n\n
NerfEngine Server (192.168.1.185)<\/p>\n\n\n\n
    \u251c\u2500\u2500 Scythe Orchestrator (port 5001)<\/p>\n\n\n\n
    \u2502       \u2514\u2500\u2500 Multi-instance manager (dynamic port assignment)<\/p>\n\n\n\n
    \u251c\u2500\u2500 rf_scythe_api_server.py (per-instance, dynamic port)<\/p>\n\n\n\n
    \u2502       \u251c\u2500\u2500 HypergraphEngine (SQLite-WAL, incremental)<\/p>\n\n\n\n
    \u2502       \u251c\u2500\u2500 Gemma inference pipeline (circuit breaker, shadow graph)<\/p>\n\n\n\n
    \u2502       \u251c\u2500\u2500 EmbeddingGemma semantic repair<\/p>\n\n\n\n
    \u2502       \u251c\u2500\u2500 FTP\/PCAP ingest pipeline<\/p>\n\n\n\n
    \u2502       \u251c\u2500\u2500 Timing geolocation (probe\/traceroute\/TDoA)<\/p>\n\n\n\n
    \u2502       \u251c\u2500\u2500 \/api\/gravity\/* (Threat Gravity Map backend)<\/p>\n\n\n\n
    \u2502       \u2514\u2500\u2500 \/api\/shadow\/* (Shadow Graph inspection)<\/p>\n\n\n\n
    \u251c\u2500\u2500 stream_relay.py (WebSocket relay, ws:\/\/…:8765\/ws)<\/p>\n\n\n\n
    \u251c\u2500\u2500 shadow_graph.py (pre-reality edge buffer, 5-min TTL)<\/p>\n\n\n\n
    \u2514\u2500\u2500 tak_ml_gemma_runner.py (Gemma + circuit breaker)<\/p>\n\n\n\n
“`<\/p>\n\n\n\n
—<\/strong><\/p>\n\n\n\n
##<\/strong> What’s Coming<\/strong><\/p>\n\n\n\n
The system is starting to show signs of *autonomous behavior*. The shadow graph is promoting edges. The gravity map is revealing clusters we didn’t know existed. The circuit breaker is keeping the pipeline alive under GPU degradation.<\/p>\n\n\n\n
**<\/strong>The next phase is temporal identity threads<\/strong>**<\/strong> \u2014 stable entity identities that persist across IP churn, session turnover, and proxy cycling. If a threat actor rotates their infrastructure, their behavioral fingerprint remains. The system will know it’s the same actor even before any rule fires.<\/p>\n\n\n\n
After that: **<\/strong>edge confidence pressure<\/strong>**<\/strong> \u2014 edges won’t be binary valid\/invalid. They’ll have a confidence score that decays over time, builds with corroborating evidence, and promotes automatically when enough sensors agree.<\/p>\n\n\n\n
The end state isn’t a tool.<\/p>\n\n\n\n
It’s a living graph that notices things before humans know what to look for.<\/p>\n\n\n\n
—<\/strong><\/p>\n\n\n\n
##<\/strong> Acknowledgements<\/strong><\/p>\n\n\n\n
Built with:<\/p>\n\n\n\n
– [Android Tactical Assault Kit (ATAK CIV)](https:\/\/github.com\/deptofdefense\/AndroidTacticalAssaultKit-CIV) \u2014 DoD open-source situational awareness platform<\/p>\n\n\n\n
– [Ollama](https:\/\/ollama.com) \u2014 local LLM inference (Gemma 3, EmbeddingGemma)<\/p>\n\n\n\n
– [Cesium](https:\/\/cesium.com) \u2014 3D geospatial rendering<\/p>\n\n\n\n
– [DuckDB](https:\/\/duckdb.org) \u2014 in-process analytical SQL<\/p>\n\n\n\n
– [Flask](https:\/\/flask.palletsprojects.com) + [Socket.IO](https:\/\/socket.io) \u2014 API server and real-time streams<\/p>\n\n\n\n
– Python 3.12, Android API 36, Pixel 7 Pro (Tensor G2)<\/p>\n\n\n\n
—<\/strong><\/p>\n\n\n\n
*SCYTHE is a research and development project. All capabilities described operate on self-hosted infrastructure with locally-run models. No data leaves the local network.*<\/p>\n\n\n\n
—<\/strong><\/p>\n\n\n\n
**<\/strong>Tags:<\/strong>**<\/strong> `#cybersecurity` `#OSINT` `#SIGINT` `#hypergraph` `#threatintelligence` `#android` `#ATAK` `#machinelearning` `#Gemma` `#RF` `#realtime` `#Python` `#infosec` ‘#ClaudeAI’ ‘#ChatGPT’ ‘#GeminiAI’ ‘#GrokAI’<\/p>\n\n\n\n
<\/p>\n","protected":false},"excerpt":{"rendered":"
# Building SCYTHE: From Android Plugin to Living Intelligence Organism > *How we turned a tactical RF sensor tool into a self-healing, graph-cognitive threat intelligence engine \u2014 one late night at a time.* — ## The Origin: A Map, a Radio, and a Crazy Idea It started with a simple enough premise: take [ATAK (Android… SCYTHE Intelligence Organism<\/span><\/a><\/p>\n","protected":false},"author":1,"featured_media":4582,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"neve_meta_sidebar":"","neve_meta_container":"","neve_meta_enable_content_width":"","neve_meta_content_width":0,"neve_meta_title_alignment":"","neve_meta_author_avatar":"","neve_post_elements_order":"","neve_meta_disable_header":"","neve_meta_disable_footer":"","neve_meta_disable_title":"","footnotes":""},"categories":[10,7],"tags":[],"class_list":["post-5200","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-signal_scythe","category-the-truben-show"],"_links":{"self":[{"href":"https:\/\/172-234-197-23.ip.linodeusercontent.com\/index.php?rest_route=\/wp\/v2\/posts\/5200","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/172-234-197-23.ip.linodeusercontent.com\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/172-234-197-23.ip.linodeusercontent.com\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/172-234-197-23.ip.linodeusercontent.com\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/172-234-197-23.ip.linodeusercontent.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=5200"}],"version-history":[{"count":2,"href":"https:\/\/172-234-197-23.ip.linodeusercontent.com\/index.php?rest_route=\/wp\/v2\/posts\/5200\/revisions"}],"predecessor-version":[{"id":5203,"href":"https:\/\/172-234-197-23.ip.linodeusercontent.com\/index.php?rest_route=\/wp\/v2\/posts\/5200\/revisions\/5203"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/172-234-197-23.ip.linodeusercontent.com\/index.php?rest_route=\/wp\/v2\/media\/4582"}],"wp:attachment":[{"href":"https:\/\/172-234-197-23.ip.linodeusercontent.com\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=5200"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/172-234-197-23.ip.linodeusercontent.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=5200"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/172-234-197-23.ip.linodeusercontent.com\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=5200"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}