Methodology

We collect from 7 distinct data source categories, split by collection strategy:

Event-based (incremental — only new data since last run):

• GitHub — Star velocity, commit frequency, new repos across 20+ Solana orgs. Uses pushed:>DATE filters to only fetch repos active since last run.
• Twitter/X — KOL monitoring (Toly, Mert, Raj Gokal, etc.), keyword tracking via SocialData API. Uses since:DATE search filter.
• Reddit — r/solana and r/solanadev posts, filtered by creation date.
• RSS / News — Helius Blog, Solana Foundation, CoinDesk, The Block. Articles filtered by publication date.

Snapshot (point-in-time — each run captures current state):

• On-chain (Helius) — Network TPS, epoch info, transaction patterns. RPC methods return live measurements; daily snapshots track evolution.
• DeFi (DeFiLlama) — Chain TVL, protocol TVL changes, DEX volumes. API returns current state with built-in 1d/7d/30d change percentages.
• Market (CoinGecko) — SOL price, 30 ecosystem tokens, trending coins. Returns current prices with pre-computed change metrics.

Each data point becomes a raw signal with source attribution, timestamps, and extracted entities. First run looks back 14 days; subsequent runs only fetch new data. Rate limiters ensure we respect every API.

Each signal receives a composite score (0-100) from four weighted dimensions:

• Magnitude (25%) — How large is this signal? (TVL size, star count, volume)
• Velocity (30%) — How fast is it changing? (Weighted highest — we care about EMERGING trends)
• Novelty (25%) — How new or unexpected is this? (New repos, big price moves score higher)
• Confidence (20%) — How reliable is the data source?

Velocity is weighted highest because our goal is to detect trends before they're obvious. Signals are classified as extreme (≥75), strong (≥55), medium (≥35), or weak.

Signals are compared against an up to 90-day rolling baseline per source category using z-score analysis to detect statistical anomalies:

• Z > 3 — Extreme anomaly (very rare signal)
• Z > 2 — Strong anomaly (notable deviation)
• Z > 1 — Medium anomaly (mild deviation)

Historical metrics are stored per-source after each collection run, building a baseline over time. Early runs (before enough history exists) fall back to within-batch statistics — comparing each signal against others collected in the same run.

Entities are extracted from each signal and normalized to canonical forms so the same protocol is recognized across sources:

• “jito-foundation” (GitHub), “Jito” (DeFiLlama), “JTO” (CoinGecko) all map to jito
• “marinade-finance”, “mSOL”, “MNDE” all map to marinade
• Common suffixes (-foundation, -labs, -exchange, -protocol) are stripped automatically

We then measure source diversity (how many different source types mention this entity), temporal density (are mentions clustering in time), and average score. Entities from 2+ source categories with score >40 become narrative candidates.

High-scoring signals (composite >30) are grouped into proto-narratives using Jaccard similarity on their normalized entity/tag sets:

• Build entity co-occurrence matrix from normalized signal data
• Cluster signals with Jaccard similarity >0.12 (with a cross-source bonus of +0.08 to prioritize multi-source clusters)
• Merge overlapping clusters (similarity >0.25)
• Filter out overly generic entities (e.g. “solana”, “defi”) that would collapse everything into one cluster
• Rank by source diversity, then average score

This produces 8-15 proto-narratives — algorithmically-detected signal clusters ready for synthesis.

Only at this stage do we use an LLM (Gemini 2.5 Flash, with Groq/Llama 3.3 70B as fallback). This step runs as part of the Analysis job (on-demand, not daily), reading a full 14-day signal window. The entire analysis uses just 2 LLM calls:

• Call 1 — Narrative Synthesis: All proto-narratives are sent in a single prompt. The LLM names each narrative, writes a detailed explanation referencing the evidence, and assigns confidence scores.
• Call 2 — Batch Idea Generation: The top 5 narratives (by confidence) are sent in a single prompt. The LLM generates 1 highly specific product idea per narrative — with creative product names, detailed target users, specific Solana programs/SDKs to integrate, and differentiation against named competitors.

The LLM cannot invent narratives — it can only describe what the algorithm already found. If synthesis fails, we fall back to algorithmic descriptions. Ideas are required to name specific on-chain programs, CPIs, and SDKs — no generic “dashboards” allowed.