SPEC // V1.0 // SYSTEM INITIALIZED

Trend-to-Trade Signal Feed

Absorb signals from across the internet — trends, news, social chatter — charge them with intelligence, and discharge matched predictions and trades when you need them.

07
Data Sources
02
Market Types
05
Background Jobs
10
Weeks to V1
01

Product Scope

Phase 01 — Link Drop MVP
PROTOTYPE // WEEK 01-02

Enter the terminal. Paste a link to an X post or news article. System scrapes content, analyzes with LLM, returns matched Polymarket predictions + HyperLiquid perps. No auth required.

Web Only No Auth Single API Call
🔋
Phase 02-04 — Full Discharge
PRODUCTION // WEEK 03-10

React Native app. Privy auth with social login. Calibrate the terminal to show markets that matter to you. Auto-generated trending feed with market overlays. Social data enrichment from X and Farcaster.

React Native Privy Auth Personalized Feed
02

System Architecture

📱 Client LayerReact Native (Expo) + Privy SDK
▼ REST / WS
⚡ API GatewayHono // /auth /feed /scrape /markets
🔐 AuthPrivy
📡 PollerBullMQ
🔍 ScraperJina / X API
🎯 MatcherLLM + Search
🗄️ PostgreSQLSupabase
⚡ RedisUpstash + BullMQ
🧠 LLMOpenAI / Claude
𝕏 Twitter
📰 NewsAPI
📊 Polymarket
📈 HyperLiquid
🔮 Farcaster

Tech Stack

LayerTechnologyRationale
Mobile ClientReact Native (Expo)Cross-platform, Privy RN SDK support
Web ClientNext.jsPrototype-first, SSR for SEO
API ServerNode.js + HonoFast, typed, edge-compatible
DatabasePostgreSQL (Supabase)Managed, real-time subscriptions
Cache / QueueRedis (Upstash) + BullMQServerless Redis, reliable job queues
LLMGPT-4o / 4o-miniBest cost/quality for classification
Image GenDALL-E 3 / FluxEditorial illustrations for feed cards
HostingVercel + Fly.ioVercel for web, Fly for API workers
03

Polymarket API

Prediction market data — completely free, no auth required. Three API surfaces.

Gamma API — Discovery & Search

gamma-api.polymarket.com · 4,000 req / 10s

EndpointPurposeKey Params
GET /public-search?q={query}Search markets by keywordq (search string)
GET /eventsList/filter eventslimit, offset, active, closed, order, tag
GET /marketsList/filter marketslimit, offset, active, closed, order, slug
GET /tagsList category tagsnone

⚠ DOUBLE-ENCODED FIELDS: outcomePrices, clobTokenIds, outcomes are JSON strings inside JSON. Parse with json.loads() in Python or JSON.parse() in JS.

CLOB API — Real-time Prices

clob.polymarket.com · 9,000 req / 10s

EndpointPurposeParams
GET /price?token_id={ID}&side=buyCurrent pricetoken_id, side
GET /midpoint?token_id={ID}Mid pricetoken_id
GET /book?token_id={ID}Full orderbooktoken_id
GET /prices-history?market={CID}Price historymarket, interval, fidelity

Data API — Trades & Open Interest

data-api.polymarket.com · 1,000 req / 10s

EndpointPurpose
GET /trades?market={CID}Recent trades
GET /oi?market={CID}Open interest
04

HyperLiquid API

Perpetual futures data — free, no auth for reads. Single POST endpoint, type field determines query.

POST https://api.hyperliquid.xyz/info // Content-Type: application/json // WebSocket: wss://api.hyperliquid.xyz/ws
TypeBodyReturns
meta{"type":"meta"}All perp assets: name, decimals, maxLeverage
metaAndAssetCtxs{"type":"metaAndAssetCtxs"}Meta + live prices, funding, OI, volume
allMids{"type":"allMids"}Current mid prices for all assets
l2Book{"type":"l2Book","coin":"BTC"}Orderbook (bids/asks)
candleSnapshot{"type":"candleSnapshot","coin":"BTC","interval":"1h",...}OHLCV candle data
fundingHistory{"type":"fundingHistory","coin":"BTC",...}Funding rate history
recentTrades{"type":"recentTrades","coin":"BTC"}Last trades for asset

Response: metaAndAssetCtxs

// Response tuple: [meta, assetContexts[]] [ { "universe": [ { "name": "BTC", "szDecimals": 5, "maxLeverage": 50 } ]}, [{ "funding": "0.00003", "openInterest": "1234.5", "markPx": "65440", "dayNtlVlm": "500000000" }] ]
05

X / Twitter API

𝕏
Official API
$100/MO BASIC

10K tweet reads/mo, search recent, single tweet fetch. Sufficient for link-drop and limited search.

⚠ TRENDING TOPICS = PRO TIER ($5,000/MO)

GET /2/tweets/search/recent GET /2/tweets/:id
🔶
Third-Party Alt
FOR PROTOTYPE

SocialData.tools: $49/mo — full Twitter-like API, no rate pain.
Apify scrapers: Pay-per-use — scraper actors for tweets and trends.

Bypass Trending Paywall

Link Drop: Tweet Extraction

// 01. Extract tweet ID from URL const match = url.match(/status\/(\d+)/) // 02. Fetch via X API with entity expansion GET /2/tweets/{tweetId}?expansions=author_id &tweet.fields=text,entities,context_annotations // 03. context_annotations → free topic/entity tags // Use tweet text + annotations for market matching
06

News & Article Scraping

News Aggregation APIs

APIFree TierEndpointBest For
NewsAPI.org100 req/day/v2/top-headlines?category=Headlines by category
GNews.io100 req/day/api/v4/top-headlinesBackup source
CryptoPanicFree w/ key/api/v1/posts/?filter=hotCrypto + sentiment
CoinGecko30 req/min/api/v3/search/trendingTrending coins

Article Content Extraction

Jina Reader
RECOMMENDED // FREE

https://r.jina.ai/{url} — returns clean markdown. Handles JS rendering, anti-bot. Zero config.

🔧
Firecrawl / Readability
PRODUCTION ALT

Self-host Mozilla Readability for speed. Firecrawl API (generous free tier) for complex pages.

08

Privy Authentication

Single SDK. Social + wallet auth. Web and React Native.

Supported Login Methods

Email OTP SMS Google Apple X / Twitter Discord Farcaster MetaMask WalletConnect Coinbase Wallet

Data Returned Post-Auth

interface PrivyUser { id: string; createdAt: Date; linkedAccounts: LinkedAccount[]; twitter?: { subject: string; // Twitter user ID username: string; // @handle name: string; // Display name profilePictureUrl: string; }; farcaster?: { fid: number; // Farcaster ID username: string; displayName: string; pfp: string; bio: string; }; }
Critical Limitation
SOCIAL GRAPH NOT INCLUDED

Privy returns identity (username, pfp) but NOT social graph (followers, interests, liked posts). Workaround:

Farcaster: Neynar API with FID → follower graph + activity

X: Bearer token → public timeline (limited on Basic)

09

Database Schema

7 tables. PostgreSQL. Users, content, and market data.

USERS

id · privy_id · display_name · avatar_url
x_username · x_user_id
farcaster_fid · farcaster_username
created_at · updated_at

USER_INTERESTS

id · user_id (FK)
category [crypto|politics|sports|ai|...]
weight (adjustable via engagement)

TRENDING_TOPICS

id · source · category · title · summary
image_url · source_url · source_data (JSONB)
keywords[] · entities[]
relevance_score · expires_at

CACHED_MARKETS

id · source [polymarket|hyperliquid]
external_id · market_type [prediction|perp]
title · category · current_price
volume_24h · open_interest · metadata (JSONB)

TOPIC_MARKET_MATCHES

id · topic_id (FK) · market_id (FK)
match_score (0-1)
match_reason (LLM explanation)

FEED_ITEMS

id · user_id (FK) · topic_id (FK)
position · seen · engaged
created_at

SCRAPED_LINKS

id · url (unique) · content_type [tweet|article] · title · body_text · author · keywords[] · entities[] · summary

10

API Endpoints

Auth & Users

MethodEndpointAuthPurpose
POST/api/auth/verifyVerify Privy JWT, upsert user
GET/api/users/meBearerCurrent user + interests
PUT/api/users/me/interestsBearerSet interest categories
POST/api/users/me/enrichBearerFetch social data, infer interests

Link Drop — Scrape & Match

MethodEndpointAuthPurpose
POST/api/scrapeOptionalScrape URL → text, keywords, entities
POST/api/matchOptionalKeywords/entities → matched markets
POST/api/analyze-linkOptionalCombined scrape + match

Feed

MethodEndpointAuthPurpose
GET/api/feed?page=&limit=&category=BearerPersonalized feed with markets
GET/api/feed/refreshBearerForce-refresh

Markets — Direct Query

MethodEndpointPurpose
GET/api/markets/polymarket/trendingTop predictions by volume
GET/api/markets/polymarket/search?q=Search predictions
GET/api/markets/hyperliquid/assetsAll perps with prices
GET/api/markets/hyperliquid/:coinDetailed perp data
11

Background Jobs

5 pipelines. BullMQ on Redis. Continuous charge cycle.

📡
Poll
*/15 * * * *
🧹
Dedup
fuzzy match
🏷️
Classify
gpt-4o-mini
✍️
Summarize
+ image gen
🎯
Match
markets

01. Trend Polling — Every 15 Min

Parallel fetch from all sources:

X API Search NewsAPI Reddit /hot Farcaster Trending CryptoPanic CoinGecko Google Trends

Dedup via fuzzy title match (Levenshtein < 0.3). Classify via GPT-4o-mini. Store in trending_topics.

02. Summarize + Image Gen

GPT-4o writes 2-3 sentence summary. Extracts keywords. DALL-E 3 generates editorial illustration per feed card.

03. Market Matching

Search Polymarket by keywords/entities. Map HyperLiquid assets via entity→ticker lookup. LLM scores relevance 0-1. Keep > 0.5.

04. Price Refresh — Every 5 Min

Batch update cached market prices. Polymarket via CLOB endpoint. HyperLiquid via single allMids call.

05. Feed Generation — Every 30 Min

Per-user: filter by interests, join market matches, rank, apply diversity rules.

12

LLM Strategy

TaskModelTokensFrequencyCost/Cycle
Topic categorizationgpt-4o-mini~200100/cycle$0.003
Summary generationgpt-4o~500100/cycle$0.250
Keyword extractiongpt-4o-mini~200100/cycle$0.003
Market scoringgpt-4o-mini~300500/cycle$0.015
Image promptsgpt-4o-mini~20050/cycle$0.001
Link analysisgpt-4o~1000on-demand$0.005/req
COST SUMMARY

$0.30/cycle · $7.20/day @ 4 cycles/hr · ~$10-15/day with caching

13

Matching Algorithm

Polymarket Matching Pipeline

01. Keyword Search

Top 5 keywords → GET /public-search?q={keyword}

02. Entity Search

Named entities (person, org, event) → Polymarket search

03. Deduplicate

Remove duplicates by conditionId

04. Filter

Active only. Volume > $1,000

05. LLM Score

GPT-4o-mini rates relevance 0-1 per topic-market pair

06. Return Top 5

Score > 0.5, sorted descending

HyperLiquid Matching

// Entity → Ticker lookup + LLM fallback "bitcoin" → BTC "ethereum" → ETH "solana" → SOL "nvidia" → NVIDIA "tesla" → TSLA "gold" → GOLD // "Bitcoin ETF" topic → match BTC perp // Unknown entity → ask LLM to extract ticker

Feed Ranking Formula

score = category_weight × 0.4 // user interest strength + relevance_score × 0.3 // trending intensity + freshness × 0.2 // decay over 24h + market_volume × 0.1 // tradeability signal
14

Implementation Phases

PHASE 01 // WEEK 01-02
Link Drop MVP
Next.js + Tailwind. Single page: paste URL → scrape → matched markets. No auth, no database. In-memory per request.
Jina Reader X oEmbed Polymarket HyperLiquid
PHASE 02 // WEEK 03-04
Backend + Polling Engine
PostgreSQL (Supabase), Redis + BullMQ. Trend polling from all sources. Background summarization + market matching. REST API for feed.
PHASE 03 // WEEK 05-06
Auth + Personalization
Privy auth (web). Interest calibration onboarding. Personalized feed. Social enrichment via Neynar. Engagement tracking.
PHASE 04 // WEEK 07-10
React Native Discharge
Expo project + Privy RN SDK. Native feed UI. Pull-to-refresh. Push notifications for high-signal matches. Deep linking.
15

Cost Projections

Prototype — ~$130/mo

X API Basic
$100
OpenAI
$30
Polymarket
$0
HyperLiquid
$0

Production @ 10K Users — ~$1,328/mo

OpenAI (LLM)
$450
NewsAPI
$449
X API
$100
Neynar
$99
SerpAPI
$75
DALL-E 3
$60
Infra
$95
16

Key Tradeoffs

LLM vs. Keyword Matching

Keyword search hits too many false positives. "Trump" matches dozens of markets. LLM understands a tariff tweet → tariff market, not primary market. Cost: ~$0.015/500 pairs.

Polling vs. Streaming

X streaming = Enterprise ($$$). Most sources lack streaming. 15-min poll is fresh enough. Exception: HL WebSocket for live prices.

Jina Reader vs. Self-Hosted

Zero infra. Handles JS/anti-bot. Free. Production: add Firecrawl backup, self-host Readability for speed.

Materialized vs. On-Demand Feed

Pre-compute per-user feeds. Lower latency, enables push notifications. Trade: up to 30 min stale.

17

End-to-End Example

Link Drop Flow

01. User Pastes Link

https://x.com/elonmusk/status/1234567890

02. Detect & Scrape

X post detected → fetch tweet → "Spoke with NVIDIA CEO about AI chip exports to China"

03. LLM Analysis

Keywords: nvidia, AI chips, China · Entities: NVIDIA, China, Musk · Sentiment: neutral

04. Polymarket Search

"US ban AI chip exports to China by 2025?" — 42% Yes ($2.1M vol)

05. HyperLiquid Match

NVIDIA-PERP: $145.20, +0.003% funding, $48M vol

06. Render Card

Feed card: tweet summary + prediction markets + perps

Sample Feed Card

🏦
BUSINESS // NEWSAPI // 12 MIN AGO
Federal Reserve Signals Rate Cut in September

The Fed indicated a likely 25bps rate cut at the September meeting, citing cooling inflation and labor market normalization.

Fed Cuts Rates in Sept?
88% YES · $4.2M vol
BTC Reaches $80K EOY?
34% YES · $1.8M vol
BTC-PERP
$67,400 · +0.01% funding
18

Future Enhancements

💰
One-Tap Trading

Trade matched markets via Privy embedded wallet

👥
Social Feed

Follow others. See what markets they watch.

🔔
Price Alerts

"Notify when this market moves 10%+"

🤖
AI Research Agent

Deep-dive any topic with citations

📊
Portfolio Tracking

Connect wallet. Show P&L tied to feed.

🎨
Creator Economy

Create and share market analysis cards