diff --git a/benchmarks/perp-active-markets.yml b/benchmarks/perp-active-markets.yml index e08063973..1a535c324 100644 --- a/benchmarks/perp-active-markets.yml +++ b/benchmarks/perp-active-markets.yml @@ -30,9 +30,9 @@ abstract: | dYdX v4 indexer perpetualMarkets, Paradex markets summary, edgeX native markets, Aster fapi exchangeInfo PERPETUAL instruments, Nado gateway, GRVT markets, Extended public markets, Aevo markets, Pacifica Solana program state, Ostium - subgraph, Polymarket perps tickers). Gains uses the Mobula perp pairs - catalog (dex=gains, mainnet chains only, arbitrum-sepolia excluded), the same - source used by the perp-cohort-stats harness for all active_markets signals. + subgraph, Polymarket perps tickers). Gains reads its own Arbitrum trading + variables and leaves out the pairs it delisted, which stay in the array with + a 0.001x max leverage. Higher is better. methodology: @@ -51,7 +51,7 @@ methodology: - "Pacifica: onchain reads on the Solana program state, count of active perp markets." - "Ostium: Arbitrum subgraph, count of active pairs. Covers FX and commodities alongside crypto." - "Polymarket: public perps tickers endpoint on Polygon, count of listed instruments." - - "Gains: Mobula perp pairs catalog (GET /api/2/perp/pairs, dex=gains, mainnet only). Covers crypto, forex, stocks, indices and commodities across Arbitrum, Base, Polygon and ApeChain." + - "Gains: GET backend-arbitrum.gains.trade/trading-variables, every pair minus the ones whose max leverage override is 1 (0.001x), which is how Gains delists a pair without removing it. Covers crypto, forex, stocks, indices and commodities. Until 2026-10-12 this row read Mobula's perp pairs catalog, which listed 355 Gains pairs where 178 can be opened today, and which Mobula has since deprecated." - "Headline: avg_over_time of the live market count gauge over the last 24 hours. Listings change infrequently so the 24h average is effectively the current count." - "Failures: a venue that errors keeps its last gauge value and its perp_venue_health gauge drops toward 0; the leaderboard tags it as stale." @@ -81,8 +81,8 @@ prometheus: faq: - q: "What does this benchmark measure?" a: "The live count of active perpetual futures markets per venue. A market is a tradable instrument with a public price feed. Higher is better: more markets means more choice for traders." - - q: "Why does Gains use Mobula as a source instead of its native API?" - a: "The perp-cohort-stats harness uses the Mobula perp pairs catalog for gains because Mobula curates the full market taxonomy with asset-class tags. The count is mainnet-only (testnet chains excluded) and matches the inventory Gains exposes on its interface." + - q: "How is a delisted Gains pair told apart from a live one?" + a: "Gains never removes a pair from its trading variables; it delists one by setting the pair's max leverage override to 1, which is 0.001x at the contract's 1e3 precision (MATIC, LUNA, FTT, FB and a long tail of forex crosses and stocks carry it). The bench counts every other pair: an override of 0 means the group maximum applies, and a real override such as 2x or 5x on the stocks is still a market a trader can open." - q: "Is market count a good proxy for venue breadth?" a: "It is the simplest proxy. A venue with {{p50:gains}} markets covering crypto, forex, stocks and commodities (Gains) gives traders far more choice than one with {{p50:hyperliquid}} crypto-only markets (Hyperliquid). Depth per market is a separate signal; pair this bench with perp-volume-share and perp-fees for the full picture." - q: "Can a venue with more asset classes rank below one with more instruments?" @@ -117,7 +117,7 @@ providers: - slug: gains name: Gains tag: Synthetic perps on Arbitrum, Base, Polygon and ApeChain, multi-asset - formula: "Count of active mainnet perp pairs from Mobula's catalog (dex=gains, testnet excluded). Covers crypto, forex, stocks, indices and commodities." + formula: "Pairs in the Arbitrum trading variables whose max leverage override is not 1 (0.001x, the delisting marker). Covers crypto, forex, stocks, indices and commodities." queries: p50: avg_over_time(perp_venue_active_markets{venue="gains"}[24h]) p90: quantile_over_time(0.90, perp_venue_active_markets{venue="gains"}[24h]) diff --git a/benchmarks/perp-asset-breadth.yml b/benchmarks/perp-asset-breadth.yml index cac7d4b3f..82bd85934 100644 --- a/benchmarks/perp-asset-breadth.yml +++ b/benchmarks/perp-asset-breadth.yml @@ -44,12 +44,12 @@ abstract: | on that total. The class comes from the venue's own metadata when it exists: Extended's category and subCategory, Paradex's RWA tag, Aster's underlyingSubType (STOCK, ETF, Commodities, USD1-RWA), Ostium's pair - group, Ondo's tags, and the Mobula perp catalog assetClass for Gains and - Lighter. Hyperliquid's HIP-3 dexes (xyz, para, mkts, io and the others - returned by the perpDexs call) and dYdX publish no class, so their - symbols go through a fixed table of forex codes, commodity references - and index names; a symbol not in the table counts as a stock on a - HIP-3 dex and as crypto on dYdX, which carries an explicit list of its + group, Ondo's tags, and Gains' pair groups. Lighter, Hyperliquid's + HIP-3 dexes (xyz, para, mkts, io and the others returned by the + perpDexs call) and dYdX publish no class, so their symbols go through + a fixed table of forex codes, commodity references and index names; a + symbol not in the table counts as a stock on a HIP-3 dex and as crypto + on Lighter and on dYdX, which carries an explicit list of its non-crypto tickers (EUR, TRY, WTI, XAG, TSLAX). GMX v2 is read from its synthetic token list, where gold, silver, WTI, Brent, natural gas, SPY, QQQ and a SpaceX pre-IPO synthetic sit next to the crypto markets. @@ -67,14 +67,15 @@ methodology: - "Ostium: pair group from the Ormi subgraph (crypto, forex, stocks, etf, indices, commodities); etf is split per symbol (URA, XLE, TLT count as stocks). Pairs with a last trade price only." - "Ondo Perps: GET /v1/perps/contracts tags (Crypto, Stock, ETF, Commodity, Index, FX), enabled perpetual contracts only; ETF is split per symbol (SPY and QQQ count as indices, EWY, SOXL, URNM as stocks)." - "dYdX v4: GET /v4/perpetualMarkets, ACTIVE markets. The indexer publishes no asset class and its tickers collide with the table (SPX-USD is the SPX6900 token, GME-USD a token), so a reviewed list marks EUR-USD and TRY-USD as forex, WTI-USD and XAG-USD as commodities, TSLAX-USD (xStocks TSLA) as a stock; everything else is crypto." - - "Gains and Lighter: Mobula GET /api/2/perp/pairs assetClass (forex, stocks, indices, commodities count; crypto, degen and new are crypto). Mainnet chains only, arbitrum-sepolia excluded." + - "Gains: the pair groups of its Arbitrum trading variables (forex, forex-minor and forex-exotic are forex; stocks-1 to 3 are stocks; indices; commodities-1 and 2; the three crypto groups are crypto), delisted pairs (max leverage 0.001x) left out. Read from Mobula's perp catalog until Mobula deprecated it on 2026-10-12." + - "Lighter publishes no class, so its symbols take the class another venue tags them with, then the fixed tables; a symbol no venue tags counts as crypto, so a pre-IPO synthetic only Lighter lists is undercounted, never invented. Read from Mobula's perp catalog until 2026-10-12." - "GMX v2: GET arbitrum-api.gmxinfra.io/tokens, synthetic tokens matched against the table (GOLD, SILVER, WTIOIL, BRENTOIL, NATGAS are commodities, SPY and QQQ indices) plus a reviewed list for the equity synthetics the table cannot recognise (SPCX, SpaceX pre-IPO). MEGA (MegaETH) and SPX6900 stay crypto." - "A venue whose catalog call fails on a tick keeps its previous count; the per-source error counter records the failure. Counts are not sampled, so p50, p90 and p99 read the same live value." findings: - "{{best_name}} leads non-crypto perp breadth with {{best_p50}} live markets across stocks, indices, forex and commodities, {{worst_name}} lists {{worst_p50}}." - "Hyperliquid reaches {{p50:hyperliquid}} non-crypto markets through HIP-3 deployer dexes (xyz, para, mkts, io): US and Asian equities, ETFs, gold, silver, oil, copper, JPY and EUR, S&P 500 and Nasdaq 100 style indices. The core Hyperliquid universe is still crypto-only." - "Extended lists {{p50:extended}} and Aster {{p50:aster}} non-crypto markets, both dominated by US stock perps, with a handful of commodities, indices and forex pairs on the side." - - "Gains lists {{p50:gains}} non-crypto markets, the broadest forex catalog of the cohort (18 pairs) next to 68 stocks, 14 indices and 10 commodities. Ostium lists {{p50:ostium}} with forex, indices and commodities as its core catalog rather than an add-on." + - "Gains lists {{p50:gains}} non-crypto markets, stocks first, then forex, indices and metals. Ostium lists {{p50:ostium}} with forex, indices and commodities as its core catalog rather than an add-on." - "Ondo Perps lists {{p50:ondo}}, Lighter {{p50:lighter}}, Paradex {{p50:paradex}}. GMX v2 lists {{p50:gmx-v2}} (gold, silver, WTI, Brent, natural gas, SPY, QQQ, a SpaceX synthetic) and dYdX v4 {{p50:dydx}} live markets from its reviewed list (EUR, TRY, WTI, XAG, TSLAX, whichever are ACTIVE): the two older venues have added a handful of non-crypto markets, not a catalog." source: https://github.com/ChainBench/OpenChainBench/tree/main/harnesses/perp-cohort-stats prometheus: @@ -85,7 +86,7 @@ faq: - q: "What counts as a non-crypto market?" a: "A perpetual whose underlying is not a crypto token: a stock or ETF (AAPL, TSLA, NVDA, EWY), an equity or rates index or the ETF that tracks one (S&P 500, SPY, Nasdaq 100, QQQ, KOSPI 200, US Treasuries), a forex pair (EUR/USD, USD/JPY) or a commodity (gold, silver, platinum, copper, WTI, Brent, natural gas). Tokenized gold such as PAXG or XAUT is an onchain token and stays in the crypto class." - q: "Where does the asset class come from?" - a: "From the venue itself whenever it publishes one: Extended's category and subCategory, Paradex's RWA tag, Aster's underlyingSubType, Ostium's pair group, Ondo's tags, Mobula's assetClass for Gains and Lighter. Hyperliquid HIP-3 dexes and dYdX publish none, so their symbols are matched against a fixed table of forex codes, commodity references and index names, and dYdX carries a reviewed list of its five non-crypto tickers." + a: "From the venue itself whenever it publishes one: Extended's category and subCategory, Paradex's RWA tag, Aster's underlyingSubType, Ostium's pair group, Ondo's tags, Gains' pair groups. Lighter, Hyperliquid HIP-3 dexes and dYdX publish none, so their symbols are matched against a fixed table of forex codes, commodity references and index names, and dYdX carries a reviewed list of its five non-crypto tickers." - q: "Why does Hyperliquid rank high when its own perps are crypto-only?" a: "Because HIP-3 lets third-party deployers list perps on Hyperliquid's orderbook under a namespaced symbol (xyz:TSLA, mkts:US500, para:AVGO). Those markets settle on the same HyperBFT engine and are reachable from the same account, so the bench counts them as Hyperliquid markets. The core universe (BTC, ETH, SOL and the other core coins) contributes zero to the non-crypto count." - q: "Does a market count if it only trades during exchange hours?" @@ -134,8 +135,8 @@ providers: - slug: gains name: Gains - tag: Synthetic perps on Arbitrum, Base, Polygon and ApeChain, Mobula catalog - formula: "Count of mainnet Mobula catalog pairs for dex=gains where assetClass in {forex, stocks, indices, commodities}. Live." + tag: Synthetic perps on Arbitrum, Base, Polygon and ApeChain, native pair groups + formula: "Count of live Arbitrum pairs whose Gains group is forex, stocks, indices or commodities; delisted pairs (max leverage 0.001x) excluded. Live." queries: p50: perp_venue_noncore_markets_total{venue="gains"} p90: perp_venue_noncore_markets_total{venue="gains"} @@ -170,8 +171,8 @@ providers: - slug: lighter name: Lighter - tag: zk-rollup orderbook perps, Mobula catalog - formula: "Count of mainnet Mobula catalog pairs for dex=lighter where assetClass in {forex, stocks, indices, commodities}. Live." + tag: zk-rollup orderbook perps, cross-venue symbol classification + formula: "Count of active perp markets in orderBookDetails classified forex, stocks, indices or commodities: a symbol another venue tags as non-crypto takes that class, then the forex, commodity and index tables; any other symbol is crypto. Live." queries: p50: perp_venue_noncore_markets_total{venue="lighter"} p90: perp_venue_noncore_markets_total{venue="lighter"} diff --git a/benchmarks/perp-capital-efficiency.yml b/benchmarks/perp-capital-efficiency.yml index 44e12e025..3643b7200 100644 --- a/benchmarks/perp-capital-efficiency.yml +++ b/benchmarks/perp-capital-efficiency.yml @@ -42,7 +42,7 @@ abstract: | methodology: - "Cadence: every 5 minutes. Both active_markets and TVL are re-read on the same sweep cycle." - - "Active markets: native venue API per the perp-active-markets bench methodology. Gains: Mobula perp pairs catalog (mainnet only). Hyperliquid: info metaAndAssetCtxs. dYdX: indexer perpetualMarkets. Others: native endpoints." + - "Active markets: native venue API per the perp-active-markets bench methodology. Gains: its Arbitrum trading variables minus delisted pairs. Hyperliquid: info metaAndAssetCtxs. dYdX: indexer perpetualMarkets. Others: native endpoints." - "TVL: DefiLlama /protocol/{slug} currentChainTvls. Staking and pool2 keys excluded. Multi-chain TVL summed across all deployment chains." - "Gains TVL: GToken vault across Arbitrum, Base, Polygon and ApeChain. This is the shared liquidity pool that backs all synthetic pairs simultaneously." - "Hyperliquid TVL: total user deposits (USDC). Funds serve as both margin and implicit market-maker collateral; every listed pair draws from this pool for maker fills." diff --git a/harnesses/perp-cohort-stats/README.md b/harnesses/perp-cohort-stats/README.md index 6208c8483..b2203ff62 100644 --- a/harnesses/perp-cohort-stats/README.md +++ b/harnesses/perp-cohort-stats/README.md @@ -19,7 +19,7 @@ reads on every SSR render of the perp pages. | `perp_venue_volume_30d_usd{venue}` | DefiLlama HTML | (none) | | `perp_venue_oi_usd{venue}` | HL native / Lighter native | DefiLlama HTML | | `perp_venue_fees_30d_usd{venue}` | DefiLlama HTML | (none) | -| `perp_venue_active_markets{venue}` | HL native / Lighter native | DefiLlama HTML | +| `perp_venue_active_markets{venue}` | HL native / Lighter native / Gains trading variables (delisted pairs excluded) | DefiLlama HTML | | `perp_venue_top_market_volume_24h_usd{venue}` | HL native / Lighter native | (none) | | `perp_venue_health{venue}` | derived per tick | n/a | | `perp_venue_funding_24h_bps{venue, asset}` | Mobula funding-rate | (none) | diff --git a/harnesses/perp-cohort-stats/cmd/script/adapter.go b/harnesses/perp-cohort-stats/cmd/script/adapter.go index a153a496d..9fcf5de07 100644 --- a/harnesses/perp-cohort-stats/cmd/script/adapter.go +++ b/harnesses/perp-cohort-stats/cmd/script/adapter.go @@ -140,7 +140,6 @@ const ( srcHLXyzNative = "hl_xyz_native" srcCexCoinGecko = "cex_coingecko" srcDefillama = "defillama" - srcMobulaPairs = "mobula_pairs" srcMobulaFund = "mobula_funding" ) @@ -317,13 +316,13 @@ func priorityMap(venue, metric string) []string { return []string{srcHLNative} case "lighter": // Lighter native is the authoritative count: it filters - // market_type=="perp" + status=="active". Mobula pairs - // counts the whole catalog and diverges constantly here, - // so we keep it for `gains` (where Lighter does not - // publish) but drop it from the lighter priority list. + // market_type=="perp" + status=="active". return []string{srcLighterNative} case "gains": - return []string{srcMobulaPairs} + // The Arbitrum trading variables minus delisted pairs. Mobula's + // perp pairs catalog served this until Mobula deprecated it + // (404 since 2026-10-12). + return []string{srcGainsNative} case "gmx-v2": return []string{srcGMXNative, srcDefillama} case "dydx": @@ -438,6 +437,13 @@ func priorityMap(venue, metric string) []string { return nil } +// cryptoDefaultVenues are the venues whose unclassified symbols default +// to crypto: mainnet Lighter lists a few hundred tokens and a minority of +// stocks, forex and metals, the opposite of a HIP-3 RWA dex. A pre-IPO +// synthetic only Lighter lists therefore counts as crypto there, an +// undercount of its non-crypto catalog, never an overcount. +var cryptoDefaultVenues = map[string]bool{"lighter": true} + // Router orchestrates one sweep across all registered sources. // classifyUnclassified turns every SourceResult.Unclassified list into a // Breadth entry. Two sets accumulate across ticks (a source that failed @@ -445,8 +451,9 @@ func priorityMap(venue, metric string) []string { // lists as a non-crypto market and the symbols some venue lists as a // crypto market. On a HIP-3 dex the RWA reading wins, then the crypto // one (xyz:BOT is a token also listed on Lighter), then the fixed -// tables, and a symbol nobody knows counts as a stock. The Hyperliquid -// core universe arrives through the crypto map. +// tables, and a symbol nobody knows counts as a stock, except on a +// crypto-first venue (cryptoDefaultVenues) where it counts as crypto. +// The Hyperliquid core universe arrives through the crypto map. func (r *Router) classifyUnclassified(byName map[string]*SourceResult) { r.cryptoMu.Lock() defer r.cryptoMu.Unlock() @@ -479,7 +486,7 @@ func (r *Router) classifyUnclassified(byName map[string]*SourceResult) { // that reading wins over a same-named token elsewhere. class = rwaClass(sym) } else { - class = symbolClass(sym, true, r.knownCrypto) + class = symbolClass(sym, !cryptoDefaultVenues[venue], r.knownCrypto) } b.add(class) if class == classCrypto { @@ -562,7 +569,6 @@ func NewRouter(cfg *Config) *Router { if cfg.MobulaAPIKey != "" { sources = append(sources, NewMobulaFundingSource(cfg.MobulaAPIKey, cfg.MobulaFundingVenues), - NewMobulaPairsSource(cfg.MobulaAPIKey), ) } else { fmt.Println("[router] Mobula sources disabled (MOBULA_API_KEY unset)") diff --git a/harnesses/perp-cohort-stats/cmd/script/breadth.go b/harnesses/perp-cohort-stats/cmd/script/breadth.go index e71478c11..5c9ef7c55 100644 --- a/harnesses/perp-cohort-stats/cmd/script/breadth.go +++ b/harnesses/perp-cohort-stats/cmd/script/breadth.go @@ -13,8 +13,8 @@ import ( // perp_venue_noncore_markets_total{venue} (bench perp-asset-breadth). // // Class names match the Mobula perp catalog taxonomy the bench started -// with (Gains and Lighter still come from Mobula), so a Mobula row and a -// native row are comparable: +// with (Mobula deprecated that catalog in 2026-10; Gains now reads its +// own pair groups and Lighter goes through the symbol path below): // // crypto tokens and crypto indices (TOTAL2, BTC dominance) // forex fiat pairs (EUR/USD, USD/JPY) @@ -28,8 +28,8 @@ import ( // own metadata decides when it exists (Extended category, Paradex tags, // Aster underlyingSubType, Ostium group, Ondo tags); the symbol tables // below only split a venue-tagged RWA bucket into forex, indices and -// commodities, and cover the two venues with no metadata at all -// (Hyperliquid HIP-3 dexes, dYdX). +// commodities, and cover the venues with no metadata at all +// (Hyperliquid HIP-3 dexes, dYdX, Lighter). const ( classCrypto = "crypto" classForex = "forex" diff --git a/harnesses/perp-cohort-stats/cmd/script/config.go b/harnesses/perp-cohort-stats/cmd/script/config.go index 29d4c062e..251c23444 100644 --- a/harnesses/perp-cohort-stats/cmd/script/config.go +++ b/harnesses/perp-cohort-stats/cmd/script/config.go @@ -58,7 +58,7 @@ func loadConfig() *Config { } if c.MobulaAPIKey == "" { - fmt.Println("[config] WARN: MOBULA_API_KEY is unset; Mobula funding + Mobula pairs sources will be skipped this run.") + fmt.Println("[config] WARN: MOBULA_API_KEY is unset; the Mobula funding source will be skipped this run.") } fmt.Printf("Config: venues=%d, tick=%v, mobula_venues=%q, mobula_key=%s\n", diff --git a/harnesses/perp-cohort-stats/cmd/script/main.go b/harnesses/perp-cohort-stats/cmd/script/main.go index df42f1b41..d8affef19 100644 --- a/harnesses/perp-cohort-stats/cmd/script/main.go +++ b/harnesses/perp-cohort-stats/cmd/script/main.go @@ -42,7 +42,7 @@ import ( func main() { installLogCapture() fmt.Println("=== perp-cohort-stats harness ===") - fmt.Println("Multi-source per-venue perp cohort stats (HL native, Lighter native, DefiLlama, Mobula funding + pairs).") + fmt.Println("Multi-source per-venue perp cohort stats (HL native, Lighter native, DefiLlama, Mobula funding).") fmt.Println("Exposes /metrics, /health, /logs on :2112.") cfg := loadConfig() diff --git a/harnesses/perp-cohort-stats/cmd/script/metrics.go b/harnesses/perp-cohort-stats/cmd/script/metrics.go index 0c0f41b42..aa2acf3df 100644 --- a/harnesses/perp-cohort-stats/cmd/script/metrics.go +++ b/harnesses/perp-cohort-stats/cmd/script/metrics.go @@ -48,14 +48,14 @@ var ( perpVenueActiveMarkets = prometheus.NewGaugeVec( prometheus.GaugeOpts{ Name: "perp_venue_active_markets", - Help: "Number of perp markets actively trading, per venue. Source: HL native (len(ctxs)), Lighter native (len(orderBookDetails)), Mobula pairs (cross-check). For GMX, derived from DefiLlama protocol breakdown.", + Help: "Number of perp markets actively trading, per venue. Source: HL native (len(ctxs)), Lighter native (len(orderBookDetails)), Gains trading variables minus delisted pairs. For GMX, derived from DefiLlama protocol breakdown.", }, []string{"venue"}, ) perpVenueMarketsByClass = prometheus.NewGaugeVec( prometheus.GaugeOpts{ Name: "perp_venue_markets_by_class", - Help: "Number of active markets per venue per asset class (crypto/forex/stocks/indices/commodities). Source: each venue's native catalog and asset tags (breadth.go); Mobula perp pairs for Gains and Lighter.", + Help: "Number of active markets per venue per asset class (crypto/forex/stocks/indices/commodities). Source: each venue's native catalog and asset tags, or the cross-venue symbol classification for venues without tags (breadth.go).", }, []string{"venue", "class"}, ) diff --git a/harnesses/perp-cohort-stats/cmd/script/source_gains.go b/harnesses/perp-cohort-stats/cmd/script/source_gains.go index 2802ec11b..fcedb367b 100644 --- a/harnesses/perp-cohort-stats/cmd/script/source_gains.go +++ b/harnesses/perp-cohort-stats/cmd/script/source_gains.go @@ -5,6 +5,8 @@ import ( "fmt" "io" "net/http" + "strconv" + "strings" "time" ) @@ -36,15 +38,21 @@ import ( // Derived metrics: // // volume_24h_usd = today(auto+direct) + yesterday(auto+direct) * (24 - h) / 24 +// +// The market list and its asset classes come from the Arbitrum backend's +// trading variables (see gainsMarkets). Until 2026-10-12 they came from +// Mobula's perp pairs catalog, which Mobula deprecated (404 since then). type GainsNativeSource struct { client *http.Client base string + tvURL string } func NewGainsNativeSource() *GainsNativeSource { return &GainsNativeSource{ client: &http.Client{Timeout: 15 * time.Second}, base: "https://backend-global.gains.trade", + tvURL: "https://backend-arbitrum.gains.trade/trading-variables", } } @@ -89,6 +97,8 @@ func (s *GainsNativeSource) Fetch() (*SourceResult, error) { today := now.Format("2006-01-02") yesterday := now.AddDate(0, 0, -1).Format("2006-01-02") + s.markets(res, venue) + mixToday, err := s.volumeMix(today) if err != nil { perpCohortFetchErrors.WithLabelValues(venue, srcGainsNative, classifyError(err.Error())).Inc() @@ -110,6 +120,122 @@ func (s *GainsNativeSource) Fetch() (*SourceResult, error) { return res, nil } +type gainsTradingVariables struct { + Pairs []struct { + From string `json:"from"` + To string `json:"to"` + GroupIndex string `json:"groupIndex"` + } `json:"pairs"` + Groups []struct { + Name string `json:"name"` + } `json:"groups"` + PairInfos struct { + // Per-pair max leverage override, 1e3 precision; 0 means the + // group's maximum applies. + MaxLeverages []json.Number `json:"maxLeverages"` + } `json:"pairInfos"` +} + +// gainsGroupClass maps a Gains pair group to a breadth class. The groups +// are the venue's own taxonomy (crypto, altcoins, crypto-degen, forex, +// forex-minor, forex-exotic, stocks-1..3, indices, commodities-1..2); +// ok is false for a group this table has never seen. +func gainsGroupClass(group string) (string, bool) { + switch { + case group == "crypto", group == "altcoins", group == "crypto-degen": + return classCrypto, true + case strings.HasPrefix(group, "forex"): + return classForex, true + case strings.HasPrefix(group, "stocks"): + return classStocks, true + case group == "indices": + return classIndices, true + case strings.HasPrefix(group, "commodities"): + return classCommodities, true + } + return "", false +} + +// gainsMarket is one listed pair after the delisting filter. +type gainsMarket struct { + base string // the non-USD leg, upper-cased + class string +} + +// gainsMarkets returns the pairs a trader can open. Gains never removes a +// pair from the array; it delists one by setting its max leverage +// override to 1, which is 0.001x at the 1e3 precision (MATIC, LUNA, FTT, +// FB carry it). Everything else, a 0 override (group maximum) or a real +// one (2x, 5x on the stocks), is tradable. unknown lists the group names +// the class table does not cover; the caller must not publish a breadth +// split with a hole in it. +func gainsMarkets(tv gainsTradingVariables) (markets []gainsMarket, unknown []string) { + seen := map[string]bool{} + for i, p := range tv.Pairs { + if i < len(tv.PairInfos.MaxLeverages) { + if lev, err := tv.PairInfos.MaxLeverages[i].Int64(); err == nil && lev > 0 && lev < 1000 { + continue + } + } + gi, err := strconv.Atoi(p.GroupIndex) + if err != nil || gi < 0 || gi >= len(tv.Groups) { + unknown = append(unknown, "group#"+p.GroupIndex) + continue + } + class, ok := gainsGroupClass(tv.Groups[gi].Name) + if !ok { + if !seen[tv.Groups[gi].Name] { + seen[tv.Groups[gi].Name] = true + unknown = append(unknown, tv.Groups[gi].Name) + } + continue + } + base := strings.ToUpper(p.From) + if base == "USD" && p.To != "" { + base = strings.ToUpper(p.To) // USD/JPY is a yen market + } + markets = append(markets, gainsMarket{base: base, class: class}) + } + return markets, unknown +} + +// markets publishes active_markets and the asset-class split from the +// trading variables, and seeds the cohort's crypto and RWA symbol sets +// that classify the HIP-3 dexes (breadth.go). +func (s *GainsNativeSource) markets(res *SourceResult, venue string) { + body, err := s.get(s.tvURL) + if err != nil { + perpCohortFetchErrors.WithLabelValues(venue, srcGainsNative, classifyError(err.Error())).Inc() + fmt.Printf("[perp-cohort][%s][%s] err trading-variables: %v\n", venue, srcGainsNative, err) + return + } + var tv gainsTradingVariables + if err := json.Unmarshal(body, &tv); err != nil { + perpCohortFetchErrors.WithLabelValues(venue, srcGainsNative, "parse").Inc() + fmt.Printf("[perp-cohort][%s][%s] err parse trading-variables: %v\n", venue, srcGainsNative, err) + return + } + markets, unknown := gainsMarkets(tv) + res.SetIfPositive(venue, mActiveMarkets, float64(len(markets)+len(unknown))) + b := breadthCounter{} + for _, m := range markets { + b.add(m.class) + if m.class == classCrypto { + res.AddCryptoSymbol(m.base) + } else { + res.AddRWASymbol(m.base) + } + } + if len(unknown) > 0 { + perpCohortFetchErrors.WithLabelValues(venue, srcGainsNative, "unknown_group").Inc() + fmt.Printf("[perp-cohort][%s][%s] breadth withheld, unknown groups: %v\n", venue, srcGainsNative, unknown) + } else { + res.SetBreadth(venue, b) + } + fmt.Printf("[perp-cohort][%s][%s] ok trading-variables: pairs=%d active=%d breadth: %s\n", + venue, srcGainsNative, len(tv.Pairs), len(markets), b) +} + func (s *GainsNativeSource) volumeMix(day string) (gainsVolumeMix, error) { var mix gainsVolumeMix body, err := s.get(fmt.Sprintf("%s/api/volume-mix?from=%s&to=%s", s.base, day, day)) diff --git a/harnesses/perp-cohort-stats/cmd/script/source_gains_test.go b/harnesses/perp-cohort-stats/cmd/script/source_gains_test.go index 01e09de9c..714001b8d 100644 --- a/harnesses/perp-cohort-stats/cmd/script/source_gains_test.go +++ b/harnesses/perp-cohort-stats/cmd/script/source_gains_test.go @@ -1,6 +1,7 @@ package main import ( + "encoding/json" "math" "testing" "time" @@ -46,3 +47,55 @@ func TestGainsVolume24hFromDays_Degenerate(t *testing.T) { t.Fatalf("negative: %v", v) } } + +func TestGainsMarkets_DelistedAndClasses(t *testing.T) { + var tv gainsTradingVariables + body := `{ + "groups": [{"name":"crypto"},{"name":"forex"},{"name":"stocks-2"},{"name":"indices"},{"name":"commodities-1"},{"name":"altcoins"},{"name":"forex-exotic"}], + "pairs": [ + {"from":"BTC","to":"USD","groupIndex":"0"}, + {"from":"EUR","to":"USD","groupIndex":"1"}, + {"from":"USD","to":"JPY","groupIndex":"1"}, + {"from":"AAPL","to":"USD","groupIndex":"2"}, + {"from":"SPY","to":"USD","groupIndex":"3"}, + {"from":"XAU","to":"USD","groupIndex":"4"}, + {"from":"MATIC","to":"USD","groupIndex":"5"}, + {"from":"USD","to":"ZAR","groupIndex":"6"} + ], + "pairInfos": {"maxLeverages": [0, 0, 0, 5000, 0, 0, 1, 1]} + }` + if err := json.Unmarshal([]byte(body), &tv); err != nil { + t.Fatal(err) + } + markets, unknown := gainsMarkets(tv) + if len(unknown) != 0 { + t.Fatalf("unknown groups: %v", unknown) + } + if len(markets) != 6 { + t.Fatalf("a max leverage of 1 (0.001x) is a delisted pair: got %d markets, want 6", len(markets)) + } + got := map[string]string{} + for _, m := range markets { + got[m.base] = m.class + } + want := map[string]string{"BTC": classCrypto, "EUR": classForex, "JPY": classForex, "AAPL": classStocks, "SPY": classIndices, "XAU": classCommodities} + for k, v := range want { + if got[k] != v { + t.Fatalf("%s: got %q want %q (all: %v)", k, got[k], v, got) + } + } +} + +func TestGainsMarkets_UnknownGroupWithholdsNothingSilently(t *testing.T) { + var tv gainsTradingVariables + body := `{"groups":[{"name":"crypto"},{"name":"prediction"}], + "pairs":[{"from":"BTC","to":"USD","groupIndex":"0"},{"from":"X","to":"USD","groupIndex":"1"}], + "pairInfos":{"maxLeverages":[0,0]}}` + if err := json.Unmarshal([]byte(body), &tv); err != nil { + t.Fatal(err) + } + markets, unknown := gainsMarkets(tv) + if len(markets) != 1 || len(unknown) != 1 || unknown[0] != "prediction" { + t.Fatalf("markets=%v unknown=%v", markets, unknown) + } +} diff --git a/harnesses/perp-cohort-stats/cmd/script/source_leverage_terms.go b/harnesses/perp-cohort-stats/cmd/script/source_leverage_terms.go index 2de1e0d03..30c0a3dbe 100644 --- a/harnesses/perp-cohort-stats/cmd/script/source_leverage_terms.go +++ b/harnesses/perp-cohort-stats/cmd/script/source_leverage_terms.go @@ -196,8 +196,14 @@ func publishLeverageTerms(variants []venueTerms) { for _, v := range variants { lev := b.lev if lev == 0 { - // "all" is the venue's own maximum, so each product is asked - // at its own ceiling and the best of them wins. + // "all" is the venue's own maximum: only the product that + // reaches the ceiling answers it, so the room sits next to the + // leverage the max leverage column prints. Gains' standard pair + // at 200x leaves more room than the degen pair at 500x, and + // letting it win printed 500x beside the 200x room. + if v.maxLev != ceiling { + continue + } lev = v.maxLev } room, lost, kept, ok := termsAt(v, lev) diff --git a/harnesses/perp-cohort-stats/cmd/script/source_leverage_terms_test.go b/harnesses/perp-cohort-stats/cmd/script/source_leverage_terms_test.go index 085b1f5f0..3f4737c04 100644 --- a/harnesses/perp-cohort-stats/cmd/script/source_leverage_terms_test.go +++ b/harnesses/perp-cohort-stats/cmd/script/source_leverage_terms_test.go @@ -6,6 +6,8 @@ import ( "math/big" "os" "testing" + + "github.com/prometheus/client_golang/prometheus/testutil" ) // Fixtures captured live on 2026-10-12. @@ -576,3 +578,20 @@ func TestLeverageTermsLive(t *testing.T) { t.Fatalf("full fetch: %v", err) } } + +// TestAllTabReadsTheCeilingProduct pins the "Each venue at its max" tab to the +// product that reaches the ceiling. The standard pair at its own 200x leaves +// 0.305% against the degen pair's 0.12% at 500x; letting the larger room win +// published 500x in the max leverage column beside the 200x room. +func TestAllTabReadsTheCeilingProduct(t *testing.T) { + std := gainsVenue(t, stdLiqParams, 200, 0.035) + std.minLev = 1.1 + degen := gainsVenue(t, degenLiqParams, 500, 0.02) + degen.minLev = 500 + + publishLeverageTerms([]venueTerms{std, degen}) + if got := testutil.ToFloat64(perpLeverageMax.WithLabelValues("gains", "BTC", "all")); got != 500 { + t.Fatalf("max leverage on the all tab = %v, want 500", got) + } + approx(t, "all tab room", testutil.ToFloat64(perpLiqPriceMovePct.WithLabelValues("gains", "BTC", "all")), 0.12, 1e-9) +} diff --git a/harnesses/perp-cohort-stats/cmd/script/source_lighter.go b/harnesses/perp-cohort-stats/cmd/script/source_lighter.go index b0c81a4a7..7e989f344 100644 --- a/harnesses/perp-cohort-stats/cmd/script/source_lighter.go +++ b/harnesses/perp-cohort-stats/cmd/script/source_lighter.go @@ -29,10 +29,6 @@ type LighterNativeSource struct { venue string base string name string - // classify hands the perp symbols to the router's cross-venue - // classification (the Robinhood deployment lists equities next to - // tokens and has no asset class); mainnet keeps Mobula's taxonomy. - classify bool } func NewLighterNativeSource() *LighterNativeSource { @@ -45,11 +41,10 @@ func NewLighterNativeSource() *LighterNativeSource { // order books and volume. func NewLighterDeploymentSource(venue, base, name string) *LighterNativeSource { return &LighterNativeSource{ - client: &http.Client{Timeout: 15 * time.Second}, - venue: venue, - base: base, - name: name, - classify: venue != "lighter", + client: &http.Client{Timeout: 15 * time.Second}, + venue: venue, + base: base, + name: name, } } @@ -135,9 +130,11 @@ func (s *LighterNativeSource) Fetch() (*SourceResult, error) { } res.SetIfPositive(venue, mOI, oiSum) res.SetIfPositive(venue, mActiveMarkets, float64(active)) - if s.classify { - res.SetUnclassified(venue, syms) - } + // Lighter publishes no asset class, so the perp symbols go to + // the router's cross-venue classification (breadth.go). Until + // 2026-10-12 mainnet used Mobula's perp pairs taxonomy, which + // Mobula deprecated. + res.SetUnclassified(venue, syms) fmt.Printf("[perp-cohort][%s][%s] ok orderBookDetails: active=%d oi=%.0f\n", venue, s.name, active, oiSum) } diff --git a/harnesses/perp-cohort-stats/cmd/script/source_mobula_pairs.go b/harnesses/perp-cohort-stats/cmd/script/source_mobula_pairs.go deleted file mode 100644 index 223610f75..000000000 --- a/harnesses/perp-cohort-stats/cmd/script/source_mobula_pairs.go +++ /dev/null @@ -1,141 +0,0 @@ -package main - -import ( - "encoding/json" - "fmt" - "io" - "net/http" - "strings" - "time" -) - -// MobulaPairsSource hits the Mobula perp pairs catalog: -// -// GET https://api.mobula.io/api/2/perp/pairs -// -// One call returns the full catalog across every dex Mobula tracks -// today (Lighter, Gains, plus Mobula-side test fixtures). We bucket -// by `dex` field and emit `active_markets` per venue. -// -// Why this source: Mobula already curates the markets list for these -// two venues with `assetClass` tags (crypto, forex, stocks, commodities, -// indices, degen, new). Hitting Mobula once beats hitting each venue -// native API and avoids duplicating the asset-class taxonomy harness -// side. -type MobulaPairsSource struct { - apiKey string - client *http.Client -} - -func NewMobulaPairsSource(apiKey string) *MobulaPairsSource { - return &MobulaPairsSource{ - apiKey: apiKey, - client: &http.Client{Timeout: 15 * time.Second}, - } -} - -func (s *MobulaPairsSource) Name() string { return srcMobulaPairs } - -type mobulaPair struct { - Name string `json:"name"` // "BTC/USD" - Dex string `json:"dex"` - Chain string `json:"chain"` - AssetClass string `json:"assetClass"` -} - -type mobulaPairsResponse struct { - Data []mobulaPair `json:"data"` -} - -func (s *MobulaPairsSource) Fetch() (*SourceResult, error) { - res := newSourceResult() - url := "https://api.mobula.io/api/2/perp/pairs" - req, err := http.NewRequest("GET", url, nil) - if err != nil { - return res, err - } - if s.apiKey != "" { - req.Header.Set("Authorization", s.apiKey) - } - req.Header.Set("User-Agent", "OpenChainBench-PerpCohort/1.0 contact@openchainbench.com") - - resp, err := s.client.Do(req) - if err != nil { - perpCohortFetchErrors.WithLabelValues("all", srcMobulaPairs, classifyError(err.Error())).Inc() - return res, err - } - defer resp.Body.Close() - - body, err := io.ReadAll(resp.Body) - if err != nil { - return res, err - } - if resp.StatusCode != 200 { - perpCohortFetchErrors.WithLabelValues("all", srcMobulaPairs, fmt.Sprintf("http_%d", resp.StatusCode)).Inc() - return res, fmt.Errorf("mobula pairs http %d", resp.StatusCode) - } - - var parsed mobulaPairsResponse - if err := json.Unmarshal(body, &parsed); err != nil { - perpCohortFetchErrors.WithLabelValues("all", srcMobulaPairs, "parse").Inc() - return res, err - } - - // Bucket by dex -> count. Mobula's dex tag uses lower-case venue - // slugs that match the OCB harness convention (gains, lighter, ...). - // We skip the testnet `arbitrum-sepolia` chain rows so the active - // markets gauge reflects mainnet inventory only. - counts := map[string]int{} - byClass := map[string]map[string]int{} // dex -> class -> count - for _, p := range parsed.Data { - if p.Chain == "arbitrum-sepolia" { - continue - } - counts[p.Dex]++ - if byClass[p.Dex] == nil { - byClass[p.Dex] = map[string]int{} - } - byClass[p.Dex][p.AssetClass]++ - // Only the pairs Mobula has classified as crypto feed the cohort - // known-crypto set used for the HIP-3 dexes (breadth.go). The - // `new` bucket is unclassified, not crypto: Lighter's recent - // stock, gold and JPY listings sit there and would turn the same - // symbols on Hyperliquid xyz into crypto. The base is the part - // before the slash ("BOT/USD" -> BOT). - if i := strings.Index(p.Name, "/"); i > 0 { - switch p.AssetClass { - case classCrypto: - res.AddCryptoSymbol(baseSymbol(p.Name[:i])) - case classForex, classStocks, classIndices, classCommodities: - res.AddRWASymbol(baseSymbol(p.Name[:i])) - } - } - } - for dex, n := range counts { - res.Set(dex, mActiveMarkets, float64(n)) - } - - // Mobula's degen (leverage variants of crypto pairs) and new - // (listed, not yet classified) buckets are crypto for the breadth - // gauges; the four non-crypto classes map one to one. See breadth.go. - for dex, classes := range byClass { - b := breadthCounter{} - for class, n := range classes { - switch class { - case classForex, classStocks, classIndices, classCommodities: - b[class] += n - default: - // crypto, degen (leverage variants), new (unclassified): - // none of them is a non-crypto market for the ranking. - b[classCrypto] += n - } - } - res.SetBreadth(dex, b) - } - - fmt.Printf("[perp-cohort][mobula_pairs] ok: %d dexes, counts=%v\n", len(counts), counts) - for dex, classes := range byClass { - fmt.Printf("[perp-cohort][mobula_pairs] %s by class: %v\n", dex, classes) - } - return res, nil -} diff --git a/harnesses/perp-cohort-stats/go.mod b/harnesses/perp-cohort-stats/go.mod index dacc5b7ac..eb2c5b38c 100644 --- a/harnesses/perp-cohort-stats/go.mod +++ b/harnesses/perp-cohort-stats/go.mod @@ -8,6 +8,7 @@ require ( github.com/beorn7/perks v1.0.1 // indirect github.com/cespare/xxhash/v2 v2.3.0 // indirect github.com/klauspost/compress v1.17.9 // indirect + github.com/kylelemons/godebug v1.1.0 // indirect github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822 // indirect github.com/prometheus/client_model v0.6.1 // indirect github.com/prometheus/common v0.55.0 // indirect