Skip to content

grade each oracle feed against its own published promise, not a uniform bar - #2923

Merged
Flotapponnier merged 1 commit into
devfrom
fix/oracle-freshness-per-feed
Oct 9, 2026
Merged

Flotapponnier merged 1 commit into
devfrom
fix/oracle-freshness-per-feed

Conversation

@Flotapponnier

Copy link
Copy Markdown
Collaborator

You pushed back that a 7-hour Chainlink price could not be real and the bug was probably ours. I read the contracts directly: the number is real, description() confirms the right aggregators, AVAX/USD 7.6h and BNB/USD 8.3h. But the instinct behind the objection was right, and it found a real bug.

The bug

stale_but_moved claims to fire when a feed is late by its own standard. It did not. Both thresholds were uniform constants, and the comment justifying one was false: 0.5% was described as matching "the widest deviation trigger configured on the measured Chainlink feeds". Per Chainlink's own published directory the widest is 2%, four times that, and four of seven feeds run 24h heartbeats against a 300s bar.

Measured, not theoretical: AVAX/USD and BNB/USD both read stale_but_moved=1 while behaving exactly as configured. SOL/USD too, at 457s.

The real configuration

From reference-data-directory.vercel.app/feeds-mainnet.json (fields threshold, heartbeat), checked 2026-10-09, hardcoded so every value is auditable in the diff:

feeds threshold heartbeat
BTC, ETH, LINK 0.5% 3600s
BNB, MATIC 1% 86400s
SOL, AVAX 2% 86400s

Each feed is now graded against its own promise. The 300s becomes a grace period — the time an oracle legitimately needs to observe a move, aggregate and land a transaction — not a staleness bar. New ocb_oracle_heartbeat_breach covers the other half of the promise, published only for feeds that state a heartbeat so an absent series cannot be read as "never breached".

RedStone paused, and I was wrong about it

I said a URL change would restore it. It will not. Its per-symbol API now returns an empty array, and every documented replacement was probed: the /v2/.../ETH path is a 29-byte health stub, both dataFeedId(s) variants return the full payload, /v1/prices 404s. The only live source is an unfiltered 1.98 MB response taking 12.7 s — ~237 MB/hour at this cadence, to learn an age that reads 15-25 s whenever you look.

Dead 64 days, invisible because bench 082's page is retired so these metrics have no reader.

Nothing here is published today, so no reader was ever shown a false flag. It matters because I was about to propose publishing exactly this metric.

go build, go vet, pnpm validate clean.

…rm bar

stale_but_moved claimed to fire when a feed is "late by its own standard".
It did not. Both thresholds were uniform constants, and the comment
justifying one of them was false: 0.5% was described as matching "the widest
deviation trigger configured on the measured Chainlink feeds" when per
Chainlink's own published directory the widest is 2%, four times that.

Measured, not theoretical: AVAX/USD at 7.6h and BNB/USD at 8.3h both read
stale_but_moved=1 at the time of writing while behaving exactly as
configured, because a 2%/24h feed was being graded against a 0.5%/300s bar.
SOL/USD was flagged the same way at 457s.

The real configuration, from
reference-data-directory.vercel.app/feeds-mainnet.json (fields threshold and
heartbeat), checked 2026-10-09 and hardcoded in config.go so every value is
auditable in the diff:

  BTC, ETH, LINK      0.5%   3600s
  BNB, MATIC          1%    86400s
  SOL, AVAX           2%    86400s

Each feed is now judged against its own promise. stale_but_moved fires when
the CEX reference crosses THAT feed's trigger and it still has not written
after a 300s grace, which is the time an oracle legitimately needs to observe,
aggregate and land a transaction. The new ocb_oracle_heartbeat_breach covers
the other half of the promise, and is published only for feeds that state a
heartbeat so an absent series cannot be read as "never breached".

Also pauses the RedStone poller, dead 64 days and invisible because bench
082's page is retired. Not a paywall and not a URL fix: its per-symbol API now
returns an empty array, and every documented replacement was probed. The only
live source is an unfiltered 1.98 MB payload taking 12.7 s, so ~237 MB/hour at
this cadence to learn an age that reads 15-25 s whenever you look.

go build and go vet clean, pnpm validate clean.
@Flotapponnier
Flotapponnier merged commit ff983aa into dev Oct 9, 2026
2 checks passed
Flotapponnier added a commit that referenced this pull request Oct 9, 2026
…rm bar (#2923) (#2924)

stale_but_moved claimed to fire when a feed is "late by its own standard".
It did not. Both thresholds were uniform constants, and the comment
justifying one of them was false: 0.5% was described as matching "the widest
deviation trigger configured on the measured Chainlink feeds" when per
Chainlink's own published directory the widest is 2%, four times that.

Measured, not theoretical: AVAX/USD at 7.6h and BNB/USD at 8.3h both read
stale_but_moved=1 at the time of writing while behaving exactly as
configured, because a 2%/24h feed was being graded against a 0.5%/300s bar.
SOL/USD was flagged the same way at 457s.

The real configuration, from
reference-data-directory.vercel.app/feeds-mainnet.json (fields threshold and
heartbeat), checked 2026-10-09 and hardcoded in config.go so every value is
auditable in the diff:

  BTC, ETH, LINK      0.5%   3600s
  BNB, MATIC          1%    86400s
  SOL, AVAX           2%    86400s

Each feed is now judged against its own promise. stale_but_moved fires when
the CEX reference crosses THAT feed's trigger and it still has not written
after a 300s grace, which is the time an oracle legitimately needs to observe,
aggregate and land a transaction. The new ocb_oracle_heartbeat_breach covers
the other half of the promise, and is published only for feeds that state a
heartbeat so an absent series cannot be read as "never breached".

Also pauses the RedStone poller, dead 64 days and invisible because bench
082's page is retired. Not a paywall and not a URL fix: its per-symbol API now
returns an empty array, and every documented replacement was probed. The only
live source is an unfiltered 1.98 MB payload taking 12.7 s, so ~237 MB/hour at
this cadence to learn an age that reads 15-25 s whenever you look.

go build and go vet clean, pnpm validate clean.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant