Repository navigation
api: a token cache that follows every token standard, in any build - #128
Merged
Merged
Conversation
The token methods of the API (TokensListGet, TokenBalancesGet, TokenHoldersGet, ...) answer from a cache the node keeps next to the chain; it affects the API only, never consensus. - [api] tokens_cache=true turns it on in any build (RelMonitor/RelWallet: always on); off, the methods answer "This node doesn't provide such info" as before. - v1/v2 tokens: transfer/transferFrom with a BigDecimal amount are transfers (only string amounts were), standards 3 and 4 are known (register() of standard 4 was ignored). - Only successful calls, and only the contract called, update it: a failed call counted transfers and added holders, the state of a contract reached by invokeExternalContract was booked with the caller's method, and a deploy seen twice reset a token. - Holders: the initiator and every argument that is a wallet address (mint, airdrop, ...); balances are read from the contract for the wallets a call involves (every holder of a token with up to 50), 50 per executor request instead of all at once with a 100 ms budget. A getter that fails keeps the known name/symbol/supply; the holders count is recounted. - approve() without arguments read params[0] out of bounds (crash on monitor/wallet nodes). - Amounts in BigDecimal.toString() exponent form were read as 0; 256 zeros overflowed. - TokensListGet filters before paging (count and pages ignored the filters) with fixed filter logic; the lists sort with strict weak orderings and without std::stod exceptions. - TokenTransfersListGet: the signer is the sender (it was the token contract), and only calls to tokens are listed; chain reads happen outside the cache lock. - At start the balances are read again in the background, a token at a time under the lock; a checkpoint without tokens (saved with the cache off) is rebuilt from the deployed contracts and their successful calls.
1 of 2 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
The token methods of the Thrift API (
TokensListGet,TokenInfoGet,TokenBalancesGet,TokenHoldersGet,TokenTransfersGet,TokenWalletTransfersGet,TokenTransfersListGet,TokenTransferGet,TokenTransactionsGet) answer from a cache the node keeps next to the chain. It affects the API only, never consensus. Built on #126 and #127 (merge those first).Availability
[api] tokens_cache=trueturns the cache on in any build (off by default: the default behaviour does not change). RelMonitor/RelWallet builds keep it always on. Before, it existed only in those builds and every other node answered "This node doesn't provide such info".Tokens the cache missed or got wrong
BigDecimalamounts):transfer/transferFromwere transfers only with string amounts, so transfers were not counted, receivers never became holders and their balances were missing. Standards 3 and 4 are known (register()of standard 4 was ignored).invokeExternalContractwas booked with the caller's method; a deploy seen again reset a token.mint, airdrops, ...). Balances are always the contract's own (balanceOf), for the wallets a call involves (every holder of a token with up to 50), 50 per executor request. Before: all holders in one request with a 100 ms budget, so large tokens stopped updating.approve()without arguments readparams[0]out of bounds (crash risk on monitor/wallet nodes).BigDecimal.toString()exponent form (1E+3) were read as 0; 256 zeros overflowed auint8_t.Lists
TokensListGetfilters before paging (count and pages ignored the filters) and with a correct filter logic; the token and holder lists sort with strict weak orderings (desc ^ (a < b)is not one: UB instd::multiset) and withoutstd::stodexceptions.TokenTransfersListGet: the signer is the sender (it was the token contract), only calls to tokens are listed, chain reads happen outside the cache lock.Start
Docs:
docs/tokens.md(standards, argument types, the cache, the tests),differences.md,CHANGES.md, README index.Test plan
NODES=5 CONTRACTS=1 TOKENS=1 RESTART_NODES=5 RESTART_AT=100 TOKENS_VIA=5: 0 failures; through the restarted node: TokensListGet (standards 4 and 2, names, symbols), TokenBalancesGet (Alice 909.5 / 5.5, Bob 98.5 / 5, Carol 20), TokenInfoGet (4 holders, 3 transfers, supply 9999000), TokenTransfersGet (3), TokenHoldersGet; same states and balances on every nodetokens_cache=false(checkpoint at block 1020 with no tokens), then started withtrue:API: token cache is empty, looking for tokens among 6 contracts,2 tokens found, holders and balances equal to the contracts'🤖 Generated with Claude Code