Repository navigation
p2p: run the EnzinoBB/p2p fork (packet size limit, host data refresh, ordered shutdown) — P9 - #121
Merged
Merged
Conversation
… refresh, ordered shutdown) The p2p submodule now comes from EnzinoBB/p2p, branched from the commit the official node pins (8fb3c516): - a packet header announcing more than 256 MiB of data closes the connection before any allocation, with one error line a minute per address; - a peer's host data (version, snapshot capability) is refreshed from its own datagrams, and the full-discovery peer list shows that value; - a received datagram is passed with its own length; - ~Host stops the routing table (threads joined, UDP socket closed) after its io thread and clears the Network singleton's reference to it. ~Transport no longer clears that reference itself: ~Host does it once the io thread has ended; doing it before ~Host raced with Network::OnConnected. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… suite - run.sh HUGE_FRAME_AT: huge_frame.py opens connections to the p2p TCP port of every node of this build and sends packet headers announcing 2^40, 2^30 and 256 MiB + 1 bytes; each node must close them at once, log the limit, keep running with its memory flat and stay in consensus. Row net-huge-frame. - snapshot-p2p-wait is no longer manual, and the capability check now expects the late serving nodes to be seen as serving (peers refresh host data). - robustness/testnet/scale: the node under comparison (node_alt_ref, the official node) is built with the p2p commit it points to, since this repository now points to its fork. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…m ping answers - p2p at 09252ec: a registration may announce at most 64 bytes, later packets 256 MiB, read in 1 MiB chunks so memory follows the bytes received; host data refreshed only from answers to our pings; a zero k-bucket value does not hide the explorer's; RoutingTable::Stop waits for the routing-table notification threads (at most 5 s). - net-huge-frame: oversized headers before and after a registration (closed at once, both limit lines logged) and just-below-limit headers without data (64 bytes, 256 MiB - 1: held until the idle timeout), memory checked each step. - docs: differences.md and CHANGES.md follow. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Collaborator
Author
|
Updated with origin/master (a025182, CHANGES rows kept in PR order). CI: Linux run 41068972357 green; Windows/macOS run 41068972310 green; robustness run 41069022155 (net-huge-frame, snapshot-p2p-wait, snapshot-p2p-join) green; official mix run 41069036891 (net-huge-frame, node_alt_ref=official) green. |
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Plan step P9 of the p2p snapshot work: the node runs its own fork of the p2p library, EnzinoBB/p2p (branch
community/p9, commit09252ec, on top of8fb3c516, the commit the official node pins). Review: CreditsCommunity/p2p#1. This changes the network transport of every node: not to be merged without the user's review.p2p changes
kMaxPacketSize= 256 MiB of data drops the connection withkProtocolCorruptedbefore any allocation. OneLOG(ERROR)line at most once a minute per address, with counts.Packet::GetandUnserializer::Get(std::vector)get the same kind of bound. 256 MiB is far above any packet the node sends; the inbox already drops its second-priority list above 512 MB in total.user_datais updated; address and ports stay as recorded and are checked again under the lock. Withfull_net_discovery(the node's setting)GetKnownNodesshows the k-bucket value for nodes in the k-buckets, instead of the first value seen, which another node often relayed. Without this the official code kept the first value until the entry was evicted. A received UDP datagram is now passed with its own length, not the whole receive buffer.~Host: stopio_, join the io thread, thenRoutingTable::Stop()(idempotent: joins the pinger and explorer threads, then really closes the UDP socket;Close()was a no-op) andNetwork::Instance().SetRoutingTable(nullptr).~RoutingTablestops the threads before deleting the k-buckets.Review fixes (p2p
09252ec, node02a3dd24)blockPoolsCount, 100 by default, configurable).ping_sent_) refreshes an entry; unsolicited datagrams no longer do. A change is seen at the node's next ping, andsnapshot-p2p-waitstill passes.RoutingTable::Stopwaits, at most 5 s, for the detachedNotifyHostthreads (they call into the host being destroyed); none can start once the io, pinger and explorer threads have ended.user_datano longer replaces a non-zero explorer value.net-huge-framenow checks, with memory read via--pidat each step:Node side
.gitmodules: p2p fromhttps://github.com/EnzinoBB/p2p.git; the fork's own submodules (leveldb, miniupnp) keep their upstream URLs and commits.~Transportno longer callsSetRoutingTable(nullptr)(added in Windows and macOS nodes load Linux checkpoints and sync mainnet with the same digests #115):~Hostnow does it after its io thread has ended. Clearing it earlier, while the io thread could still read the pointer inNetwork::OnConnected, was a data race. Clearing twice would have been harmless (ashared_ptrreset); the call is removed so only one safe place remains.node_alt_refbuilds (robustness, testnet, scale): the node under comparison (the official node) is built with the p2p commit it points to (8fb3c516). This repository no longer points to the same p2p commit; the alt build checks that the p2p submodules (leveldb, miniupnp) match.net-huge-frame(HUGE_FRAME_AT,ci/testnet/huge_frame.py). See above for the cases; every node of this build (the odd ones in mixed runs) must:snapshot-p2p-waitis no longer manual. The capability check now also expects the late serving nodes to be seen as serving.docs/differences.md,docs/CHANGES.md.Test runs
After the review fixes (
02a3dd24):net-huge-frame,garbage-datagrams,net-corrupt,snapshot-p2p-wait,snapshot-p2p-join,snapshot-capability,chaos-mixed.net-huge-frame, n1..n7: over-limit headers closed at once (+0 MiB). The just-below-limit headers were held and then closed on the idle timeout: +4 MiB for 4 connections at 256 MiB - 1, +0 MiB at 64 bytes. Both limit lines were logged and the nodes kept matching digests.snapshot-p2p-wait: the joiner waited for n7; the check reported1 of 1 late serving nodes do since their restart.node_alt_ref=official: 40979079273. Green:net-huge-frame(odd nodes; same figures) andsnapshot-p2p-join.Before the review fixes (
7690ffab):net-lossandsnapshot-p2p-serve.Local, p2p alone with ASan/UBSan:
🤖 Generated with Claude Code