Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions csnode/src/node.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -4044,7 +4044,7 @@ void Node::getRoundTable(const uint8_t* data, const size_t size, const cs::Round
if (rPackage.roundTable().round == roundPackageCache_.back().roundTable().round && !stageThreeSent_) {
auto mask = roundPackageCache_.back().poolMetaInfo().realTrustedMask;
if (cs::TrustedMask::trustedSize(rPackage.poolMetaInfo().realTrustedMask) > cs::TrustedMask::trustedSize(mask)) {
csdebug() << "Current Roundpackage of " << rNum << " will be replaced by new one";
cslog() << "NODE> Current Roundpackage of " << rNum << " will be replaced by new one";
auto it = roundPackageCache_.end();
if (!roundPackageCache_.empty()) {
--it;
Expand Down Expand Up @@ -4081,7 +4081,7 @@ void Node::getRoundTable(const uint8_t* data, const size_t size, const cs::Round
&& getBlockChain().hasDeferredBlock(own.sequenceNumber);
const bool verifies = deferred && getBlockChain().packageBlockVerifies(rPackage);
if (moreTrusted) { // official rule
csdebug() << "Current Roundpackage of " << rNum << " will be replaced by new one";
cslog() << "NODE> Current Roundpackage of " << rNum << " will be replaced by new one";
roundPackageCache_.push_back(rPackage);
}
else if (verifies) {
Expand Down
4 changes: 3 additions & 1 deletion docs/CHANGES.md
Original file line number Diff line number Diff line change
Expand Up @@ -42,7 +42,9 @@ lines under the new release.

## Not released yet

Nothing yet.
| PR | Change |
|---|---|
| #178 | solver: when a confidant enters stage 1 again in the same round after a fuller round package replaced the first one, it sends the stage 1 it already signed again instead of signing a second one with a new timestamp; it signs a new one when the characteristic hash changed, the stored timestamp is not after the last block's, is more than 30 s old or cannot be read; a writer-side replacement and a replacement inside a bootstrap round still sign a second stage 1, as before; new info lines `stage1 resent unchanged` and `stage1 signed again ... reason`, and the package replacement is logged at info level (`docs/consensus-diagnostics.md`, section 5) |

## community.18 — 2026-11-09

Expand Down
65 changes: 65 additions & 0 deletions docs/consensus-diagnostics.md
Original file line number Diff line number Diff line change
Expand Up @@ -191,3 +191,68 @@ the larger group.
- `the signatures of the block are insufficient or incorrect: #<seq> hash <hex> prev <hex>`
(warning, was debug): a received block failed the signature check and is dropped; its full hash
and previous hash identify which block it was.

## 5. Stage 1 entered again in the same round: `stage1 resent unchanged` / `stage1 signed again`

A confidant can receive a second round package for the round it is in, with more real trusted
signatures than the first. It then replaces the first package and enters stage 1 again
(`SolverCore::nextRound(true)`):

```
info NODE> Current Roundpackage of 1205 will be replaced by new one
```

(this line was at debug level before). If this node already signed and sent its stage 1 for that
round and sub-round from the same seat, it now sends the same signed message again instead of
signing a second one with a new timestamp:

```
info Trusted-1: stage1 resent unchanged: R-1205.0 T3 ts 1793783441147, a new one would differ in candidates,packets
```

`a new one would differ in` lists the fields a new stage 1 would have changed (`candidates`: the
trusted candidates as a set, `packets`: the packet hashes); it is absent when nothing differs.
Both are this node's votes for the next round, and the stored message the other confidants
already hold stays valid. Receivers check the resent message like any stage 1. A receiver that
already holds it ignores it, because its timestamp is the same.

Two signed stage-1 messages from one seat with different timestamps can leave the confidants
holding different copies. They then compute different block timestamps, or see two
hash-signature pairs for that seat at stage 2. In both cases the seat, or the block, can fail the
stage-3 check.

The node signs a new stage 1, as before this change, when the stored one would be refused or
would make the seat look faulty:

```
info Trusted-1: stage1 signed again: R-1205.0 T3 ts 1793783441147 -> 1793783442010, content differs (hash), reason hash-changed
```

- `hash-changed`: the characteristic hash changed, because the new package changed the last
block. This hash must match the majority's.
- `not-after-last-block`: the stored timestamp is not after the last block's, so the receivers
would refuse it.
- `too-old`: the stored timestamp is more than `DefaultTimeStampRange` (30 s) old, so fresh
timestamps from the other confidants could fall outside this node's stage-2 window, which is
measured from its own timestamp.
- `unreadable`: the stored timestamp cannot be read as a number.

Two cases still sign a second stage 1 for the same round, as before this change, and log no
`stage1` line:

- a fuller package that reaches the writer of the round package (the `moreTrusted` branch of
`Node::getRoundTable` for a node that holds its own package): that path starts the round
again with `updateRound = false`, which clears the stored stage 1;
- a fuller package that replaces the first one inside a bootstrap round: the start of a bootstrap
round clears the stored stage 1.

`content` compares the old and the new stage 1 apart from the timestamp: `same`, or `differs`
with the fields (`hash`, `candidates`, `packets`). The stored stage 1 is kept only while the node
is in that round and sub-round on the same seat. A new round, a new sub-round and every bootstrap
round start without it.

The replacement itself also logs `fatal ConveyerBase> Meta round currently in conveyer`. This line
is older than this change and the official node logs it too, on every replacement of a round
package by one for the same round. The conveyer already holds a round table for that round, so it
keeps that table and does not store the second one. Despite its `fatal` level, the node keeps
running.
7 changes: 7 additions & 0 deletions solver/include/solver/solvercontext.hpp
Original file line number Diff line number Diff line change
Expand Up @@ -249,6 +249,13 @@ class SolverContext {

void add_stage1(cs::StageOne& stage, bool send);

// the stage 1 this node already signed and sent for the current round and sub-round from its
// current seat (nullptr if none): kept across a same-round round-package replacement
const cs::StageOne* sent_stage1() const;

// remembers the stage 1 just signed and sent, for sent_stage1()
void remember_sent_stage1(const cs::StageOne& stage);

void add_stage2(cs::StageTwo& stage, bool send);

void add_stage3(cs::StageThree& stage);
Expand Down
12 changes: 12 additions & 0 deletions solver/include/solver/solvercore.hpp
Original file line number Diff line number Diff line change
Expand Up @@ -378,6 +378,18 @@ class SolverCore {
cs::RoundNumber lastGrayUpdated_ = 0;
RoundPackage justCreatedRoundPackage_;
SentSignatures lastSentSignatures_;

// the stage 1 this node last signed and sent, with the round, sub-round and seat key it was
// signed for; unlike stageOneStorage it survives nextRound(true), the re-entry of the same
// round after a fuller round package replaced the first one; nextRound(false) and the start of
// a bootstrap round clear it, and SolverContext::sent_stage1() checks round, sub-round and seat
struct SentStageOne {
cs::RoundNumber round = 0;
uint8_t subRound = 0;
cs::PublicKey key{};
cs::StageOne stage;
};
std::optional<SentStageOne> sentStageOne_;
std::unique_ptr<IterValidator> pVal_;
std::string kLogPrefix_;
};
Expand Down
29 changes: 29 additions & 0 deletions solver/include/solver/stage.hpp
Original file line number Diff line number Diff line change
Expand Up @@ -160,6 +160,35 @@ struct StageHash {
cs::RoundNumber round;
};

// What a confidant does with its stage 1 when it enters stage 1 again in the same round and
// sub-round from the same seat (SolverCore::nextRound(true), after a fuller round package of the
// round replaced the first one) and finds the stage 1 it already signed and sent:
// - Resend: the stored signed message is sent again byte for byte, so every confidant holds the
// same stage 1 of this seat;
// - otherwise a new stage 1 is signed as before, because the stored one would be refused or
// would make this seat look faulty: HashChanged (the characteristic hash must match the
// majority's), NotAfterLastBlock (the stamp must be after the last block's), TooOld (the stored
// stamp is more than Consensus::DefaultTimeStampRange old, so fresh stamps of the others could
// fall outside this node's own stage-2 window, which is measured from its own stamp),
// Unreadable (the stored stamp does not parse as a number).
enum class StageOneResend : uint8_t {
Resend,
HashChanged,
NotAfterLastBlock,
TooOld,
Unreadable
};

// sent: the stored signed stage 1; freshHash: the characteristic hash computed now;
// lastBlockStamp: the last block's timestamp; now: the current time (ms)
StageOneResend stageOneResend(const StageOne& sent, const Hash& freshHash, uint64_t lastBlockStamp, uint64_t now);

const char* toString(StageOneResend decision);

// the content fields of two stage-1 messages of one seat that differ, timestamp and signature
// aside, comma separated (empty: the same content); trusted candidates compare as sets
std::string stageOneContentDiff(const StageOne& a, const StageOne& b);

} // namespace cs

#endif // STAGE_HPP
23 changes: 23 additions & 0 deletions solver/src/solvercontext.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -50,6 +50,29 @@ void SolverContext::add_stage1(cs::StageOne& stage, bool send) {
core.gotStageOne(stage);
}

const cs::StageOne* SolverContext::sent_stage1() const {
const auto& sent = core.sentStageOne_;
if (!sent.has_value()) {
return nullptr;
}
const cs::Conveyer& conveyer = cs::Conveyer::instance();
const uint8_t seat = own_conf_number();
if (sent->round != conveyer.currentRoundNumber() || sent->subRound != core.subRound() || sent->stage.sender != seat ||
!conveyer.isConfidantExists(seat) || conveyer.confidantByIndex(seat) != sent->key) {
return nullptr;
}
return &sent->stage;
}

void SolverContext::remember_sent_stage1(const cs::StageOne& stage) {
const cs::Conveyer& conveyer = cs::Conveyer::instance();
if (!conveyer.isConfidantExists(stage.sender)) {
core.sentStageOne_.reset();
return;
}
core.sentStageOne_ = SolverCore::SentStageOne{conveyer.currentRoundNumber(), core.subRound(), conveyer.confidantByIndex(stage.sender), stage};
}

void SolverContext::add_stage2(cs::StageTwo& stage, bool send) {
// core.stageTwoStorage.push_back(stage);

Expand Down
6 changes: 6 additions & 0 deletions solver/src/solverinterface.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -199,6 +199,12 @@ void SolverCore::nextRound(bool updateRound) {
if (!updateRound) {
recv_hash.clear();
}
// the own signed stage 1 is kept only across the re-entry of the same round after a fuller
// round package replaced the first one; a new round and every bootstrap round start without it
// (bootstrap tables start rounds with updateRound = true too)
if (!updateRound || pnode->isBootstrapRound()) {
sentStageOne_.reset();
}
deferredBlock_ = csdb::Pool{};
stageOneStorage.clear();
stageTwoStorage.clear();
Expand Down
65 changes: 65 additions & 0 deletions solver/src/stage.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,10 @@
#include <csnode/datastream.hpp>
#include <conveyer.hpp>
#include <cscrypto/cscrypto.hpp>

#include <algorithm>
#include <string>

namespace cs {
TrustedMask::TrustedMask() {}

Expand Down Expand Up @@ -239,4 +243,65 @@ namespace cs {
Stage::Stage() {
sender = cs::ConfidantConsts::InvalidConfidantIndex;
}

StageOneResend stageOneResend(const StageOne& sent, const Hash& freshHash, uint64_t lastBlockStamp, uint64_t now) {
if (sent.hash != freshHash) {
return StageOneResend::HashChanged;
}
uint64_t sentStamp = 0;
try {
sentStamp = std::stoull(sent.roundTimeStamp);
}
catch (...) {
return StageOneResend::Unreadable;
}
if (sentStamp <= lastBlockStamp) {
return StageOneResend::NotAfterLastBlock;
}
if (now > sentStamp && now - sentStamp > Consensus::DefaultTimeStampRange) {
return StageOneResend::TooOld;
}
return StageOneResend::Resend;
}

const char* toString(StageOneResend decision) {
switch (decision) {
case StageOneResend::Resend:
return "resend";
case StageOneResend::HashChanged:
return "hash-changed";
case StageOneResend::NotAfterLastBlock:
return "not-after-last-block";
case StageOneResend::TooOld:
return "too-old";
case StageOneResend::Unreadable:
return "unreadable";
}
return "?";
}

std::string stageOneContentDiff(const StageOne& a, const StageOne& b) {
std::string diff;
auto add = [&diff](const char* field) {
diff += diff.empty() ? field : std::string(",") + field;
};
if (a.sender != b.sender) {
add("sender");
}
if (a.hash != b.hash) {
add("hash");
}
auto sameSet = [](auto x, auto y) {
std::sort(x.begin(), x.end());
std::sort(y.begin(), y.end());
return x == y;
};
if (!sameSet(a.trustedCandidates, b.trustedCandidates)) {
add("candidates");
}
if (a.hashesCandidates != b.hashesCandidates) {
add("packets");
}
return diff;
}
}
48 changes: 48 additions & 0 deletions solver/src/states/trustedstage1state.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,7 @@

#include <csnode/datastream.hpp>
#include <cscrypto/cscrypto.hpp>
#include <limits>
#include <string>

namespace cs {
Expand Down Expand Up @@ -99,7 +100,54 @@ void TrustedStage1State::off(SolverContext& context) {
//}
csdebug() << name() << ": --> stage-1 [" << static_cast<int>(stage.sender) << "]";
if (min_time_expired && transactions_checked && enough_hashes) {
// a stage 1 already signed and sent for this round and sub-round from this seat: the round
// was entered again after a fuller round package replaced the first one (SolverCore::nextRound(true))
const cs::StageOne* sent = context.sent_stage1();
std::string sentStamp;
std::string differs;
cs::StageOneResend decision = cs::StageOneResend::Resend;
if (sent != nullptr) {
sentStamp = sent->roundTimeStamp;
differs = cs::stageOneContentDiff(*sent, stage);
// Do not sign a second stage 1 for the same round and sub-round: send the same signed
// bytes again, as gotStageOneRequest() does. A second signed message of the seat makes
// the confidants hold different stage-1 copies (a receiver keeps the later stamp only if
// it gets both): they choose different block timestamps, or see two hash-signature pairs
// of this seat at stage 2 and exclude it. Receivers verify the message as any other (the
// signature covers this round, sub-round and message hash); one that already holds it
// ignores it (same timestamp).
// What the stored message may lack is harmless: the trusted candidates and the packet
// hashes are this node's votes for the next round, and the ones that came in after the
// first stage 1 only miss one vote, as if the replacement had not happened; the others
// already hold and accept that message. A new stage 1 is signed as before when the
// stored one would be refused or make this seat look faulty: see cs::stageOneResend().
uint64_t lastStamp = 0;
try {
lastStamp = std::stoull(context.blockchain().getLastTimeStamp());
}
catch (...) {
lastStamp = std::numeric_limits<uint64_t>::max(); // unknown: sign a new one
}
decision = cs::stageOneResend(*sent, stage.hash, lastStamp, cs::Utils::currentTimestamp());
if (decision == cs::StageOneResend::Resend) {
stage = *sent;
cslog() << name() << ": stage1 resent unchanged: R-" << cs::Conveyer::instance().currentRoundNumber() << "."
<< static_cast<int>(context.subRound()) << " T" << static_cast<int>(stage.sender) << " ts " << sentStamp
<< (differs.empty() ? std::string{} : ", a new one would differ in " + differs);
context.add_stage1(stage, true);
return;
}
}
finalizeStage(context);
// stored before add_stage1(): that call already runs the state's onStage1() handlers
context.remember_sent_stage1(stage);
if (sent != nullptr) {
// the receivers may now hold two signed stage-1 messages of this seat
cslog() << name() << ": stage1 signed again: R-" << cs::Conveyer::instance().currentRoundNumber() << "."
<< static_cast<int>(context.subRound()) << " T" << static_cast<int>(stage.sender) << " ts " << sentStamp
<< " -> " << stage.roundTimeStamp << ", content " << (differs.empty() ? "same" : "differs (" + differs + ")")
<< ", reason " << cs::toString(decision);
}
context.add_stage1(stage, true);
}
}
Expand Down
Loading
Loading