Repository navigation
Conversation
A peer that half-closes with data still queued, as Linux does on shutdown(SHUT_WR), puts its FIN on the last data segment. In Established such a segment was handled by its flags alone: - ACK|FIN in sequence took the FIN without its payload, so the last bytes of the stream never reached the reader and the ACK number fell short of them. - ACK|PSH|FIN matched no arm and was dropped without an ACK, every retransmission of it too. - A FIN segment arriving ahead of a gap was dropped without an ACK. RFC 9293 §3.10.7.4 processes the segment text before the FIN bit. The payload is now received like that of any other data segment, PSH or not, and the FIN is taken only once every byte before it has arrived. A FIN that is not yet in sequence leaves its payload queued and is taken when the peer retransmits it. Refs narrowlink#111 Rebased onto main: the FIN arm matches ACK|FIN and ACK|PSH|FIN by name, as TcpFlags (narrowlink#115) has no Not, and wakers use AtomicWaker (narrowlink#113). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01V3Bjh4dHNPkiUL8EJgTE8H
A FIN closes only the peer's direction (RFC 9293 §3.6). On a FIN in Established with nothing in flight, the stack answered at once with its own FIN and moved to LastAck, and in CloseWait any ACK with nothing in flight did the same. A client that sends a request and then calls shutdown(SHUT_WR) therefore had the reply cut off as soon as the data already written was acknowledged. A reader in CloseWait also got Pending rather than end of stream, and waited for a timer. After the peer's FIN the stack now: - acknowledges it, wakes the reader, which gets every byte sent before the FIN and then end of stream, and keeps the write side open; - in CloseWait, acknowledges a retransmitted FIN, or segment carrying it, and answers duplicate ACKs with a fast retransmission, as in Established; - sends its own FIN when the application shuts down, once everything written is acknowledged, or when it drops the stream. The FIN sent on shutdown is retransmitted by the LastAck timer, which now lives at module level so poll_shutdown can start it. A write after it fails with BrokenPipe. close_wait_timeout remains the backstop for an application that never shuts down, and it is now armed on every FIN rather than only when data was in flight. It counts time without progress, so a reply that is still being written and acknowledged is not cut after a fixed delay. A writer still waiting for window when the connection ends is woken. Refs narrowlink#111, related to narrowlink#89. Rebased onto main: wakers use AtomicWaker (narrowlink#113). The reader registers before it reads the state, so the extra wake after Pending is not needed. A shutdown deferred in CloseWait waits on the shutdown signal from narrowlink#114, which the main loop now also raises in CloseWait once nothing is in flight. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01V3Bjh4dHNPkiUL8EJgTE8H
Codex Review Summary: ❌ FailedThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
|
You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard. |
|
Please check these tests --- a/src/stream/tcp.rs
+++ b/src/stream/tcp.rs
@@ -1837,8 +1837,48 @@
.expect("no end of stream after the peer's FIN")
.unwrap();
data
+ }
+
+ #[tokio::test(start_paused = true)]
+ async fn regression_delayed_response_after_peer_fin() {
+ let (mut stream, mut up_rx, local_seq) = established().await;
+ stream.stream_sender().send(peer_segment(1001, local_seq, ACK | FIN, u16::MAX, &[])).unwrap();
+ written(&mut up_rx).await;
+ pass(TcpConfig::default().close_wait_timeout + Duration::from_millis(100)).await;
+ let result = stream.write_all(b"delayed response").await;
+ assert!(result.is_ok(), "half-closed stream rejected delayed response: {result:?}");
+ }
+
+ #[tokio::test(start_paused = true)]
+ async fn regression_drop_retransmits_unacknowledged_response() {
+ let (mut stream, mut up_rx, local_seq) = established().await;
+ stream.stream_sender().send(peer_segment(1001, local_seq, ACK | FIN, u16::MAX, &[])).unwrap();
+ written(&mut up_rx).await;
+ stream.write_all(b"response").await.unwrap();
+ let original = up_rx.recv().await.unwrap();
+ assert_eq!(original.payload.as_deref(), Some(&b"response"[..]));
+ drop(stream);
+ written(&mut up_rx).await;
+ pass(RTO + Duration::from_millis(100)).await;
+ let packets = written(&mut up_rx).await;
+ assert!(packets.iter().any(|p| p.payload.as_deref() == Some(&b"response"[..])),
+ "unacknowledged response was not retransmitted after drop");
}
+ #[tokio::test(start_paused = true)]
+ async fn regression_fin_retry_preserves_sequence_number() {
+ let (mut stream, mut up_rx, local_seq) = established().await;
+ stream.stream_sender().send(peer_segment(1001, local_seq, ACK | FIN, u16::MAX, &[])).unwrap();
+ written(&mut up_rx).await;
+ let mut cx = Context::from_waker(Waker::noop());
+ assert!(Pin::new(&mut stream).poll_shutdown(&mut cx).is_pending());
+ let original = fins_written(&mut up_rx).await;
+ assert_eq!(original.len(), 1);
+ pass(TcpConfig::default().last_ack_timeout).await;
+ let retry = fins_written(&mut up_rx).await;
+ assert_eq!(retry, original, "FIN retransmission must reuse the original sequence number");
+ }
+
/// RFC 9293 §3.6: a FIN closes only the peer's direction. The stack acknowledges it and keeps
/// sending, and its own FIN waits for the application to shut down, which takes it to LastAck.
#[tokio::test(start_paused = true)] |
| tokio::time::sleep(close_wait_timeout).await; | ||
| let tcb = tcb.lock().unwrap(); | ||
| let now = progress(&tcb); | ||
| if now == last || tcb.get_state() != TcpState::CloseWait { |
There was a problem hiding this comment.
Could we keep the write side open until the application shuts it down, or make this timeout opt-in? A client can send a request and half-close, then wait more than five seconds for the server to prepare a response. In that case this timer sends our FIN before the application writes the response. Could we add a test for a delayed response
| { | ||
| let mut tcb = self.tcb.lock().unwrap(); | ||
| if let Err(e) = send_fin_n_change_state_to_fin_wait1("[drop]", nt, &self.up_packet_sender, &mut tcb) { | ||
| let sent = if tcb.get_state() == TcpState::CloseWait { |
There was a problem hiding this comment.
What should happen if response data is still in flight here? This sends FIN and then aborts the task that retransmits unacknowledged data. If a response segment was lost, the peer can be left waiting for the missing bytes. The abort behavior already exists on main, but this new path also sends FIN; could we cover this case or track it separately?
| return; | ||
| } | ||
| log::debug!("{nt} {state:?}: {hint} timer expired, resending ACK|FIN (retry {idx}/{last_ack_max_retries})"); | ||
| _ = write_packet_to_device(&pkt_sdr, nt, &tcb, None, ACK | FIN, None, None); |
There was a problem hiding this comment.
The initial FIN increments tcb.seq, so this retry appears to use the sequence number after the original FIN. Should the retry use the original FIN sequence instead? This logic is inherited from main, but the new shutdown path calls it. The retransmission test could assert that both FINs have the same sequence number.
Review on narrowlink#123: - CloseWait sent our FIN once close_wait_timeout passed without progress, which still cut a response the application had not written yet. The timer is removed: after the peer's FIN, the write side stays open until the application shuts it down or drops the stream (RFC 9293 §3.6). TcpConfig::close_wait_timeout is kept but deprecated, as it no longer has an effect. - task_last_ack resent our FIN at the sequence number after it, since the first FIN had already advanced seq. The retry now uses the FIN's own sequence number. Adds the two regression tests from the review, and a test that our FIN waits for shutdown however long the reply takes. The writer test now ends the connection with a peer RST instead of the removed timer. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01V3Bjh4dHNPkiUL8EJgTE8H
Marking the field #[deprecated] is a minor change (cargo-semver-checks struct_field_marked_deprecated), and the next release is 1.0.2. The field keeps its doc comment saying it is no longer used. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01V3Bjh4dHNPkiUL8EJgTE8H
|
Thanks for the review and the tests. Pushed 7c804fd and 6c0a4e0:
The CI run on 6c0a4e0 is waiting for your approval. |
This is #112 by @frontierlabs-admin, rebased on current main. The two commits keep their author. Fixes #111.
#112 no longer applies to main. The rebase adapts it to:
The behaviour is otherwise the one described in #112. Its 16 regression tests are included and pass, with the existing ones (72 unit tests). fmt and clippy are clean.
This is the first part of #118, split as you asked. The next ones (ACK validation, FIN retransmission and crossed closes, close after drop, Linux qualification) will follow one at a time.