Skip to content

Keep the payload of a FIN segment and the write side open after the peer's FIN - #123

Open
Musyg wants to merge 4 commits into
narrowlink:mainfrom
Musyg:split/1-half-close
Open

Musyg wants to merge 4 commits into
narrowlink:mainfrom
Musyg:split/1-half-close

Conversation

@Musyg

@Musyg Musyg commented Nov 9, 2026

Copy link
Copy Markdown

This is #112 by @frontierlabs-admin, rebased on current main. The two commits keep their author. Fixes #111.

#112 no longer applies to main. The rebase adapts it to:

The behaviour is otherwise the one described in #112. Its 16 regression tests are included and pass, with the existing ones (72 unit tests). fmt and clippy are clean.

This is the first part of #118, split as you asked. The next ones (ACK validation, FIN retransmission and crossed closes, close after drop, Linux qualification) will follow one at a time.

frontierlabs-admin and others added 2 commits November 9, 2026 19:12
A peer that half-closes with data still queued, as Linux does on
shutdown(SHUT_WR), puts its FIN on the last data segment. In Established
such a segment was handled by its flags alone:

- ACK|FIN in sequence took the FIN without its payload, so the last bytes
  of the stream never reached the reader and the ACK number fell short of
  them.
- ACK|PSH|FIN matched no arm and was dropped without an ACK, every
  retransmission of it too.
- A FIN segment arriving ahead of a gap was dropped without an ACK.

RFC 9293 §3.10.7.4 processes the segment text before the FIN bit. The
payload is now received like that of any other data segment, PSH or not,
and the FIN is taken only once every byte before it has arrived. A FIN
that is not yet in sequence leaves its payload queued and is taken when
the peer retransmits it.

Refs narrowlink#111

Rebased onto main: the FIN arm matches ACK|FIN and ACK|PSH|FIN by name, as
TcpFlags (narrowlink#115) has no Not, and wakers use AtomicWaker (narrowlink#113).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01V3Bjh4dHNPkiUL8EJgTE8H
A FIN closes only the peer's direction (RFC 9293 §3.6). On a FIN in
Established with nothing in flight, the stack answered at once with its
own FIN and moved to LastAck, and in CloseWait any ACK with nothing in
flight did the same. A client that sends a request and then calls
shutdown(SHUT_WR) therefore had the reply cut off as soon as the data
already written was acknowledged. A reader in CloseWait also got Pending
rather than end of stream, and waited for a timer.

After the peer's FIN the stack now:

- acknowledges it, wakes the reader, which gets every byte sent before
  the FIN and then end of stream, and keeps the write side open;
- in CloseWait, acknowledges a retransmitted FIN, or segment carrying
  it, and answers duplicate ACKs with a fast retransmission, as in
  Established;
- sends its own FIN when the application shuts down, once everything
  written is acknowledged, or when it drops the stream. The FIN sent on
  shutdown is retransmitted by the LastAck timer, which now lives at
  module level so poll_shutdown can start it. A write after it fails
  with BrokenPipe.

close_wait_timeout remains the backstop for an application that never
shuts down, and it is now armed on every FIN rather than only when data
was in flight. It counts time without progress, so a reply that is
still being written and acknowledged is not cut after a fixed delay.
A writer still waiting for window when the connection ends is woken.

Refs narrowlink#111, related to narrowlink#89.

Rebased onto main: wakers use AtomicWaker (narrowlink#113). The reader registers
before it reads the state, so the extra wake after Pending is not needed.
A shutdown deferred in CloseWait waits on the shutdown signal from narrowlink#114,
which the main loop now also raises in CloseWait once nothing is in
flight.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01V3Bjh4dHNPkiUL8EJgTE8H
@chatgpt-codex-connector

Copy link
Copy Markdown
Codex Review Summary: ❌ Failed

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ❌ Failed 2026-11-09T19:08:32.697992Z
We couldn’t complete this review. Comment @codex review to try again.
4f1e7b3 PR opened
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@chatgpt-codex-connector

Copy link
Copy Markdown

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.
To continue using code reviews, add credits to your account and enable them for code reviews in your settings.

@SajjadPourali

SajjadPourali commented Nov 10, 2026 •

Copy link
Copy Markdown
Collaborator

Please check these tests

--- a/src/stream/tcp.rs
+++ b/src/stream/tcp.rs
@@ -1837,8 +1837,48 @@
             .expect("no end of stream after the peer's FIN")
             .unwrap();
         data
+    }
+
+    #[tokio::test(start_paused = true)]
+    async fn regression_delayed_response_after_peer_fin() {
+        let (mut stream, mut up_rx, local_seq) = established().await;
+        stream.stream_sender().send(peer_segment(1001, local_seq, ACK | FIN, u16::MAX, &[])).unwrap();
+        written(&mut up_rx).await;
+        pass(TcpConfig::default().close_wait_timeout + Duration::from_millis(100)).await;
+        let result = stream.write_all(b"delayed response").await;
+        assert!(result.is_ok(), "half-closed stream rejected delayed response: {result:?}");
+    }
+
+    #[tokio::test(start_paused = true)]
+    async fn regression_drop_retransmits_unacknowledged_response() {
+        let (mut stream, mut up_rx, local_seq) = established().await;
+        stream.stream_sender().send(peer_segment(1001, local_seq, ACK | FIN, u16::MAX, &[])).unwrap();
+        written(&mut up_rx).await;
+        stream.write_all(b"response").await.unwrap();
+        let original = up_rx.recv().await.unwrap();
+        assert_eq!(original.payload.as_deref(), Some(&b"response"[..]));
+        drop(stream);
+        written(&mut up_rx).await;
+        pass(RTO + Duration::from_millis(100)).await;
+        let packets = written(&mut up_rx).await;
+        assert!(packets.iter().any(|p| p.payload.as_deref() == Some(&b"response"[..])),
+            "unacknowledged response was not retransmitted after drop");
     }

+    #[tokio::test(start_paused = true)]
+    async fn regression_fin_retry_preserves_sequence_number() {
+        let (mut stream, mut up_rx, local_seq) = established().await;
+        stream.stream_sender().send(peer_segment(1001, local_seq, ACK | FIN, u16::MAX, &[])).unwrap();
+        written(&mut up_rx).await;
+        let mut cx = Context::from_waker(Waker::noop());
+        assert!(Pin::new(&mut stream).poll_shutdown(&mut cx).is_pending());
+        let original = fins_written(&mut up_rx).await;
+        assert_eq!(original.len(), 1);
+        pass(TcpConfig::default().last_ack_timeout).await;
+        let retry = fins_written(&mut up_rx).await;
+        assert_eq!(retry, original, "FIN retransmission must reuse the original sequence number");
+    }
+
     /// RFC 9293 §3.6: a FIN closes only the peer's direction. The stack acknowledges it and keeps
     /// sending, and its own FIN waits for the application to shut down, which takes it to LastAck.
     #[tokio::test(start_paused = true)]

Comment thread src/stream/tcp.rs Outdated
tokio::time::sleep(close_wait_timeout).await;
let tcb = tcb.lock().unwrap();
let now = progress(&tcb);
if now == last || tcb.get_state() != TcpState::CloseWait {

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could we keep the write side open until the application shuts it down, or make this timeout opt-in? A client can send a request and half-close, then wait more than five seconds for the server to prepare a response. In that case this timer sends our FIN before the application writes the response. Could we add a test for a delayed response

Comment thread src/stream/tcp.rs
{
let mut tcb = self.tcb.lock().unwrap();
if let Err(e) = send_fin_n_change_state_to_fin_wait1("[drop]", nt, &self.up_packet_sender, &mut tcb) {
let sent = if tcb.get_state() == TcpState::CloseWait {

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What should happen if response data is still in flight here? This sends FIN and then aborts the task that retransmits unacknowledged data. If a response segment was lost, the peer can be left waiting for the missing bytes. The abort behavior already exists on main, but this new path also sends FIN; could we cover this case or track it separately?

Comment thread src/stream/tcp.rs Outdated
return;
}
log::debug!("{nt} {state:?}: {hint} timer expired, resending ACK|FIN (retry {idx}/{last_ack_max_retries})");
_ = write_packet_to_device(&pkt_sdr, nt, &tcb, None, ACK | FIN, None, None);

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The initial FIN increments tcb.seq, so this retry appears to use the sequence number after the original FIN. Should the retry use the original FIN sequence instead? This logic is inherited from main, but the new shutdown path calls it. The retransmission test could assert that both FINs have the same sequence number.

Musyg and others added 2 commits November 10, 2026 04:34
Review on narrowlink#123:

- CloseWait sent our FIN once close_wait_timeout passed without progress,
  which still cut a response the application had not written yet. The
  timer is removed: after the peer's FIN, the write side stays open until
  the application shuts it down or drops the stream (RFC 9293 §3.6).
  TcpConfig::close_wait_timeout is kept but deprecated, as it no longer
  has an effect.
- task_last_ack resent our FIN at the sequence number after it, since the
  first FIN had already advanced seq. The retry now uses the FIN's own
  sequence number.

Adds the two regression tests from the review, and a test that our FIN
waits for shutdown however long the reply takes. The writer test now ends
the connection with a peer RST instead of the removed timer.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01V3Bjh4dHNPkiUL8EJgTE8H
Marking the field #[deprecated] is a minor change (cargo-semver-checks
struct_field_marked_deprecated), and the next release is 1.0.2. The field
keeps its doc comment saying it is no longer used.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01V3Bjh4dHNPkiUL8EJgTE8H
@Musyg

Musyg commented Nov 10, 2026

Copy link
Copy Markdown
Author

Thanks for the review and the tests. Pushed 7c804fd and 6c0a4e0:

  1. CloseWait no longer has a timer. After the peer's FIN, the write side stays open until the application shuts it down or drops the stream. close_wait_timeout stays in TcpConfig so the API does not change, documented as unused (a #[deprecated] would need a minor release). Your delayed-response test is included, plus one that waits 30 s before replying.
  2. task_last_ack now resends our FIN at its own sequence number. Your test is included; it fails before the change (102 instead of 101).
  3. Drop with data in flight: as you noted, this is inherited from main, so I kept it out of this PR. It is fixed in part 4 of the split (close after drop), where the session task outlives the stream until the data and the FIN are acknowledged. Your test is included there.

The CI run on 6c0a4e0 is waiting for your approval. cargo semver-checks passes locally.

@Musyg
Musyg requested a review from SajjadPourali November 10, 2026 18:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Client half-close: payload of a FIN segment is dropped, and our FIN is sent immediately (no half-close)

3 participants