Skip to content

pysimlin: run the notebook journey on a kernel that answers - #1079

Merged
bpowers merged 1 commit into
mainfrom
sweep/journey-ipykernel
Oct 3, 2026
Merged

bpowers merged 1 commit into
mainfrom
sweep/journey-ipykernel

Conversation

@bpowers

@bpowers bpowers commented Oct 3, 2026

Copy link
Copy Markdown
Owner

Why

The JupyterLab journey kept failing after #1074, always the same way: a notebook's first cell was run and produced no output at all. It failed once on main and three times across open pull requests. #1074 fixed a real race in the harness (the kernel session start); this is a second cause, and it is in the kernel, not the harness.

ipykernel through 7.3 writes shell replies raw on the shell ROUTER socket that its ZMQStream reads. The raw send drains the socket's edge-triggered read edge, so a request arriving at the same moment is never dispatched and the kernel sits idle for good (ipython/ipykernel#1529, first released in 7.4.0). JupyterLab sends a burst of shell requests when it connects a notebook (history, debug, the widget manager's comm), which makes a notebook's first cell the likely victim: Lab sends the execute_request, the prompt shows [*], and nothing ever comes back.

What

  • The e2e extra requires ipykernel>=7.4 (the lock moves ipykernel 7.2.0 to 7.4.0, with jupyter-client and tornado at 7.4.0's own floors).
  • Running a cell is an action with a checked effect. runCell waits for a new execution count and otherwise fails at the run, saying which side never acted ([*]: Lab sent it and the kernel has not answered; no new count: Lab never ran it), with the kernel state and the server log's last lines. There is no retry: a wedged kernel answers nothing, so pressing again cannot help.
  • The setup waits out a partially written server runtime file instead of throwing on it (jupyter_server writes it in place; this failed the setup once in 121 runs).
  • The failure artifact was always empty: actions/upload-artifact@v4 skips hidden files, and the journey writes to e2e/.output. The upload includes hidden files, so the trace, screenshots and server log reach the artifact.

Evidence

  • A reproduced failure (four concurrent journeys sharing four CPUs): the websocket trace shows the execute_request sent on the notebook's own connection and never answered, and a stack dump of that kernel shows every thread idle in select().
  • A kernel-level reproducer with no browser (jupyter_client, shell requests pipelined eight deep): ipykernel 7.2.0 wedged in 10 of 10 trials; 7.4.0 in 0 of 10 trials of 3,000 requests each.
  • The journey under the same concurrent load: 3 failures in 121 runs before (two kernel wedges, one runtime-file race), 120 of 120 after.
  • Unit specs in kernel-session.spec.ts cover every arm of the new pure decisions (every truncation of a runtime file, the prompt forms, each before-and-after pair of a cell run, both report verdicts).
  • The pre-commit hook passed in full.

What this does not establish

  • The failure on main left no artifact (see above), so it is attributed to this wedge by its shape, not by a trace.
  • At a base rate of about 2%, 120 clean runs happen by chance about one time in eleven; the kernel reproducer carries the claim, not the loop.
  • Nothing here has run on GitHub's runners yet; this PR's own check is the first.
  • pysimlin users on ipykernel 7.0 to 7.3 with JupyterLab 4.4 or later can hit the same wedge in their own notebooks. This changes only the test extra, not pysimlin's runtime dependencies.

🤖 Generated with Claude Code

https://claude.ai/code/session_01LYR7N9WZwQuFnZFW8JDUgx

The journey's remaining flake was the kernel, not the page. ipykernel
through 7.3 sends shell replies raw on the shell ROUTER that its
ZMQStream reads; the send drains the socket's edge-triggered read edge,
so a request arriving at that moment is never dispatched and the kernel
sits idle for good (ipython/ipykernel#1529, first released in 7.4.0).
JupyterLab sends a burst of shell requests when it connects a notebook,
so the victim is a notebook's first cell: Lab sends the
execute_request, the prompt shows [*], and nothing comes back. A
websocket trace and a kernel stack dump of a reproduced failure show
exactly that. The e2e extra requires ipykernel>=7.4.

A cell run is an action with a checked effect: runCell waits for a new
execution count and otherwise fails at the run, saying whether Lab sent
the cell or never ran it, with the kernel state and the server log. The
setup waits out a server runtime file that is still being written, and
the failure artifact, which was always empty because upload-artifact
skips the hidden .output directory, includes hidden files.
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 3, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-10-03T08:15:11.635625Z e19cfc5 PR opened
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@claude

claude Bot commented Oct 3, 2026

Copy link
Copy Markdown

Review summary

No blocking issues found. The kernel-wedge diagnosis (ipython/ipykernel#1529) is well-supported by the kernel-level reproducer, the ipykernel>=7.4 pin is appropriate, and the runCell check turns a silent timeout into a diagnosable failure. The pure helpers (readServerInfo, parsePrompt, cellRan, unrunCellReport, tail) have thorough per-arm coverage in kernel-session.spec.ts, and the CI artifact fix (include-hidden-files: true) is a straightforward correction. The readServerInfo parses JSON twice (once to probe, once in parseServerInfo), but that is cheap for a small runtime file and keeps the two call sites' intents clearly separated.

Overall correctness: correct

@codecov

codecov Bot commented Oct 3, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 92.66%. Comparing base (afdc027) to head (e19cfc5).

Additional details and impacted files
@@           Coverage Diff           @@
##             main    #1079   +/-   ##
=======================================
  Coverage   92.66%   92.66%           
=======================================
  Files         308      308           
  Lines      188506   188506           
=======================================
+ Hits       174672   174673    +1     
+ Misses      13834    13833    -1     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@bpowers
bpowers merged commit 06e46ca into main Oct 3, 2026
17 checks passed
@bpowers
bpowers deleted the sweep/journey-ipykernel branch October 3, 2026 13:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant