Skip to content

Decode the real packet format (fixes the diagonal shear and the horizontal offset) - #10

Open
dccote wants to merge 16 commits into
masterfrom
packet-format-fix
Open

dccote wants to merge 16 commits into
masterfrom
packet-format-fix

Conversation

@dccote

@dccote dccote commented Oct 22, 2026 •

Copy link
Copy Markdown
Contributor

unpack_mosaic assumed a 16384-byte header with no pixels, then 5461 triplets per 16384-byte block. The camera actually sends 16384-byte packets, each starting with a 4-byte header 90 90 <seq16>:

  • packet 0: the header, a tag list (TSI, FCNT, PCKH, PCKL, IFMT, IOFF, ENDT, 60 bytes in all), then the first pixels of the image;
  • packets n >= 1: the header, then 16380 bytes of pixels.

Once the headers are stripped, the Mono12p pixels form one continuous stream: a triplet straddles packets 0 and 1, and the last byte of a packet is pixel data, not padding. A full frame is 460 packets (7 536 640 bytes).

The old reading produced two artefacts visible in the images: 2 extra pixels per packet sheared the image by 0.45 px/row (the "diagonal"), and dropping the pixels of packet 0 offset the whole image horizontally (a vertical seam). An intermediate version of this branch still dropped the last byte of each packet and restarted triplets per packet, which corrupted one pixel per packet (456 per frame, about 120 ADU off); it was found by comparing with an independent driver on a real raw frame, and the decode is now identical to theirs. The write-up, including the three decoders and how they were checked, is in protocol-analysis/docs/PACKET-FORMAT.md.

Also in this branch

  • Stale-data drain. After an abrupt stop (a killed process) the camera keeps ~190 KB of the interrupted frame in its pipe and every following capture starts with it, which looks like a decoding bug. Both drivers drain EP 0x83 once at initialisation, right after the first STOP (drain_image_endpoint, bounded). The stream also drains when it stops. Checked on the real camera.
  • Faster single-shot capture (KiraluxCameraDevice.doCaptureFrame). STOP costs ~110 ms and was sent twice per frame, plus HWTRIG_SET, NFRAMES_SET and the sensor size every time. The single-shot mode is now set up on the first frame only (and again after a change of roi, binning or frame_count, or a failed read); each later frame is ARM, START, read. Measured on the real camera: 410 ms to 113 ms per frame (2.4 to 8.7 frames/s). Kiralux in camera.py still has the old sequence.
  • Continuous acquisition (startStreaming() / stopStreaming() / rearm()). The camera is armed once (nframes 0) and frames arrive back to back; snap() then returns the next frame without any command. Reads are 4 MB chunks of whole packets, and the reader resynchronises on the packet 0 header. Measured on the real camera at full frame: 30 to 32 frames/s (34.8 for the independent driver). Exposure and gain are only read when the camera is armed, so changing them while streaming re-arms.
  • PolarFrame.raw_channels: the four analyser images (I0, I45, I90, I135) as uint16 strided views of the mosaic, no copy and no conversion; channels is built from them (float32).
  • Live previews. examples/preview.py --intensity (matplotlib, S0 only); examples/preview_mytk.py (minimal myTk app, S0 only); examples/preview_mytk_channels.py (myTk): I0, I45, I90, I135, plus the angle (AoLP, as a hue) and the degree (DoLP, gray) of linear polarization, shown at about 30 frames/s in streaming mode. A column of controls on the left: Start/Stop, a logarithmic exposure slider (0.1 to 1000 ms) and "Save as..." with N images (default 1). The controls are bound to app properties with myTk's bind_property_to_widget_value. The app keeps the last N full-size raw mosaics in a deque(maxlen=N); a save waits for N fresh frames, averages them in float32 and writes one file: a 16-bit TIFF (the 12-bit range stretched to the full 16 bits, 4095 ADU becomes 65535) or a float32 .npy in ADU. The 8-bit display is scaled to the 99.5th percentile; a uint16 overflow in that scaling made bright zones look dark and speckled, and is fixed. The apps join the grab thread before shutting the camera down, so closing the window is a clean stop. The orientation convention of AoLP has not been checked with a known polarizer.
  • Tests: tests/support.py (the fake camera now serves the real format, also in continuous mode), tests/test_frame.py and tests/test_camera_device_mock.py updated or extended for the real format, the drain, raw_channels, the capture sequence and streaming (213 pass without hardware). The hardware tests were not re-run on this branch.

This branch is stacked on #9, so its preview example appears in this diff until #9 merges. #11 (selectable stream, e.g. a single polarimetric image) is stacked on this one.

🤖 Generated with Claude Code

https://claude.ai/code/session_01W4vCghAnCueWcZijuwUhpx

dccote and others added 16 commits October 22, 2026 13:20
…composition

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W4vCghAnCueWcZijuwUhpx
…et, first pixels in packet 0

Each 16384-byte packet on EP 0x83 starts with 0x90 0x90 <seq> and ends with a
padding byte; packet 0 carries a tag list (ending in ENDT) followed by the first
pixels. Decoding 5461 triplets per block sheared the image by 0.45 px/row and
offset it horizontally. Adds PACKET-FORMAT.md (including the stale-data trap after
an abrupt stop), --intensity for the live preview, and updates the test packer.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W4vCghAnCueWcZijuwUhpx
The capture runs in a thread and hands each frame to the window through
schedule_on_main_thread; closing the window waits for the capture in flight so the
camera is left idle. Tested with the simulated camera only.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W4vCghAnCueWcZijuwUhpx
…t stop)

A killed capture leaves part of the frame in the camera's pipe and every following
frame comes out shifted. Both drivers now read the image endpoint until it times out
right after the first STOP. The fake port only serves data once a capture is started,
like the camera, and the drain is tested. Documents the saturation pitfall too.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W4vCghAnCueWcZijuwUhpx
PolarFrame.raw_channels returns the four analyser images as strided uint16 views of
the mosaic (no copy, no float); channels now converts those to float32. The new
example keeps everything uint16 until the 12->8 bit shift needed to draw, and drops
frames the window cannot keep up with instead of queueing them.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W4vCghAnCueWcZijuwUhpx
STOP costs ~110 ms and was sent twice per frame, plus HWTRIG_SET, NFRAMES_SET and the
sensor size every time. Now they are sent on the first frame (and again after a change
of roi, binning or frame count, or a failed read); each later frame is ARM, START, read.
Shutdown stops a camera left armed. Measured on the real camera: 410 ms -> 113 ms per
frame (2.4 -> 8.7 frames/s).

The channels preview also scales its 8-bit display to the 99.5th percentile.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W4vCghAnCueWcZijuwUhpx
Packets carry a 4-byte header (90 90 <seq16>) and 16380 bytes of pixels; once headers
are stripped (60 bytes on packet 0) the Mono12p stream is continuous and a triplet may
straddle two packets. The previous decoder dropped the last byte of each packet as
"padding" and restarted triplets per packet, which corrupted one pixel per packet (456
per frame, ~120 ADU off). Found by comparing with an independent driver on a real raw
frame; the new decode is identical to theirs. Tests and PACKET-FORMAT.md updated.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W4vCghAnCueWcZijuwUhpx
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W4vCghAnCueWcZijuwUhpx
… back

nframes 0, ARM, START once; frames are then read in 4 MB chunks and cut on the packet 0
header (resynchronising on the 8-byte frame start). Exposure and gain re-arm a running
stream (the camera only reads them at ARM). snap() returns the next frame of the stream.
Technique from an independent driver (which reports ~30 frames/s); tested here with the
fake camera only, not yet on hardware. The channels preview uses it.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W4vCghAnCueWcZijuwUhpx
Pixels above 257 wrapped around (value * 255 in uint16), so bright areas showed dark
speckle and the image looked wrong at high intensity.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W4vCghAnCueWcZijuwUhpx
Computed in float32 on the reduced channel views. AoLP is not drawn where there is almost no light.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W4vCghAnCueWcZijuwUhpx
The window only sets what it wants; the grab thread is the only one to talk to the camera
(start/stop, exposure, collecting the next N frames of the stream). Saves full-size raw
uint16 mosaics as 16-bit TIFF/PNG or .npy, numbered when N > 1. Changing the exposure
re-arms the stream and now empties the pipe first.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W4vCghAnCueWcZijuwUhpx
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W4vCghAnCueWcZijuwUhpx
The myTk Slider only notifies its delegate of programmatic changes; trace its Tk variable instead.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W4vCghAnCueWcZijuwUhpx
…e saving

The exposure slider, its label, the button text, the number of images and the status are
properties of the app bound to their widgets (bind_property_to_widget_value), so the mouse
and the program both reach the same observer. Save as... keeps the next N frames in a deque,
averages them in float32 and writes one 32-bit TIFF or .npy.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W4vCghAnCueWcZijuwUhpx
… 16-bit TIFF

The deque of raw mosaics is now a rolling buffer of the last N frames at all times,
instead of one created when Save as... is pressed. A save waits for N fresh frames,
averages the deque and writes the result; a failed write is reported and cancels the
request. The TIFF is rounded to uint16 after the 12-bit range is stretched to the full
16 bits (4095 ADU -> 65535); the .npy stays float32 in ADU.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W4vCghAnCueWcZijuwUhpx
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant