Repository navigation
fix(gateway): download a split recording's first file, and say how many there are - #684
Merged
bburda merged 6 commits intoNov 12, 2026
Conversation
mfaferek93
reviewed
Nov 9, 2026
4 of 7 tasks
bburda
force-pushed
the
fix/split-recording-first-segment-and-file-count
branch
from
November 11, 2026 12:02
3fea8bb to
28c623d
Compare
bburda
marked this pull request as ready for review
November 11, 2026 13:21
…ny there are A recording split past snapshots.rosbag.max_bag_size_mb downloaded as whichever storage file the directory listing yielded first, and nothing in the descriptor said the recording had more. When a bag's metadata.yaml names storage files, the download now serves the first name and nothing else. If that file is missing, or the name is not a .db3 or .mcap directly inside the bag, the download answers 500. A later segment or a stray file is never served in its place. The directory scan is left only for a bag with no readable metadata or an empty file list. The rosbag descriptor's x-medkit block carries storage_files, the number of files the metadata names. It is omitted when the metadata does not say. The listing reads metadata.yaml once per row and takes the served file, the count and the size from that read.
5 of 7 tasks
bburda
force-pushed
the
fix/split-recording-first-segment-and-file-count
branch
from
November 11, 2026 16:45
28c623d to
06fb4b3
Compare
…ing it GraphNodeHasEndpoints asserted that the graph lists the node's endpoints right after the node was created. The rcl graph learns them asynchronously, through discovery of the node's own participant, so the check could run before they were listed and fail. The test now waits on the graph event, with a deadline, until the node's endpoints are listed.
…r source test_mutual_peer_faults identified each gateway's LIDAR_RANGE_INVALID by its first_occurred time and required the three to differ. Each fault manager stamps the fault with its own clock when the report arrives, and the three faulty lidars start and report together, so two gateways can hold the same time. Each gateway's lidar now runs in a namespace of its own, and the faults are identified by reporting_sources. create_demo_nodes takes an optional namespace that replaces the registry's for the nodes it launches.
On a status mismatch get_raw built its message from response.text. For a binary download of a few megabytes, requests then guesses the charset, which takes minutes, and the test hits its timeout with no message. The message now carries the first 1024 bytes of the body, decoded as UTF-8.
mfaferek93
previously approved these changes
Nov 11, 2026
test_rosbag_recording_manifest now checks on its real split recording that the recording download serves the first file the manifest lists, and that x-medkit.storage_files counts every file. With the first file moved aside and a later .mcap still in the bag, the download answers 500. The docs say that the download serves the first file of a split recording, and that a recording with a manifest is fetched file by file through the manifest and file routes.
bburda
force-pushed
the
fix/split-recording-first-segment-and-file-count
branch
from
November 12, 2026 07:09
e5fd9cd to
eb881eb
Compare
The domain wrapper tests start commands that write their ROS_DOMAIN_ID to a marker file, and the test reads the number as soon as the file exists. write_text and the shell's > create the file empty before they write, so the test could read an empty string and fail with ValueError. The commands now write to <marker>.part and rename it into place, so the marker never exists empty.
mfaferek93
approved these changes
Nov 12, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Pull Request
Summary
A recording split past
snapshots.rosbag.max_bag_size_mbdownloaded as whichever storage file the directory listing gave first, and the descriptor did not say that the recording had more files.When a bag's
metadata.yamlnames storage files,GET .../bulk-data/rosbags/{recording_id}now serves the first named file, where the recording starts. If that file is missing, is a symbolic link, or the name is not a.db3or.mcapdirectly inside the bag, the download answers500. It never serves a later segment or another file in its place. The directory scan stays only for a bag with no readable metadata or an empty file list.The descriptor's
x-medkitblock carriesstorage_files, the number of files the metadata names. It is omitted when the metadata does not say. When it is more than 1 and the descriptor carries"manifest": true, a client fetches the whole recording through the manifest and file routes from #785. The descriptorsizeis the sum of the storage files (#679), so for a split recording it is larger than the download'sContent-Length.docs/api/rest.rstand the snapshots tutorial describe the rule.The listing reads
metadata.yamlonce per row and takes the served file, the count and the size from that read.Three tests could fail at random, also on main. They are fixed here:
test_graph_node_listnow waits until the graph lists the probe node's endpoints. The rcl graph learns them asynchronously, so the check could run too early.test_mutual_peer_faultsnow tells the three peer gateways' faults apart byreporting_sources. It used the fault time, and two fault managers can stamp the same time. Each peer's lidar runs in its own namespace for this.test_domain_wrappercould read a command's domain file while it was still empty. The commands now write the number to a temporary file and rename it into place.Stacked on #679.
Issue
Type
Testing
test_rosbag_recording_manifestrecords a real split recording. It checks that the download hands over the manifest's first file, by sha256, and thatstorage_filesequals the number of files in the manifest. With the first file moved aside and a later.mcapstill in the bag, the download answers500.Checklist