Repository navigation
Recording manifest and per-file download routes, keep the recording just stored - #785
Merged
Merged
Conversation
A recording larger than max_total_storage_mb was evicted by its own quota sweep right after it was written. A bag name that was already taken made the open fail, and the cleanup removed the earlier recording.
The fault manager writes manifest.json when it closes a recording. The gateway serves it and each storage file by index with Range, and tells a missing recording (404 with a vendor code) from no answer (503). The rosbags listing answers 503 when it cannot be read in full.
… Range cpp-httplib applied the Range to the JSON body and answered 200 with a slice. JSON document routes now ignore the header. Also clears three clang-tidy warnings in the bag path code and tags the new recording tests.
…llowed The manifest is written by hand in its documented key order, so the ROS-free core builds with nlohmann-json 3.7. A symbolic link in a recording directory is neither listed nor served. A declined lookup for an entity with no source is a plain 404: nothing proves the recording absent.
The example listed one file under the total of twelve, against the rule stated right below it. It now lists two files that add up.
Contributor
There was a problem hiding this comment.
Copilot was unable to review this pull request because the user who requested the review has insufficient remaining quota.
An organization owner can choose the repository organization as the billing source for reviews requested by members with a paid Copilot license. Learn about Copilot code review billing.
bburda
reviewed
Nov 10, 2026
The post-fault timer closed the recording, hashed every storage file and stored its rows on the only executor thread, so no service answered and the capture dropped messages meanwhile. A worker owned by the capture now writes the manifest, then the rows, and is drained by stop().
The recording is larger than max_total_storage_mb, so the real fault manager's sweep runs on it: the test checks it is kept and that the "alone exceeds" warning is logged. The fixed 4 s wait becomes a wait for the capture's subscription and for the publisher to have sent enough.
cpp-httplib applies the request's ranges to every error body right after the global error handler, so a resume with an expired token got an empty 416 instead of its 401. The handler now clears them for every 4xx and 5xx, pre-routing 401, 403 and 429 included.
binary_download resolves the Range against the file: a last byte past the end is clamped, a part starting past it is dropped, Content-Range is an absolute first-last, and only a request with nothing left gets a 416, with Content-Range: bytes */size. A provider that cannot seek ignores the Range.
The downloaded files have no metadata.yaml, so ROS 2 cannot open the directory. ros2 bag reindex writes it again, and the directory plays.
Changelogs are written on the release branch when the release is cut. The text moves to the PR description.
The store worker checked that a fault was not cleared, then inserted its row. A clear landing between the two deleted nothing and the row stayed. The check and the insert now hold one mutex with on_fault_cleared()'s delete. Hashing stays outside it.
The racer kept confirming B after its boundary bag opened, so a second recording of B started once that one closed. With the store off the executor it can hold buffered history, and the duration check read it. The racer now stops at the boundary bag, and B must have one recording.
bburda
approved these changes
Nov 11, 2026
This was referenced Nov 11, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Pull Request
Summary
A recording split past
snapshots.rosbag.max_bag_size_mbcan now be downloaded in full, one storage file at a time, with resume.manifest.jsonnext tometadata.yaml: the storage files inrelative_file_pathsorder, the size and sha256 of each, andtotal_bytes. It is written to a temporary file, synced and renamed, before the recording row is stored.GET .../bulk-data/rosbags/{recording_id}/manifestreturns the manifest as stored.GET .../bulk-data/rosbags/{recording_id}/files/{index}returns one storage file withRangesupport. A range that cannot be satisfied answers 416.x-medkit-recording-not-found. A fault manager that does not answer gives 503, so a client can tell "gone" from "try again".x-medkit.manifest: truewhen the manifest exists. The listing answers 503 when alist_rosbagscall fails, instead of an empty 200.max_total_storage_mb, a warning names it. A startup warning fires whenmax_bag_size_mbis larger thanmax_total_storage_mb.The existing whole-recording download route is unchanged.
max_bag_size_mbkeeps its default of 50.This touches the same files as #679, #684, #699 and #700 (
rosbag_capture.cpp,bulkdata_handlers.cpp,rest.rst,snapshots.rst), so whichever lands second needs a rebase.Changelog
Text for the CHANGELOGs when the release is cut.
ros2_medkit_fault_manager
manifest.jsonnext to its storage files. It lists the files in the ordermetadata.yamlnames them, each with its size and sha256, and their total size, so a recording split across several files pastsnapshots.rosbag.max_bag_size_mbcan be fetched and checked as a whole. The manifest is written once, through a temporary file and a rename, after the writer is closed and before the recording is listed, and it is never rewritten. A recording whose manifest could not be written is still listed, without one, and recordings made before this change get none. The storedsize_bytesof a recording includes the manifest.stop()drains the worker. A fault cleared while its recording is being stored takes the row with it.snapshots.rosbag.max_total_storage_mbis no longer deleted by its own quota sweep right after it is written. The sweep evicts oldest first and stopped only under the quota, so a recording that alone exceeded it, the newest row, went the same way as everything before it and the fault kept no black box at all. The recording just stored is now never evicted: every older recording may go, it stays, and a warning says the storage is over the limit. A warning is also logged at startup whenmax_bag_size_mbis larger thanmax_total_storage_mb.ros2_medkit_gateway
GET /{entity}/bulk-data/rosbags/{recording_id}/manifestreturns the recording'smanifest.jsonbyte for byte: its files in order, each with size and sha256, and their total size.GET /{entity}/bulk-data/rosbags/{recording_id}/files/{index}serves one of those files by its position in the manifest, withRangesupport. Both take a recording id only, with no fallback to a fault code, and authorize like the existing download. A descriptor in the rosbags listing carriesx-medkit.manifest: truewhen its recording has a manifest.404with vendor codex-medkit-recording-not-foundis sent only when the fault manager answered and holds no recording with that id.503is sent when the fault manager did not answer. Every other404(unknown entity, another entity's recording, no manifest, an index out of range, a file missing on disk) carries no such vendor code.Rangeas RFC 9110 says, on the file route and on the existing download route alike. A last byte past the end is clamped to the file, a part that starts past the end is dropped, and only a request with no part left answers416, withContent-Range: bytes */<size>.Content-Rangealways carries the absolute first and last byte, multipart parts included. A download that cannot seek ignores theRangeand answers200. cpp-httplib served a content provider without checking the range, so a first byte past the end produced a206with an underflowedContent-Length.Range. cpp-httplib applies the range to whatever body a response carries, so a client resuming a download got the error document as a slice of itself, or an empty416when its offset was past the end of that document, in place of the real status. This covers every4xxand5xx, including the401,403and429answered before routing. The same applied to a ranged response an aggregating gateway relays from a peer, which was ranged a second time.GET /{entity}/bulk-data/rosbagsanswers503when the fault manager cannot be read for any of the sources the entity covers. A failed read used to be skipped, so the listing answered200with those recordings missing, or with an empty collection when every read failed, and a client could not tell "no recordings" from "no answer". A200listing is now complete.Issue
Type
Testing
Built and tested in Docker on Jazzy (Release, 0 compiler warnings in the touched packages):
colcon testforros2_medkit_fault_manager(36/36),ros2_medkit_gateway(143/143),ros2_medkit_integration_tests(140/140).test_recording_manifest.cpp(13),test_rosbag_recording.cpp(36, lookup, Range, index, handlers, listing 503), 8 new cases intest_rosbag_capture.cpp, integration teststest_rosbag_recording_manifest.test.py(a real 16 MB split into 1 MB files) andtest_rosbag_recording_unavailable.test.py(no fault manager).--dry-run --Werroron the changed files, Sphinx-Wbuild clean.After review: the same three packages pass again (fault_manager 36/36, gateway 143/143).
test_rosbag_recording_manifest.test.pynow runs over a 1 MB quota and waits on events,test_range_on_refused_requests.test.pycovers the 401, 403 and 429 under a Range, andAClearWhileTheRowIsStoredTakesTheRowWithItandStoringARecordingDoesNotHoldTheExecutorcover the store worker. The tutorial's reindex step was run on a real 11-file download on Jazzy.Manual run with a real capture (
ros2 bag play --loopof an 11-topic bag,max_bag_size_mb: 50): recordings of 232 MB (5 files) to 498 MB (10 files). For each, the manifest from the route equals the file on disk, every file's download sha256 equals the manifest, every file fetched in two Range parts and joined matches, out-of-range requests give 416, and with the fault manager stopped the listing, manifest and file routes give 503.Checklist