Skip to content

fix(redis): preserve pending RDB writes on graceful shutdown (0.9.0) - #158

Merged
sciabarracom merged 1 commit into
apache:0.9.0from
miki3421:fix/redis-graceful-shutdown-0.9.0
Nov 4, 2026
Merged

sciabarracom merged 1 commit into
apache:0.9.0from
miki3421:fix/redis-graceful-shutdown-0.9.0

Conversation

@miki3421

@miki3421 miki3421 commented Nov 1, 2026

Copy link
Copy Markdown
Contributor

Redis currently runs as a child of a shell that remains process 1. During an ordinary Kubernetes pod deletion, Redis does not receive the stop signal and cannot complete its final RDB save. Pending acknowledged writes can therefore be lost when the grace period expires.

Fixes apache/openserverless#284. This is separate from the ACL persistence work in #157 and is based directly on Apache 0.9.0.

Change

Add exec to the Redis command in both the deployment manifest and the operator template, so Redis becomes process 1. Image version, config path, storage layout, RDB/AOF settings and permissions are unchanged.

The regression test runs each manifest's configured entrypoint with a controlled server process and verifies that a stop signal reaches that server and permits a clean exit. Both original manifests fail this test; the corrected manifests pass.

Validation

  • 132 Python regression tests passed. The new signal test failed for both manifests before the production change.
  • Same Redis 8.2.1 image tested in isolated pod/PVC experiments: ordinary deletion with the original command loses the newest unsaved write after exit 137 at 30.7 seconds; exec exits 0, logs SIGTERM and the final RDB save, and preserves the write. Explicit SHUTDOWN SAVE and deliberate SIGKILL controls distinguish this defect from the normal RDB crash window.
  • Existing 0.9.0 baseline updated using an explicit SHUTDOWN SAVE during the first transition from the defective command. A subsequent ordinary Redis pod replacement, with a newly acknowledged write and no intervening manual save, exits 0 in 0.9 seconds and preserves the write.
  • Existing Alpha/Beta credentials, ACL fingerprint, tenant definitions and Redis PVC identity preserved. Temporary keys and the test finalizer removed. Alpha/Beta application regression suite: 15 checks passed.
  • Local operator image, its deploy.tar Redis manifest and the mounted deployment file were verified to contain the same fix. The baseline image retains its existing local fixes, including the ACL candidate; it is not a pristine upstream-image test.

Full Kind integration is left to CI. The live tests used small synthetic records, not a production-sized dataset.

Existing installations

The updated Redis StatefulSet command must be applied through the component deployment procedure; replacing only the operator image does not rewrite an existing StatefulSet. The documented first rollout from the old shell command requires an authenticated SHUTDOWN SAVE before the old process can be forcibly terminated. Subsequent ordinary replacements use the corrected signal path.

This does not eliminate the RDB loss window after a hard crash or SIGKILL. The termination grace period must still be sufficient to save the actual dataset.

@sciabarracom
sciabarracom merged commit 93fc94a into apache:0.9.0 Nov 4, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants