Skip to content

About

CLI tool for taking snapshots of AWS Lambda logs and comparing them across deployments

Resources

Stars

0 stars

Watchers

0 watching

Forks

Latest commit

 

History

9 Commits

Folders and files

NameName
Last commit message
Last commit date
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

lambda-deploy-log-compare

A CLI tool for capturing and comparing AWS Lambda invocation logs across deployments. Helps you verify that a new deployment isn't introducing errors, performance regressions, or unexpected behavior by comparing log snapshots taken before and after a deploy.

Install

Download a binary for your OS from release page. Put the executable in a directory in $PATH.

Or you can install by building from source directly as follows. Go 1.25 or later is necessary.

go install github.com/naterator/lambda-deploy-log-compare@latest

To check for a newer GitHub release and replace the current executable:

lambda-deploy-log-compare selfupdate

Quick Start

# Capture baseline snapshots
./lambda-deploy-log-compare capture \
    --function my_func_a,my_func_b \
    --label pre-deploy \
    --out ./snapshots

# Deploy your code...

# Capture post-deploy snapshots
./lambda-deploy-log-compare capture \
    --function my_func_a,my_func_b \
    --label post-deploy \
    --out ./snapshots

# Compare
./lambda-deploy-log-compare compare \
    --a ./snapshots/my_func_a_pre-deploy.json \
    --b ./snapshots/my_func_a_post-deploy.json \
    --strict

If ./snapshots does not exist yet, the tool creates it before writing snapshot files.

Commands

Use --help or <command> --help for help (exit status zero). Commands accept named flags; unexpected positional arguments are rejected before work starts, including arguments that would otherwise hide subsequent flags.

capture

Fetches CloudWatch Logs for one or more Lambda functions and saves a snapshot of invocations.

lambda-deploy-log-compare capture --function <name>[,<name>,...] --label <label> [options]
Flag Default Description
--function Unique Lambda function name(s), comma-separated (required)
--label Label for the snapshot, e.g. pre-deploy (required)
--count 20 Number of invocations to capture. Must be greater than 0.
--offset 0 Skip this many recent invocations before capturing. Must be 0 or greater.
--out . Output directory for snapshot JSON files
--region us-west-2 AWS region
--profile AWS CLI profile name
--overwrite false Replace an existing snapshot file with the same function and label

Log groups are derived as /aws/lambda/<function-name>.

Duplicate function names are rejected so one capture run cannot silently rescan the same Lambda and overwrite the same snapshot path.

Output files are named from sanitized <function-name>_<label>.json components in the output directory. Path separators and traversal-style input such as .. are rewritten. Before AWS access, capture rejects output-name collisions (including case-insensitive collisions), destination symlinks, and existing files unless --overwrite is set. Each function gets its own snapshot file, and the output directory is created automatically when a snapshot is written.

Snapshots are staged and flushed in the output directory, then published atomically. Exclusive creation remains safe under concurrent writers; overwrite replaces the directory entry without following destination symlinks and never broadens an existing file's permissions. Failures before publication preserve the old file. A directory-flush failure after publication explicitly reports that the snapshot was already published.

The --offset flag lets you look further back in time. For example, --offset 100 --count 20 skips the 100 most recent invocations and captures the 20 after that. This is useful for grabbing a historical baseline to compare against.

If no log streams are found for a function, the command exits successfully for that function and does not write a snapshot file.

compare

Loads two snapshot files and prints a side-by-side comparison:

lambda-deploy-log-compare compare --a <baseline.json> --b <new.json> [options]
Flag Default Description
--a Path to baseline snapshot JSON file (required)
--b Path to new snapshot JSON file (required)
--strict false Fail if either snapshot is missing function_name / log_group, or if those fields disagree
--strict-records false Validate invocation IDs, timestamps, metric values, and error consistency
--fail-on-regression false Exit non-zero if the new snapshot has more errors or new error patterns
--max-duration-regression-pct 0 Exit non-zero if new p90 duration exceeds baseline p90 by more than this percentage. 0 disables this gate.
--max-memory-regression-pct 0 Exit non-zero if new max peak memory exceeds baseline max peak memory by more than this percentage. 0 disables this gate.
--json false Print a machine-readable JSON comparison summary instead of the human report

The comparison includes:

Numeric thresholds must be finite and nonnegative. Malformed, negative, and non-finite snapshot metrics are excluded and counted. Relative metric gates cannot evaluate a zero baseline or an unrepresentable percentage; when enabled, these cases return nonzero with an explanation. JSON marks them with relative_change_defined: false rather than encoding infinity or treating the result as zero change.

  • Error count — warns if errors increased
  • Error patterns — new patterns that appeared, old patterns that disappeared
  • Regression gates — --fail-on-regression returns a non-zero exit after printing the report if errors increased or new error patterns appeared; the duration and memory threshold flags also return non-zero when their configured gates are exceeded
  • Duration stats — min, avg, p50, p90, max (in milliseconds), ignoring malformed values with an explicit ignored count
  • Memory usage — min, avg, max peak memory (in MB), ignoring malformed values with an explicit ignored count
  • Log pattern diff — new and gone log line patterns, normalizing request-specific values such as UUIDs, request IDs, timestamps, ARNs, long numeric IDs, and durations
  • JSON output — --json emits counts, pattern diffs, metric stats, warnings, and regression reasons for CI consumers
  • Snapshot mismatch warnings — warns when the two files appear to be from different Lambda functions or log groups; --strict also fails when snapshot identity fields are missing

selfupdate

Checks the latest GitHub release, downloads the matching binary and .sha256 checksum for the current OS/architecture, verifies the checksum, and replaces the running executable when a newer release is available.

lambda-deploy-log-compare selfupdate

version

Prints the current build version and exits. Release builds can set this with -ldflags "-X main.appVersion=v1.2.3".

lambda-deploy-log-compare version

Snapshot Format

Each snapshot file contains one Snapshot object with metadata plus an invocations array.

New captures include schema_version: 1 and tool_version. Unversioned snapshots remain readable; unknown future versions are rejected. Empty arrays serialize as []. See the snapshot/report contract for compatibility rules and the shared design for completeness, structured logs, and provenance.

Comparison JSON includes schema/tool versions, status, exit_code, and stable regression.reason_codes. Invalid input in JSON mode emits an error envelope with a stable code and diagnostic message. Exit statuses are: 0 success, 1 operational failure, 2 invalid input, 3 capture failure, 4 insufficient evidence under an enabled policy, and 5 regression under an enabled gate. Regression reports are printed before the nonzero exit.

lambda-deploy-log-compare compare --a baseline.json --b new.json \
  --strict --strict-records --fail-on-regression --json > comparison.json

CI should retain comparison.json on nonzero exits and interpret its status; invalid input and a detected regression require different actions.

Important field notes:

  • duration and billed_ms come from Lambda REPORT lines.
  • memory_size_mb stores Lambda Memory Size, which is the configured memory size for the function.
  • max_memory_used_mb stores Lambda Max Memory Used, which is the value used for memory comparison stats and peak_mem in compare output.
  • Invocations that never emit a REPORT line are still captured, with blank duration and memory fields, so crashes and truncated runs are not silently dropped.
  • Runtime REPORT statuses such as error and timeout, and runtime error types, mark an invocation as failed. A stable diagnostic is retained in error_lines. Explicit error severity takes precedence over benign counter wording on the same line.
  • Older snapshots that used mem_used_mb and max_mem_mb still load correctly.
  • Compare rejects files that have neither snapshot metadata nor invocation records, so unrelated JSON is not treated as an empty snapshot.

AWS Authentication

Uses the standard AWS SDK credential chain (environment variables, ~/.aws/credentials, IAM roles, etc.). Use --profile to select a named profile and --region to set the region.

The IAM principal needs these permissions:

  • logs:DescribeLogStreams on the target log groups
  • logs:GetLogEvents on the target log groups

How It Works

  1. Stream discovery — Fetches recent log streams ordered by last event time (most recent first) page by page until the requested offset + count is satisfied or the log group is exhausted.
  2. Invocation parsing — Reads each stream from newest events backward, tracks invocations from Lambda START and REPORT markers, and uses inline RequestId hints plus current stream context to associate log lines that land outside the normal START -> logs -> REPORT sequence. Extracts duration, billed duration, memory size, and max memory used from REPORT lines when present, while still preserving recent invocations that crashed or were truncated before REPORT. Each stream gets a 30-second timeout, collection stops early once enough newest invocations have their START boundary in the collected pages, and pages fetched before a later stream read failure are still used with a warning.
  3. Error detection — Flags explicit error-style log lines such as error, panic, fatal, traceback, exception, and Lambda runtime failure messages, while skipping common benign counter-style phrases like error_count=0.
  4. Snapshot selection — From all discovered invocations (sorted by time, most recent first), skips the first offset invocations, then takes the next count.
  5. Pattern normalization — For comparison, complete log lines are normalized by collapsing request-specific values such as UUID-like hex strings (32+ chars), Lambda request IDs, timestamps, ARNs, long numeric IDs, and durations. Pattern identity and JSON retain the full normalized line. Human reports display up to 20 entries per pattern category and truncate each entry to 120 Unicode characters; display limits never affect regression decisions.

Development and release checks

Run bash scripts/check.sh for formatting, vet, race-enabled tests, a build, and native CLI smoke tests. CI runs this on Linux and macOS with Go 1.25.0 and the current stable toolchain. Release asset publication waits for the same checks on the release commit.

bash scripts/build-release.sh v1.2.3 /tmp/release-assets builds every supported target into an empty directory, verifies target build metadata, generates SHA-256 checksums, and verifies all expected binary/checksum pairs before upload. The release builder uses the version in go.mod; toolchain changes are explicit repository changes.

OS Release architectures Verification
Linux amd64, arm64 Cross-builds; native CI smoke tests on the Linux runner architecture
macOS amd64, arm64 Cross-builds; native CI smoke tests on the macOS runner architecture
FreeBSD, NetBSD, OpenBSD amd64, arm64 Cross-build and artifact checks
Windows None Release binaries and self-update are unsupported

Cross-compilation does not establish runtime behavior on the BSD targets or on architectures without a native CI runner.

Project Structure

main.go       CLI entry point, flag parsing, AWS client setup
capture.go    LogsClient interface, log stream/event fetching, snapshot writing
compare.go    Snapshot loading, diff/stats computation, report output
parse.go      Invocation parsing from CloudWatch log events
types.go      Shared data types (InvocationSummary, InvocationRecord, Snapshot)

Example Output

=== Comparison: my_transformer_func ===
  Baseline: ./snapshots/my_transformer_func_pre-deploy.json (label: pre-deploy, captured: 2026-02-25T10:00:00Z)
  New:      ./snapshots/my_transformer_func_post-deploy.json (label: post-deploy, captured: 2026-02-25T12:00:00Z)

--- BASELINE: my_transformer_func [pre-deploy] (log group: /aws/lambda/my_transformer_func) ---
  Invocations captured: 20
  Errors: 0
  Recent invocations:
    2026-02-25T09:59:58Z  dur=45.2 ms  peak_mem=85 MB  mem_size=128 MB
    2026-02-25T09:59:52Z  dur=98.3 ms  peak_mem=90 MB  mem_size=128 MB

--- NEW: my_transformer_func [post-deploy] (log group: /aws/lambda/my_transformer_func) ---
  Invocations captured: 20
  Errors: 0
  Recent invocations:
    2026-02-25T11:59:59Z  dur=42.1 ms  peak_mem=84 MB  mem_size=128 MB
    2026-02-25T11:59:54Z  dur=95.0 ms  peak_mem=91 MB  mem_size=128 MB

=== Error Comparison ===
  Baseline errors: 0 / 20 invocations
  New errors:      0 / 20 invocations
  Error count unchanged

=== Duration Comparison ===
  Baseline: min=45.2ms avg=120.5ms p50=98.3ms p90=210.1ms max=250.7ms (n=20, ignored=0 malformed)
  New     : min=42.1ms avg=115.8ms p50=95.0ms p90=205.3ms max=245.2ms (n=20, ignored=0 malformed)

=== Memory Usage Comparison ===
  Baseline: min=85MB avg=92MB max=105MB (n=20, ignored=0 malformed)
  New     : min=84MB avg=91MB max=103MB (n=20, ignored=0 malformed)

=== Log Pattern Diff ===
  No significant log pattern differences detected

About

CLI tool for taking snapshots of AWS Lambda logs and comparing them across deployments

Resources

Stars

0 stars

Watchers

0 watching

Forks

Releases

Used by

Contributors

Languages