feat(exec-harness): declare the benchmarked command's pid

[lvaroqui]

Make exec-harness declare the pid of the command it benchmarks, so its own spawning cost can be left out of simulation results.

Since #531, exec-harness turns instrumentation on in its own process and spawns the command, which inherits that state across fork and exec. The dumped part therefore also holds the harness's cost of spawning the command and waiting for it. That is a fixed overhead on every benchmark, about 207k instructions per command locally.

Changes:

• instrument-hooks binding: new set_executed_benchmark_for_pid(pid, uri). set_executed_benchmark keeps its behavior and delegates to it.
• exec-harness: spawns the command and passes its pid, instead of calling Command::status(). In memory mode the pid only travels to the runner's FIFO, where nothing reads it for exec-harness.
• Submodule bump: feat(valgrind): declare the pid a benchmark ran in instrument-hooks#32 writes desc: Benchmark pid: <pid> in the part when the pid is not the caller's. This bump also brings the thread-safe C API exports and the callgrind_toggle_collect helper.

Checked locally under the patched valgrind, with the runner's simulation flags on sh -c '/bin/true; /bin/true; :':

part: 2
desc: Spawned pid: 289126
desc: Benchmark pid: 289126
desc: Trigger: Client Request: exec_harness::true_twice

Still to do before this is ready:

• Bump the valgrind pin to 0codspeed8 once feat(callgrind): add CALLGRIND_ADD_DESC client request valgrind-codspeed#43 is released. Until then the request is ignored with a warning and results keep the harness's cost.
• Repoint the submodule once feat(valgrind): declare the pid a benchmark ran in instrument-hooks#32 merges.
• The backend support for leaving out the declared launcher cost isn't released yet.

Closes COD-3722

[codspeed]

Merging this PR will degrade performance by 58.02%

⚠️ Unknown Walltime execution environment detected

Using the Walltime instrument on standard Hosted Runners will lead to inconsistent data.

For the most accurate results, we recommend using CodSpeed Macro Runners: bare-metal machines fine-tuned for performance measurement consistency.

⚠️ Different runtime environments detected

Some benchmarks with significant performance changes were compared across different runtime environments,
which may affect the accuracy of the results.

Open the report in CodSpeed to investigate

#### 🎉 Hooray! `exec-harness` just leveled up to 5.4.0!

A heads-up, this is a breaking change and it might affect your current performance baseline a bit. But here's the exciting part - it's packed with new, cool features and promises improved result stability 🥳!
Curious about what's new? Visit our releases page to delve into all the awesome details about this new version.

❌ 1 regressed benchmark
✅ 30 untouched benchmarks
⏩ 6 skipped benchmarks¹

Warning

Please fix the performance issues or acknowledge them on CodSpeed.

Performance Changes

	Mode	Benchmark	BASE	HEAD	Efficiency
❌	Simulation	sleep 1	245.7 µs	585.4 µs	-58.02%

Tip

Investigate this regression by commenting @codspeedbot fix this regression on this PR, or directly use the CodSpeed MCP with your agent.

---

_{Comparing cod-3722-ignore-process-spawning-overhead-in-exec-harness-simulation (a301550) with main (8368147)²}

Footnotes

1. 6 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩

2. No successful run was found on spike/cod-3440-memtrack-musl (ecbfb5b) during the generation of this report, so main (8368147) was used instead as the comparison base. There might be some changes unrelated to this pull request in this report. ↩

[greptile-apps]

Confidence Score: 5/5

[High impact] The reviewed changes appear safe to merge; no actionable defect was found.

Summary

The harness now declares the PID of the command it benchmarks. This lets later readers distinguish the command’s work from the harness’s work.

• Adds set_executed_benchmark_for_pid while preserving the existing method.
• Updates the hooks and Valgrind pins, and allows the runner to run under Valgrind without its file capabilities.
• Adds a test that checks the declared PID has a separate profile.
• lvaroqui explicitly acknowledged that backend support for excluding the harness’s cost is not released yet and that the hooks submodule needs repointing after its upstream PR merges. These known follow-ups are not findings.

Diagram

sequenceDiagram
    participant H as Exec harness
    participant C as Benchmark command
    participant I as Instrument hooks
    participant V as Valgrind
    H->>I: Start benchmark
    H->>C: Spawn command and save PID
    C-->>H: Exit status
    H->>I: Stop benchmark
    H->>I: Declare command PID and URI
    I->>V: Add benchmark PID description
    I->>V: Dump benchmark part

Loading

_{Reviews (1) · Last reviewed commit: "feat(instrument-hooks): declare benchmar..." · Reviewed by Greptile}

[not-matthias]

I think we shouldn't bump valgrind-codspeed in this PR, but rather do it when we're releasing the new runner (e.g. to make it easier revertable).

It shouldn't be needed, as you can use newer versions as an "experimental version". So locally or in CI you can just build from source or install the .deb