You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Update BenchmarkDotNet globally to nightly662 - #5348
Update the single global BenchmarkDotNet version from 0.16.0-nightly.20260801.595 to 0.16.0-nightly.20261029.662, separately from the SDK/TFM migration in dotnet/performance#5344.
Leave NuGet.config unchanged and consume the verified, locally source-built packages from the existing benchmark-dotnet-prerelease feed in the dnceng public project. The upstream comparison baseline is the successful nightly CI run, source 7d72eaef1caeb1b511250368bcfcafc0cdd650bf.
Preserve historical WASM runtime spellings and --wasmRuntimeFlavor CoreCLR, with explicit errors for conflicting options rather than silently selecting another runtime.
Preserve native Mono AOT+LLVM through the custom runtime-pack toolchain, adapted from the earlier compatibility work in dotnet/performance#5279. Keep the existing compiler modes, runtime pack, and target-framework identity; do not replace it with legacy Mono AOT. Reject mixed CoreRun or active in-process requests explicitly, and resolve compiler/runtime-pack paths against the invocation directory before generating a child project.
Retain only the URI/TLS argument typing, direct async-LINQ dependency and AOT filtering required by the newer APIs. The optional HTTP cleanup, SSL setup and TLS capability-discovery async conversions have been removed after testing the original synchronous code with BDN662.
Preserve the enum benchmarks' historical Colors parameter specialization and values through explicit casts. Use the existing FullNameProvider unchanged throughout; the added BenchmarkNameProvider, its consumer changes and exporter test-injection machinery have been removed.
Align NuGetPackageRoot with an explicit restore cache before Arcade imports so generated/published children receive the weaver targets. Defer WASM R2R/WebCIL work only during reference gathering; retain the final publish modes and output guards.
Ignore the newer runner's project-relative .bdn/ generated build/publish tree. Its generated gen.cs/gen.csproj, reference-gathering files, binaries and binlogs are separate from result reports in BenchmarkDotNet.Artifacts/, so the existing result-directory ignore does not cover them.
Raise the Roslyn runner's direct Microsoft.CodeAnalysis.CSharp dependency from 5.3.0 to 5.6.0 to satisfy the updated dependency floor. This compiler dependency change must be considered separately in Roslyn comparisons.
There are no per-TFM BenchmarkDotNet version splits, target-framework retargeting, pool changes, diagnostic capture hooks, lane disabling, benchmark-body probes, or measurement-policy/timeout workarounds in this PR.
Why these compatibility changes are needed
The scope was re-audited against the actual 662 package, not retained solely because it appeared in an earlier PR:
Change
Necessity and verified behavior
HTTP GetAwaiter().GetResult() cleanup
Async conversions removed. Original synchronous SocketsHttpHandler/WinHttpHandler cleanup compiled and completed all 16 selected parameter cases with actual workloads/final statistics under BDN662. No change remains in either HTTP source file.
SSL setup and TLS capability discovery
Async conversions removed. Original Task.WaitAll setup and synchronous TLS 1.3 capability detection completed discovery and three actual SSL setup/handshake cases. Only the required typed TLS argument source remains.
URI/TLS object[] to scalar sources
Required build compatibility. Restoring the original one-argument IEnumerable<object[]> sources produces eight BDN1507 errors: the source yields object[] where the method takes string or SslProtocols. The scalar source preserves the same values; it does not change benchmark bodies.
Explicit (Colors) on two enum attributes
Required generated-code compatibility. Without the casts, the host builds but the actual generated child fails CS0266 converting DayOfWeek to Perf_Enum.Colors, with zero executed cases. The casts preserve the Colors method signatures/generic specialization and values 6 and 42 rather than changing the workload to DayOfWeek.
FullNameProvider / identity tradeoff
No replacement provider or mapping. The correctly typed value 6 is displayed/exported as Orange, Yellow, not the old attribute's Saturday. With the existing 15-way hash, the Span case remains P4 and the StringBuilder case moves from P10 to P12. Tests check the actual names/partitions and unchanged value-42 identity. Retained raw results remain source-qualified; they are not rewritten. The three diagnostic partitions below do not contain these two cases.
BenchmarkDotNetGatherReferences checks
Required stage compatibility. BDN662 itself introduces this property for a reference-gathering build with SkipCompilerExecution=true and no generated entry DLL. We do not add a measurement mode: we skip final R2R/WebCIL/output work only in that stage. Removing the guard fails seven property/target checks on nonexistent outputs. Real per-assembly/composite publish flags and output guards remain enabled, as verified in the retained actual-publish evidence.
System.Linq.AsyncEnumerable reference
Required dependency. Removing it produces CS1061 for the existing ToAsyncEnumerable() calls in all three validators. The updated BDN package no longer supplies that transitive API. Keep the existing enumerable adapters rather than introducing a different async iterator/state-machine implementation.
Restore-root alignment and its regression test
Required generated-child weaving.dotnet/BenchmarkDotNet#3274 changed generated build/publish commands from explicit /p:NuGetPackageRoot=... to GetPackagesPathArgument, which returns /p:RestorePackagesPath=... (source). Those separate MSBuild child processes do not inherit the outer Python build command's global properties. Aligning the root before Arcade imports keeps generated annotations/weaver imports on the explicit custom restore cache. Removing the four-line alignment makes the custom-cache regression fail; the explicit command-line-property case still passes. Earlier actual host/published-child inspection demonstrated missing NoInlining before this fix and correct flags afterward. This supports the custom-cache integration requirement, not a bisected universal upstream regression.
Native Mono port, runtime guards and paths
Required preservation of an existing lane. The old Mono LLVM CLI/toolchain was removed upstream; the port retains the custom compiler/pack and existing modes. Actual parser tests demonstrate that CoreRun or in-process expansion can otherwise replace the default toolchain. Active in-process options are rejected before consuming Mono arguments; explicit false values are handled without losing the Mono job. BDN's boolean option is presence-based even when followed by false, so disabled options are removed only for this Mono compatibility path. Ordinary non-Mono parsing is unchanged. Relative compiler/pack paths are resolved before changing to the generated project's directory.
Runtime selectors, SDK/cache values and response files
Recognize BDN662's case-insensitive -R and long --runtimes=... forms before the single-runtime check, and pass --cli=.../--packages=... to the final Mono toolchain; mixed separate/equals duplicates fail without consuming the caller's input. Response-file limitation: legacy Mono/WASM compatibility options must be inline without @files, including when the legacy selector is inside a file. BDN's response expansion is private, so native response files are validated by its public parser and passed through with its own quoting rules; no custom tokenizer or hidden runtime override is introduced.
AOT filter and Roslyn dependency
The old AOT filter API is no longer supplied; the compatibility filter preserves existing exclusions, including the custom Mono LLVM runtime. The Roslyn runner needs the 5.6.0 direct dependency to avoid the updated BDN dependency floor being downgraded; this is disclosed separately from measurement effects.
.bdn/ ignore
BDN662's CsProjBuilder.ProjectLocalFolderName is .bdn. Actual generated gen.cs/gen.csproj, reference files, binlogs and build/publish children are outside the already ignored result-report directory. Keep only the narrow generated-output ignore.
The added narrative in docs/benchmarkdotnet.md has also been removed; that file now matches base. These reasons belong in this PR overview rather than expanding user-facing documentation for preserved CLI behavior.
Package provenance and local validation
The established flow builds this exact nightly locally on Linux with the CI SDK/build settings, verifies it against the upstream published artifacts, then publishes the verified source-built packages to the existing dnceng public-project feed. That work is already complete for all eight packages at source 7d72eaef1caeb1b511250368bcfcafc0cdd650bf; the correction reuses the immutable packages and publication receipts rather than rebuilding, pushing again, overwriting or replacing them with downloaded upstream ZIPs.
The local ZIPs are not byte-identical to the upstream CI/MyGet packages. The narrow equivalence proof covered a symmetric inventory of eight packages/118 entries: 100 identical entries, 16 fully decoded differing DLLs, and two full-content XML member reorderings. It checked signatures, IL/constants, attributes, resources, actual generated-source/PDB provenance, permitted polyfill ordering and exact generated-JS-name differences, structural PE coverage, and the independent equal-DLL-file-size condition. Five semantic and fourteen structural negative controls were rejected. This is a package-content equivalence proof, not a performance-equivalence or SAFE claim.
Current availability/readback checks confirm that all eight feed ZIP SHA256s still exactly match the verified Linux publication. Production consumers use these source-built packages through the unchanged repository source configuration, as did the retained diagnostic campaign.
The feed correction was checked using a fresh restore cache and a repository-compatible SDK, with the existing public downlevel reference-pack provisioning API. NuGet's actual package-source metadata identifies the unchanged dnceng public-project feed, and the freshly consumed BenchmarkDotNet/Annotations ZIP SHA256s match the verified Linux publication and current feed readbacks.
python -m pytest -q scripts/tests/test_wasm_coreclr_r2r.py scripts/tests/test_run_performance_job.py scripts/tests/test_dotnet.py -p no:cacheprovider — 150 passed, with the repository script import path configured.
dotnet test src/tests/harness/BenchmarkDotNet.Extensions.Tests/BenchmarkDotNet.Extensions.Tests.csproj -c Release --nologo -v minimal -p:PERFLAB_TARGET_FRAMEWORKS=net10.0 -p:UseAppHost=false "-p:RestorePackagesPath=$env:NUGET_PACKAGES" — 206 passed, 0 skipped, including actual FullNameProvider/partition identities, final parsed in-process/Mono jobs, relative/absolute generated-project path contracts, runtime-selector aliases, response files, and final SDK/cache settings.
dotnet build src/benchmarks/micro/MicroBenchmarks.csproj -c Release -f <tfm> --nologo -v minimal -p:UseAppHost=false -p:PERFLAB_TARGET_FRAMEWORKS=<tfm> "-p:RestorePackagesPath=$env:NUGET_PACKAGES" — passed with 0 warnings/errors for each of net8.0, net9.0, net10.0 and net11.0, using the existing feed.
Actual local CoreCLR functional execution of the three typed URI argument-source methods and both enum interpolation methods using --launchCount 1 --warmupCount 1 --iterationCount 1 --invocationCount 1 --unrollFactor 1 --strategy ColdStart — 26 cases produced finite final statistics and Actual workloads. Together with the 16 original synchronous HTTP and three original synchronous SSL cases described above, the minimality audit completed 45 local functional cases, not a new lab campaign.
These are focused local restore/build/functional checks, not new lab measurements or a performance-equivalence claim.
For the subsequent argument-parsing review, the same full harness passed 206 tests, and five actual entrypoint parse/list probes preserved an ordinary native response file and explicitly rejected hidden or mixed runtime overrides without executing benchmarks. BDN662's list-only entrypoint still returns 1 for an empty summary; this was not counted as a successful workload. No new lab run or package publication was performed.
The minimality checks above separately exercised the actual 662 analyzers, generated enum compilation, dependency removal, reference-gathering targets and restore-root regression. The original synchronous HTTP/SSL checks used the actual generated runner with --launchCount 1 --warmupCount 1 --iterationCount 1 --invocationCount 1 --unrollFactor 1 --strategy ColdStart; 16 HTTP cases and three SSL cases completed. Negative probes were retained as evidence, not counted as successful workloads.
The earlier compatibility checks for this same update also covered the supported .NET 8/9/10/11 microbenchmark builds, the Roslyn runner build, historical CLI forms, custom-cache forced publishing with an empty BenchmarkDotNet fallback, and actual isolated plain/composite WASM execution using a pinned V8 engine. The focused feed-correction checks above are distinguished from those retained results. Native Mono runtime execution is not established by a CoreCLR smoke test.
Measured differences retained for follow-up
This PR intentionally records the unresolved differences rather than claiming the update is SAFE or regression-free. Further attribution is deferred; the following is a narrow, configuration-matched diagnostic, not full-profile/global validation.
Exactly three original whole partitions were compared using the same retained runtime/SDK/workload cohort: SDK 12.0.100-alpha.1.26471.108, WASM pack 12.0.0-ci, V8 15.6.75, and identical runtime identity, original 15-way partition assignment, modes and measurement settings within each comparison. All 1,155 candidate cases produced final statistics, and generated-source/published-child artifacts and publish modes were checked after measurement.
Profile
Valid percentage pairs vs 595
Median case delta
p90 case delta
Composite P11, affected, 365 cases
363; 2 excluded
+9.910667%
+29.855580%
Composite P14, stable, 381 cases
378; 3 excluded
-0.080226%
+8.177995%
Plain P5, 409 cases
409
-0.100384%
+5.448498%
These are percentiles of the distribution of per-benchmark case comparisons, not benchmark tail latency or statistical significance. Excluded percentage pairs have nonpositive denominators; they are not missing candidate workloads. No numerical noise-floor subtraction was performed.
The affected composite partition still has 208 cases more than 5% slower and 77 more than 5% faster than the matched 595 reference. Its median delta versus the captured 641 reference is approximately -0.187871%; the newer version does not remove this composite shift. The stable partition's median remains near zero.
For System.Collections.Sort<IntClass>.Array_Comparison(Size: 512) in plain P5, 662 completed with a final mean of 4.9361 ms, versus 4.8184 ms in the retained 595 recovery reference, approximately +2.44%. The retained 641 attempt has partial Actual measurements but no final statistics. It must not be presented as a formal 47% regression fix or a formal final-statistics comparison against 641.
The actual plain child retained benchmark NoInlining and generated-caller flags. Its native/runtime/timing payload bytes match the retained references and its runtimeconfig properties match 641. Those observations do not establish full generated/native-code equivalence or a causal explanation.
CPU model, OS/kernel and timer metadata were checked within this cohort. Actual CPU frequency, affinity and governor state were not established. These results neither attribute an underlying runtime regression nor establish global compatibility.
Separate SDK integration gap
The retained 12.0.100-alpha.1.26471.108 cohort does not clear the separate composite generated-project NETSDK1112 failure observed with SDK 12.0.100-alpha.1.26528.120 targeting net11.0. Runtime-pack selection/staging across that SDK/TFM rollover still requires coherent-version validation, including a real final publish. No SDK/TFM retargeting or fix for that integration gap is included here.
Only a sanitized summary is included here. Private infrastructure identifiers, endpoints, raw captures and internal logs are not attached.
Preserve historical WASM and native Mono AOT+LLVM configuration, generated-build weaving, typed arguments, async lifecycle and AOT exclusions. Consume the official published nightly packages. Recorded WASM measurement differences remain for follow-up; this is not a regression-free claim.
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
The reason will be displayed to describe this comment to others. Learn more.
🟡 Changes recommended
Unresolved issues break relative compiler/runtime-pack paths, silently drop Mono AOT measurements when combined with CoreRun, and change enum benchmark names used for historical results and partition keys.
Updates the repository-wide BenchmarkDotNet dependency to nightly 662, separately from the SDK and target-framework migration, with compatibility changes for existing benchmark runners.
Changes:
Switches to the official nightly feed and updates supporting dependencies.
The reason will be displayed to describe this comment to others. Learn more.
🟡 Changes recommended
Active in-process options can bypass the requested Mono toolchain and produce host-runtime measurements, while the official nightly package source remains unconfigured.
Keep original synchronous HTTP and SSL lifecycles after actual BDN662 compatibility checks. Use FullNameProvider unchanged and document the typed enum name/partition migration instead of maintaining an identity abstraction. Retain proven argument, reference-gathering, async-LINQ and restore-root requirements; reject active in-process Mono LLVM requests without losing explicitly disabled jobs.
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
The reason will be displayed to describe this comment to others. Learn more.
🔵 Needs a closer look
Argument-parsing gaps allow response files and alternative runtime selectors to bypass Mono LLVM compatibility checks and select a different runtime, while equals-form SDK and cache options are ignored.
No open issues 🎉
3 previously missed issues
In code that hasn't changed since last review
Expand response files before compatibility validation
src/benchmarks/micro/Program.cs:41
These compatibility checks run before BenchmarkDotNet expands @response files. A valid inline Mono LLVM request followed by a response file containing --inProcess or --corerun therefore bypasses validation and can run a different runtime; legacy WASM options in response files also bypass normalization. Expand response files before compatibility parsing using BenchmarkDotNet-compatible tokenization, or explicitly reject response files for these compatibility requests.
Recognize runtime selector aliases and attached values
BenchmarkDotNet accepts -R and --runtimes=net8.0, but this helper recognizes neither. Appending either selector to an otherwise valid Mono LLVM request bypasses the single-runtime guard; after the Mono arguments are consumed, BenchmarkDotNet replaces its custom toolchain with CoreCLR. Handle case-insensitive aliases and attached runtime values consistently in both scanning and consumption so mixed-runtime requests fail explicitly.
The equals forms accepted by BenchmarkDotNet, such as --cli=/sdk/dotnet and --packages=/cache, are not recognized here, leaving the Mono LLVM toolchain's SDK/cache settings null. After the runtime selector is consumed, BenchmarkDotNet does not apply these options to the existing toolchain, so the generated child uses defaults instead. Recognize equals-form values and include them in duplicate-option checks.
Validate native response files through BenchmarkDotNet and explicitly reject legacy compatibility requests using response files. Recognize case-insensitive runtime aliases and attached long runtime values, preserve SDK/cache equals options in Mono toolchain settings, and reject duplicate values without consuming caller arguments. Cover final parsed jobs and unchanged native requests.
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Response files: BDN662's response expansion is private. Rather than add another tokenizer, native @files keep BDN's parser/quoting rules. Legacy Mono LLVM/WASM compatibility requests combined with response files now fail explicitly, including selectors inside the file and hidden CoreRun/in-process overrides. Use inline compatibility options without @files, or native BDN options/runtime names in response files.
Runtime selectors: The adapter now recognizes the supported case-insensitive -R alias and long --runtimes=... values before checking for mixed runtimes. Tests check the final selected runtime/toolchain, not just rewritten tokens. Ordinary native requests remain unchanged; rejected Mono requests retain their original input.
SDK/cache values:--cli=... and --packages=... reach the final Mono toolchain settings, including paths with spaces. Duplicate options across separate/equals forms and case variants fail explicitly instead of leaving defaults.
The full harness passes 206 tests, 0 skipped. Five actual microbenchmark-entrypoint parse/list checks also preserved a native response-file listing and rejected the override cases without running benchmarks. The overview documents the response-file limitation. No lab runs, package publication, feed changes, or changes to the recorded measurement/integration qualifications.
The reason will be displayed to describe this comment to others. Learn more.
🔵 Needs a closer look
Human assessment is needed because Mono AOT+LLVM can silently ignore --freshPackages, causing restores to use the shared cache rather than an empty per-build directory.
No open issues 🎉
Review effort: Balanced
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Update the single global BenchmarkDotNet version from
0.16.0-nightly.20260801.595to0.16.0-nightly.20261029.662, separately from the SDK/TFM migration in dotnet/performance#5344.NuGet.configunchanged and consume the verified, locally source-built packages from the existingbenchmark-dotnet-prereleasefeed in the dnceng public project. The upstream comparison baseline is the successful nightly CI run, source7d72eaef1caeb1b511250368bcfcafc0cdd650bf.--wasmRuntimeFlavor CoreCLR, with explicit errors for conflicting options rather than silently selecting another runtime.Colorsparameter specialization and values through explicit casts. Use the existingFullNameProviderunchanged throughout; the addedBenchmarkNameProvider, its consumer changes and exporter test-injection machinery have been removed.NuGetPackageRootwith an explicit restore cache before Arcade imports so generated/published children receive the weaver targets. Defer WASM R2R/WebCIL work only during reference gathering; retain the final publish modes and output guards..bdn/generated build/publish tree. Its generatedgen.cs/gen.csproj, reference-gathering files, binaries and binlogs are separate from result reports inBenchmarkDotNet.Artifacts/, so the existing result-directory ignore does not cover them.Microsoft.CodeAnalysis.CSharpdependency from 5.3.0 to 5.6.0 to satisfy the updated dependency floor. This compiler dependency change must be considered separately in Roslyn comparisons.There are no per-TFM BenchmarkDotNet version splits, target-framework retargeting, pool changes, diagnostic capture hooks, lane disabling, benchmark-body probes, or measurement-policy/timeout workarounds in this PR.
Why these compatibility changes are needed
The scope was re-audited against the actual 662 package, not retained solely because it appeared in an earlier PR:
GetAwaiter().GetResult()cleanupTask.WaitAllsetup and synchronous TLS 1.3 capability detection completed discovery and three actual SSL setup/handshake cases. Only the required typed TLS argument source remains.object[]to scalar sourcesIEnumerable<object[]>sources produces eightBDN1507errors: the source yieldsobject[]where the method takesstringorSslProtocols. The scalar source preserves the same values; it does not change benchmark bodies.(Colors)on two enum attributesCS0266convertingDayOfWeektoPerf_Enum.Colors, with zero executed cases. The casts preserve theColorsmethod signatures/generic specialization and values 6 and 42 rather than changing the workload toDayOfWeek.FullNameProvider/ identity tradeoffOrange, Yellow, not the old attribute'sSaturday. With the existing 15-way hash, the Span case remains P4 and the StringBuilder case moves from P10 to P12. Tests check the actual names/partitions and unchanged value-42 identity. Retained raw results remain source-qualified; they are not rewritten. The three diagnostic partitions below do not contain these two cases.BenchmarkDotNetGatherReferenceschecksSkipCompilerExecution=trueand no generated entry DLL. We do not add a measurement mode: we skip final R2R/WebCIL/output work only in that stage. Removing the guard fails seven property/target checks on nonexistent outputs. Real per-assembly/composite publish flags and output guards remain enabled, as verified in the retained actual-publish evidence.System.Linq.AsyncEnumerablereferenceCS1061for the existingToAsyncEnumerable()calls in all three validators. The updated BDN package no longer supplies that transitive API. Keep the existing enumerable adapters rather than introducing a different async iterator/state-machine implementation./p:NuGetPackageRoot=...toGetPackagesPathArgument, which returns/p:RestorePackagesPath=...(source). Those separate MSBuild child processes do not inherit the outer Python build command's global properties. Aligning the root before Arcade imports keeps generated annotations/weaver imports on the explicit custom restore cache. Removing the four-line alignment makes the custom-cache regression fail; the explicit command-line-property case still passes. Earlier actual host/published-child inspection demonstrated missing NoInlining before this fix and correct flags afterward. This supports the custom-cache integration requirement, not a bisected universal upstream regression.-Rand long--runtimes=...forms before the single-runtime check, and pass--cli=.../--packages=...to the final Mono toolchain; mixed separate/equals duplicates fail without consuming the caller's input. Response-file limitation: legacy Mono/WASM compatibility options must be inline without@files, including when the legacy selector is inside a file. BDN's response expansion is private, so native response files are validated by its public parser and passed through with its own quoting rules; no custom tokenizer or hidden runtime override is introduced..bdn/ignoreCsProjBuilder.ProjectLocalFolderNameis.bdn. Actual generatedgen.cs/gen.csproj, reference files, binlogs and build/publish children are outside the already ignored result-report directory. Keep only the narrow generated-output ignore.The added narrative in
docs/benchmarkdotnet.mdhas also been removed; that file now matches base. These reasons belong in this PR overview rather than expanding user-facing documentation for preserved CLI behavior.Package provenance and local validation
The established flow builds this exact nightly locally on Linux with the CI SDK/build settings, verifies it against the upstream published artifacts, then publishes the verified source-built packages to the existing dnceng public-project feed. That work is already complete for all eight packages at source
7d72eaef1caeb1b511250368bcfcafc0cdd650bf; the correction reuses the immutable packages and publication receipts rather than rebuilding, pushing again, overwriting or replacing them with downloaded upstream ZIPs.The local ZIPs are not byte-identical to the upstream CI/MyGet packages. The narrow equivalence proof covered a symmetric inventory of eight packages/118 entries: 100 identical entries, 16 fully decoded differing DLLs, and two full-content XML member reorderings. It checked signatures, IL/constants, attributes, resources, actual generated-source/PDB provenance, permitted polyfill ordering and exact generated-JS-name differences, structural PE coverage, and the independent equal-DLL-file-size condition. Five semantic and fourteen structural negative controls were rejected. This is a package-content equivalence proof, not a performance-equivalence or SAFE claim.
Current availability/readback checks confirm that all eight feed ZIP SHA256s still exactly match the verified Linux publication. Production consumers use these source-built packages through the unchanged repository source configuration, as did the retained diagnostic campaign.
The feed correction was checked using a fresh restore cache and a repository-compatible SDK, with the existing public downlevel reference-pack provisioning API. NuGet's actual package-source metadata identifies the unchanged dnceng public-project feed, and the freshly consumed BenchmarkDotNet/Annotations ZIP SHA256s match the verified Linux publication and current feed readbacks.
python -m pytest -q scripts/tests/test_wasm_coreclr_r2r.py scripts/tests/test_run_performance_job.py scripts/tests/test_dotnet.py -p no:cacheprovider— 150 passed, with the repository script import path configured.dotnet test src/tests/harness/BenchmarkDotNet.Extensions.Tests/BenchmarkDotNet.Extensions.Tests.csproj -c Release --nologo -v minimal -p:PERFLAB_TARGET_FRAMEWORKS=net10.0 -p:UseAppHost=false "-p:RestorePackagesPath=$env:NUGET_PACKAGES"— 206 passed, 0 skipped, including actual FullNameProvider/partition identities, final parsed in-process/Mono jobs, relative/absolute generated-project path contracts, runtime-selector aliases, response files, and final SDK/cache settings.dotnet build src/benchmarks/micro/MicroBenchmarks.csproj -c Release -f <tfm> --nologo -v minimal -p:UseAppHost=false -p:PERFLAB_TARGET_FRAMEWORKS=<tfm> "-p:RestorePackagesPath=$env:NUGET_PACKAGES"— passed with 0 warnings/errors for each of net8.0, net9.0, net10.0 and net11.0, using the existing feed.--launchCount 1 --warmupCount 1 --iterationCount 1 --invocationCount 1 --unrollFactor 1 --strategy ColdStart— 26 cases produced finite final statistics and Actual workloads. Together with the 16 original synchronous HTTP and three original synchronous SSL cases described above, the minimality audit completed 45 local functional cases, not a new lab campaign.These are focused local restore/build/functional checks, not new lab measurements or a performance-equivalence claim.
For the subsequent argument-parsing review, the same full harness passed 206 tests, and five actual entrypoint parse/list probes preserved an ordinary native response file and explicitly rejected hidden or mixed runtime overrides without executing benchmarks. BDN662's list-only entrypoint still returns 1 for an empty summary; this was not counted as a successful workload. No new lab run or package publication was performed.
The minimality checks above separately exercised the actual 662 analyzers, generated enum compilation, dependency removal, reference-gathering targets and restore-root regression. The original synchronous HTTP/SSL checks used the actual generated runner with
--launchCount 1 --warmupCount 1 --iterationCount 1 --invocationCount 1 --unrollFactor 1 --strategy ColdStart; 16 HTTP cases and three SSL cases completed. Negative probes were retained as evidence, not counted as successful workloads.The earlier compatibility checks for this same update also covered the supported .NET 8/9/10/11 microbenchmark builds, the Roslyn runner build, historical CLI forms, custom-cache forced publishing with an empty BenchmarkDotNet fallback, and actual isolated plain/composite WASM execution using a pinned V8 engine. The focused feed-correction checks above are distinguished from those retained results. Native Mono runtime execution is not established by a CoreCLR smoke test.
Measured differences retained for follow-up
This PR intentionally records the unresolved differences rather than claiming the update is SAFE or regression-free. Further attribution is deferred; the following is a narrow, configuration-matched diagnostic, not full-profile/global validation.
Exactly three original whole partitions were compared using the same retained runtime/SDK/workload cohort: SDK
12.0.100-alpha.1.26471.108, WASM pack12.0.0-ci, V815.6.75, and identical runtime identity, original 15-way partition assignment, modes and measurement settings within each comparison. All 1,155 candidate cases produced final statistics, and generated-source/published-child artifacts and publish modes were checked after measurement.These are percentiles of the distribution of per-benchmark case comparisons, not benchmark tail latency or statistical significance. Excluded percentage pairs have nonpositive denominators; they are not missing candidate workloads. No numerical noise-floor subtraction was performed.
The affected composite partition still has 208 cases more than 5% slower and 77 more than 5% faster than the matched 595 reference. Its median delta versus the captured 641 reference is approximately -0.187871%; the newer version does not remove this composite shift. The stable partition's median remains near zero.
For
System.Collections.Sort<IntClass>.Array_Comparison(Size: 512)in plain P5, 662 completed with a final mean of 4.9361 ms, versus 4.8184 ms in the retained 595 recovery reference, approximately +2.44%. The retained 641 attempt has partial Actual measurements but no final statistics. It must not be presented as a formal 47% regression fix or a formal final-statistics comparison against 641.The actual plain child retained benchmark NoInlining and generated-caller flags. Its native/runtime/timing payload bytes match the retained references and its runtimeconfig properties match 641. Those observations do not establish full generated/native-code equivalence or a causal explanation.
CPU model, OS/kernel and timer metadata were checked within this cohort. Actual CPU frequency, affinity and governor state were not established. These results neither attribute an underlying runtime regression nor establish global compatibility.
Separate SDK integration gap
The retained
12.0.100-alpha.1.26471.108cohort does not clear the separate composite generated-projectNETSDK1112failure observed with SDK12.0.100-alpha.1.26528.120targetingnet11.0. Runtime-pack selection/staging across that SDK/TFM rollover still requires coherent-version validation, including a real final publish. No SDK/TFM retargeting or fix for that integration gap is included here.Only a sanitized summary is included here. Private infrastructure identifiers, endpoints, raw captures and internal logs are not attached.