[qti-horodnic]

Summary

This PR is the second of Phase 2 Stream B and completes the compilation stage. B1 lowered one graph to one .pte. B2 adds the multi-graph and multi-component path:

• several graphs into one weight-sharing .pte;
• per-component compile specs and ExecutorchBackendConfig;
• derivation of the pass configuration;
• compilation of unquantized models (FP16 / GPU) through the same path.

Nothing under examples/ changes. The legacy llama.py flow remains CI's reference until Phase 3, which is gated on PR-A2's parity tests.

What's included

Group lowering (default_compiler_adapter.py)

• compile_graph_group(): one to_edge_transform_and_lower_to_qnn call with graph-name-keyed dicts for module / inputs / compiler_specs, producing one multi-method .pte.
• Prefill and decode lower together because the runtime shares a single KVManager and RPC buffer between them with no requantization, so their KV scales must match. Weight sharing within one artifact guarantees that.
• _validate_group() runs before any graph work. It rejects an empty group, a per-graph mapping (compile_specs, dep_table, passes_job) whose keys differ from the graph names, and a bundle with no inputs.
• _needs_spill_fill() gates update_spill_fill_size on the decoded specs showing HTP and use_multi_contexts. The specs set use_multi_contexts only when the resolved shard count is above 1, so this is the legacy num_sharding > 1 gate.
• compile_model() and compile_graph_group() take a required executorch_config. The strategy chooses it per component; the adapter does not choose a config.

LLM specialisation (llm_compiler_adapter.py, new)

• LLMCompilerAdapter(DefaultCompilerAdapter), three overrides:
  • _with_llm_skip_ops unions llama.fallback.default into any caller-supplied skip_node_op_set. It marks the sharding boundary, so a caller's own skip set must not drop it.
  • _decode_constant_methods defaults constant_methods to the decode graph's meta, looked up by GRAPH_KV_FORWARD rather than by position.
  • _on_lowered prints per-graph delegation info when verbose.

Component spec policy (compilation/component_spec_policy.py, new)

• ComponentSpecPolicy + get_component_policy() + build_component_compile_specs(), reproducing legacy's generate_htp_compiler_spec call sites as a table keyed by artifact key. The decoder sets use_mha2sha=True and honours sharding; tok_embedding takes defaults; encoders disable weight sharing and ignore online_prepare.
• use_fp16 is per component: a component compiles in FP16 if and only if its graphs were not quantized (GraphBundle.quantized). This matches legacy, where FP16 follows skip_quantize per component.
• build_executorch_config() builds each component's ExecutorchBackendConfig from ComponentSpecPolicy.uses_shared_buffer:
  • The decoder, tok_embedding and the attention-sink evictor leave graph IO unallocated (alloc_graph_input/output=False), because the runner supplies RPC memory.
  • Encoders use default memory planning, as in legacy Modality.compile.
  • All components run BuildQuantIo.
  • Before this PR, every component got the RPC config, which was wrong for encoders.

Pass policy (compilation/pass_policy.py, new) and model layout (compilation/model_layout.py, new)

• PassPolicy table + get_pass_policy() + build_pass_config() + build_component_pass_config(). Derives per component the base passes, SplitGraph plus both dependency edges (SplitGraph←FoldQDQ, TagQuantIO←SplitGraph), the I64toI32 tokens skip, and TagQuantIO activation with both _tag_ios equivalents transcribed rule-for-rule from legacy.
• Compilation derives this rather than receiving it on the bundle (agreed with @DannyYuyang-quic). Every mutation is computable from meta, ControlArgs and the shard count.
• resolve_num_sharding() checks the requested count against get_n_layers (legacy llm_wrappers.py L1329). The strategy resolves it once per component and uses the same value for the specs and the passes. Above the layer count, it clamps with a WARNING. Legacy instead disables the split but still requests multi-contexts; here specs and passes cannot disagree.
• Model-specific data lives in model_layout.py, not in pass_policy.py: ModelLayout (decoder/encoder layer prefixes, prefix offsets from meta, extra head dim, gemma4 stack-trace and donor-K/V tagging rules) and a registry (get_model_layout, register_model_layout). pass_policy.py has no model-name conditionals. A new model, including HF module paths (model.layers), registers a layout only when it differs from the default.
• build_component_pass_config() logs one INFO line per component with the model name, the shard count and the resolved ModelLayout, so users can see which layout patterns the run applies.
• TagQuantIO dtypes come from the quantization recipe via GraphBundle.quant_io_dtypes. Decoders carry {kv_type, io_type}; encoders carry {io_type} (uint16, as in Qualcomm AI Engine Direct - Tag multimodal encoder quantized I/O as uint16 #22924). None means quantization was skipped, so TagQuantIO stays inactive.
• The token embedding has no recipe. As in legacy, it is tagged with the decoder decode graph's dtypes and shapes (uses_decoder_boundary). Compilation raises if the decode graph is absent.

Component fan-out (executorch_compilation_strategy.py)

• invoke takes graphs when quantization produced them. When quantization was skipped and model is keyed by component, it builds unquantized bundles (unquantized_bundles.py): drops the calibration graphs and flattens the decoder inputs. Both cases then use _compile_components. _compile_single_graph remains for a true one-graph model. Result: LLM and MLLM on GPU / FP16 get one .pte per component, as in legacy.
• Per component it resolves the shard count, specs, pass config and ExecutorchBackendConfig, calls compile_graph_group once, and merges the artifact paths. It rejects an unknown artifact key, an empty component, and a component that mixes quantized and unquantized graphs.
• The first non-None ETRecord is kept, and an INFO log names its component.

Shared graph helpers (new, used by quantization and compilation)

• graph_roles.py: GraphRole, GRAPH_ROLES, get_graph_role(), is_deployed(). Moved from A1's quantization strategy.
• export_inputs.py: to_export_inputs(), flattening the nested decoder inputs into the export signature. Moved from A1's quantization strategy.

Quantization strategy changes (executorch_quantization_strategy.py, from A1)

• Uses the shared helpers above. No behaviour change from the move.
• Fix: GraphBundle.inputs now holds the flattened export signature the graph was exported with, not the nested decoder inputs. Lowering re-exports with bundle.inputs.
• Per-component skip: quantize_options["skip_quantize"] leaves encoders (components whose graphs are all SHARED) unquantized. A skipped encoder is exported but not prepared, still runs in float during MLLM calibration, and is bundled with quantized=False. Skipping the decoder this way raises; the decoder is skipped by skipping the stage (use_fp16).

Config surface

• GraphBundle.quantized: bool = True. An unquantized bundle cannot carry quant_io_dtypes.
• Removed GraphBundle.executorch_config and the executorch_backend_config extra option (both from B1). build_executorch_config() replaces them.
• CompilerAdapter.compile_model / compile_graph_group: new required executorch_config parameter.
• CompilationInputConfig.graphs: Optional[Dict[str, Dict[str, GraphBundle]]]: artifact key, then graph name.
• CompilationInputConfig.num_sharding: Union[int, Dict[str, int]] = 1: per component. The orchestrator forwards ModelPreparationOutputConfig.num_shardings.
• CompilationInputConfig.meta: used to build unquantized bundles when quantization is skipped. Forwarded by the orchestrator.
• compile_spec_builder.py: docstring changes only (shared_buffer must follow enable_x86_64, not the runner flag ControlArgs.shared_buffer).

Deliberately deferred

• GraniteSpeech's int32 attention-distance tagging is gated on artifact_key == ARTIFACT_AUDIO_ENCODER rather than on the encoder config class as legacy does. Correct while GraniteSpeech is the only audio encoder.
• CLI wiring of skip_quantize for multi-image input (legacy: skip encoders when len(image_path) > 1). The CLI has no image_path argument yet; this lands in PR-A2.

Unit tests (9 new files, 5 updated)

• New: test_pass_policy.py, test_component_spec_policy.py, test_model_layout.py, test_compile_graph_group.py, test_llm_compiler_adapter.py, test_compilation_strategy_fan_out.py, test_unquantized_bundles.py, test_graph_roles.py, test_export_inputs.py.
• Updated: test_compilation_input_config.py, test_default_compiler_adapter.py (required executorch_config), test_executorch_compilation_strategy.py, test_graph_bundle.py, test_executorch_quantization_strategy.py (bundle inputs are the export signature; encoder skip).

PR Review Checklist

• All new classes follow single responsibility (one class per file) - Yes.
• All dependencies are injected via constructor with sensible defaults - Yes.
• All external calls are behind injectable interfaces - Yes (lowering imported lazily inside the adapter).
• Unit tests cover every public method - Yes.
• Docstrings on all public classes and methods - Yes.
• Type annotations on all function signatures - Yes.
• Logging follows the strategy in the LLD - Yes.
• No existing behavior changed - Partly. No examples/ files are touched. Changes to merged Phase 2 code:
  • The A1 quantization strategy changes as described above (bundle-inputs fix; opt-in encoder skip).
  • Encoders now get default memory planning instead of the RPC config.
  • The adapter protocol adds a required executorch_config argument, and the B1 executorch_config overrides are removed.

Related PRs

Phase 2 flow. PR-A1 and PR-B1 are merged. PR-B2 depends on both. PR-A2 depends on PR-B2, and closes Phase 2 by wiring the runner and adding the parity test that gates the Phase-3 deletion of the legacy flow.

PR-A1 (merged) ──┐
                 ├─► PR-B2 (this PR) ──► PR-A2
PR-B1 (merged) ──┘

Phase 1 (merged):

• PR 1: Core data model, engine routing & exceptions: Qualcomm AI Engine Direct - [GenAI Pipeline] PR1: Core Data Model, Engine Routing & Exceptions #20409
• PR 2: Strategy interfaces & stage wrappers: Qualcomm AI Engine Direct - [GenAI Pipeline] PR2: Strategy Interfaces & Stage Wrappers #20795
• PR 3: Pipeline orchestrator: Qualcomm AI Engine Direct - [GenAI Pipeline] PR3: Pipeline Orchestrator #21149
• PR 4: Adapter interfaces & default implementations: Qualcomm AI Engine Direct - [GenAI Pipeline] PR4: Adapter interfaces, default implementations & dataset providers #21751
• PR 5: Model preparation & quantization strategies: Qualcomm AI Engine Direct - [GenAI Pipeline] PR5: Model preparation & quantization strategy implementations #21899
• PR 6: Compilation & inference strategy implementations: Qualcomm AI Engine Direct - [GenAI Pipeline] PR6: Compilation & inference strategy implementations #22284
• PR 7: Integration & E2E tests: cancelled, will be raised after Phase 2 is done.

Phase 2:

• PR B1: Compilation foundation - merged
• PR A1: Model registry, loader and quantizer adapters, multi-graph quantization - merged
• PR B2: this PR, depends on B1 and A1.
• PR A2: Device-runner adapter, pipeline wiring, multi-image skip_quantize CLI wiring and the llama_stories_260k parity test; depends on B2 - pending

Related upstream: #22924 (encoder quantized IO tagged as uint16) - merged.

Test plan

Run all genai_pipeline tests:

python -m pytest backends/qualcomm/genai_pipeline/tests/ -v

Result:

429 passed, 158 subtests passed in 23.45s

Run all tests with coverage:

python -m pytest backends/qualcomm/genai_pipeline/tests/ \
  --cov=backends/qualcomm/genai_pipeline \
  --cov-config=backends/qualcomm/.coveragerc \
  --cov-report=term-missing

Result for the files this PR adds or changes:

Name                                                          Stmts   Miss Branch BrPart  Cover   Missing
---------------------------------------------------------------------------------------------------------
compilation/compile_spec_builder.py                              58      1     18      0    99%   192
compilation/component_spec_policy.py                             32      2      4      1    92%   106-109
compilation/model_layout.py                                      28      0      4      0   100%
compilation/pass_policy.py                                      193      4     74      8    96%   134-137, 434, ...
configs/compilation_input_config.py                              15      0      0      0   100%
export_inputs.py                                                 11      0      4      0   100%
genai_pipeline.py                                                98      0     12      0   100%
graph_bundle.py                                                  21      0      6      0   100%
graph_roles.py                                                   17      0      4      0   100%
unquantized_bundles.py                                           12      0      4      0   100%
strategies/compilation/compiler_adapter.py                       13      0      0      0   100%
strategies/compilation/executorch_compilation_strategy.py       114      0     32      0   100%
strategies/compilation/llm_compiler_adapter.py                   37      1      6      0    98%   60
strategies/quantization/executorch_quantization_strategy.py     196      9     88     15    92%   ...
---------------------------------------------------------------------------------------------------------
TOTAL                                                           845     17    256     24    96%

The remaining uncovered lines are mostly defensive paths: unrecognized artifact keys, missing graph roles, missing get_use_kv_cache, and the GraniteSpeech attention-distance rule (see Deliberately deferred). The package-wide total (67%) includes A1 modules outside this PR. Coverage will be increased to sufficient levels (>=95%) by the end of phase 2.

Confirm the legacy flow is unaffected:

python -c "
from executorch.examples.qualcomm.oss_scripts.llama.llama import _build_parser, export_llama
from executorch.examples.qualcomm.oss_scripts.llama.wrappers import MultiModalManager, HybridAttentionSinkEvictor
print('LEGACY IMPORTS OK')
"

Result:

LEGACY IMPORTS OK

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/24563

• 📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

⚠️ 16 Awaiting Approval

As of commit 30127f8 with merge base 27de6db ():

AWAITING APPROVAL - The following workflows need approval before CI can run:

• Build documentation (gh)
• Build Presets (gh)
• Cadence Build & Test (gh)
• Lint (gh)
• periodic (gh)
• pull (gh)
• Test ARM Backend (gh)
• Test CoreML Backend (gh)
• Test Cortex-M Backend (gh)
• Test CPU Backend (gh)
• Test NXP Backend (gh)
• Test QNN Backend (gh)
• Test QNN Windows MSVC build (gh)
• Test Vulkan Backend (gh)
• Test WebGPU Backend (gh)
• Test XNNPACK Backend (gh)

This comment was automatically generated by Dr. CI and updates every 15 minutes.

[qti-horodnic]

@pytorchbot label "release notes: qualcomm"