Skip to content

Initial Driver System - #1335

Merged
jlarson4 merged 25 commits into
dev-4.xfrom
feature/driver-system
May 27, 2026
Merged

jlarson4 merged 25 commits into
dev-4.xfrom
feature/driver-system

Conversation

@jlarson4

Copy link
Copy Markdown
Collaborator

Description

Initial setup of the Driver system. The goal of this new system is to make the source for the model agnostic, so we can use a multitude of different sources.

The system is made up of several new parts:

  • DriverProtocol – This defines what every driver must satisfy
  • BridgeCore – Defines framework agnostic components of all bridges. Owns the hook registry, alias machinery, run_with_cache/run_with_hooks, _replay_captures, and _finalize_return.
    • TransformerBridge(BridgeCore, HookIntrospectionMixin, nn.Module) – torch-backed. Carries the torch-specific surface: parameters(), state_dict(), generate(), weight-access properties (W_Q etc.), compatibility_mode. Constructor falls back to building a TransformersDriver if none provided.
    • RemoteBridge(BridgeCore, HookIntrospectionMixin) - non-torch. No nn.Module parentage. Requires driver.supported_hook_points non-empty at construction; cfg.device is None; forward() routes through driver.forward() only.
  • model_bridge/sources/transformers.py split up into a package. Splits out functionality that is used by multiple bridges, allows RemoteBridge to import only the things it needs

There are two Drivers at present:

  • TransformersDriver – Handles the current 3.x standard HF/torch system
  • VLLMDriver – Handles running a RemoteBridge for a vLLM backend.

Also added a Colab notebook. it is a 7-step end-to-end integration test (HF reference → vLLM boot → capture pipeline → greedy parity → per-hook L2 acceptance → intervention smoke → lifetime). Validated on Llama-3.2-1B (fp16), Qwen2.5-0.5B (fp16, T4), Gemma-2-2B (bf16, requires A100/L4).

Type of change

  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)

Checklist:

  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation
  • My changes generate no new warnings
  • I have added tests that prove my fix is effective or that my feature works
  • New and existing unit tests pass locally with my changes
  • I have not rewritten tests relating to key interfaces which would affect backward compatibility

jlarson4 and others added 24 commits May 14, 2026 12:57
Cherry-picked from 98bbb835 on feature/vllm-integration. Resolved against
dev-4.x's Phase A type split:
- bridge.py: kept the shim (TransformerBridge content lives in
  transformer_bridge.py now).
- sources/transformers.py: accepted the scaffold's package restructure
  (transformers.py → transformers/{__init__,helpers,source}.py +
  _hf_format.py).
- sources/transformers/source.py: re-applied Phase A's explicit
  TransformersDriver construction in boot().

sources/vllm/source.py's boot_vllm currently sets bridge._forward_impl,
which Phase A removed. The function is dead until the next chunk lifts
it onto VLLMDriver + RemoteBridge — landing as-is so the scaffold tree is
on dev-4.x for incremental Phase B work.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@jlarson4 jlarson4 changed the title Feature/driver system Initial Driver System May 27, 2026
@jlarson4
jlarson4 merged commit de7531b into dev-4.x May 27, 2026
24 checks passed
@jlarson4 jlarson4 added the v4.x label Jun 6, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant