Skip to content

Fix single-turn client GRPO and use StreamingDriver - #402

Closed
Yunnglin wants to merge 2 commits into
dev-swift-v5from
fix/client-grpo-streaming
Closed

Yunnglin wants to merge 2 commits into
dev-swift-v5from
fix/client-grpo-streaming

Conversation

@Yunnglin

Copy link
Copy Markdown
Collaborator

PR type

  • Bug Fix
  • Document Updates

PR information

The single-turn client GRPO examples used the old advantage API, and the streaming math example required a separate DataPlane deployment. The synchronous GSM8K example now uses the current advantage API and shows how to reward short, correct answers. The streaming example uses StreamingDriver and the standard vLLM sampler. Existing DataPlane APIs are unchanged.

Skipped groups no longer block sampling of later groups. On exit, the SDK drains submitted rollout calls before releasing training handles. Updated the English and Chinese guides and the example server config.

Experiment results

  • 51 offline tests passed, covering both tutorials, sampled token/logprob alignment, skipped groups, repeated interrupts, and resource release.
  • Example server config and both documentation builds passed.
  • A preliminary PPU4 run on a precursor branch returned 26 real samples with DataPlane disabled. The sampler replica stopped during generation before any optimizer update. Real updates and sampling from the published LoRA remain to be verified on this PR.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant