Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions .changeset/expo-verify-cloud-ready-routes.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
---
---
15 changes: 8 additions & 7 deletions .claude/skills/verify-clerk-expo/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -253,7 +253,7 @@ When this machine cannot attach and there is no session to hand the files to, or

Work in this order, with one `up` for both runs: make the branch and `git push` it, `up`, `run` the spec without your change when it changes something that exists, make the change, commit, `git push`, `run` the spec again, open the pull request as a draft with `gh pr create --draft`, `attach` with the before run as `--before <run-id>`, `down`, then `gh pr ready <n>` once the description holds the evidence tables. Open the pull request after the last run, so that it is not left open without its evidence. Open it as a draft because review bots such as CodeRabbit review a pull request when it opens or turns ready, and judge the description they read then. A pull request opened ready is reviewed before `attach` writes the evidence. Commit before the second `run`, so that the line `attach` writes names a commit the pull request holds. In the pull request, name the run id and the specs you ran, and say what you did not run.

On a local device `attach` writes the description itself, and no run needs a push. A remote session builds only the commit it was started on and later commits of the branch, so its before run comes before the push of the change. Its evidence reaches the description about a minute after `down`, so check that `gh pr view <n> --json body` shows the `verify-evidence` block with its table, not the list of files that the first of its two writes leaves, before `gh pr ready <n>`.
On a local device `attach` writes the description itself, and no run needs a push. A remote session builds only the commit it was started on and later commits of the branch, so its before run comes before the push of the change. Its evidence reaches the description about a minute after `down`, so check that `gh pr view <n> --json body` shows the `verify-evidence` block with its table, not the list of files that the first of its two writes leaves, before `gh pr ready <n>`. Its table has a `| :---:` row, and the list does not. In a Claude Code cloud session GitHub's GraphQL API is blocked, so `gh pr create`, `gh pr view`, and `gh pr ready` fail there. Open the draft with `gh api repos/clerk/javascript/pulls -f title='<title>' -f head=<branch> -f base=main -F draft=true -F body=@<file>`, read the description with `gh api repos/clerk/javascript/pulls/<n> --jq .body`, and mark it ready with `gh api -X POST repos/clerk/javascript/pulls/<n>/ccr/ready_for_review`.
Comment thread
mikepitre marked this conversation as resolved.
Resolved

When the change is already committed or pushed, make the before run on its base commit in this worktree, so the run lands in this checkout's `.verify/runs/`. A spec in `specs/explored/` stays through the switch, because that directory is gitignored. A committed spec of the change is not on the base, so copy it to `specs/explored/` first and change its import to `'../fixtures.ts'`.

Expand All @@ -268,17 +268,18 @@ On a remote device, the before run needs a session of its own, started on the ba
```console
$ git switch -c <branch>-before "$(git merge-base origin/main HEAD)"
$ git push origin <branch>-before
$ integration/expo-native/bin/control-clerk-expo up --platform ios --backend remote
$ integration/expo-native/bin/control-clerk-expo run specs/explored/<name>.e2e.ts --platform ios
$ integration/expo-native/bin/control-clerk-expo up --platform <platform> --backend remote
$ integration/expo-native/bin/control-clerk-expo run specs/explored/<name>.e2e.ts --platform <platform>
$ integration/expo-native/bin/control-clerk-expo down
$ git switch <branch>
$ integration/expo-native/bin/control-clerk-expo up --platform ios --backend remote
$ integration/expo-native/bin/control-clerk-expo run specs/explored/<name>.e2e.ts --platform ios
$ integration/expo-native/bin/control-clerk-expo up --platform <platform> --backend remote
$ integration/expo-native/bin/control-clerk-expo run specs/explored/<name>.e2e.ts --platform <platform>
$ gh pr create --draft
$ # in a Claude Code cloud session, open the draft with gh api repos/clerk/javascript/pulls -f title='<title>' -f head=<branch> -f base=main -F draft=true -F body=@<file> instead
$ integration/expo-native/bin/control-clerk-expo attach <run-id> --pr <n> --screenshot <label> --before <before-run-id>
$ integration/expo-native/bin/control-clerk-expo down
$ gh pr view <n> --json body # wait for the verify-evidence table, about a minute after down
$ gh pr ready <n>
$ sh -c 'for i in $(seq 40); do gh api repos/clerk/javascript/pulls/<n> --jq .body | sed -n "/<!-- verify-evidence:<platform> -->/,/<!-- \/verify-evidence:<platform> -->/p" | grep -q "| :---:" && exit 0; sleep 15; done; exit 1' && gh pr ready <n>
$ # in a Claude Code cloud session, end the line above with && gh api -X POST repos/clerk/javascript/pulls/<n>/ccr/ready_for_review in place of gh pr ready
$ git push origin --delete <branch>-before && git branch -D <branch>-before
```

Expand Down
2 changes: 1 addition & 1 deletion .claude/skills/verify-clerk-expo/references/remote.md
Original file line number Diff line number Diff line change
Expand Up @@ -114,7 +114,7 @@ The workflow publishes only when all of these hold. Otherwise it prints the reas

`attach` checks the same rules before it sends anything, and fails with the reason.

So one session does both runs: push the branch, `up`, `run` for the before run, push the change, `run` again, open the pull request with `gh pr create --draft`, `attach` with `--before <run-id>`, `down`, then `gh pr ready <n>` once the description shows the evidence table. Open the pull request after the last run, so that it is not left open without its evidence, which reaches the description about a minute after `down`. It stays a draft until then so that review bots read the description with the evidence in it.
So one session does both runs: push the branch, `up`, `run` for the before run, push the change, `run` again, open the pull request with `gh pr create --draft`, `attach` with `--before <run-id>`, `down`, then `gh pr ready <n>` once the description shows the evidence table. In a Claude Code cloud session, use the REST forms that `SKILL.md` gives for those `gh pr` commands. Open the pull request after the last run, so that it is not left open without its evidence, which reaches the description about a minute after `down`. It stays a draft until then so that review bots read the description with the evidence in it.

A before run is the build without the change, so its commit is often the one the branch started from, which is no commit of the pull request. The workflow therefore does not look for that commit among the commits of the pull request. It asks GitHub whether that commit is the commit of the run or a commit before it in its history, and it publishes nothing when the answer is no. After a rebase that rewrote the commit of the before run, `run` the spec again on a commit that the branch still has, and pass that run as `--before`. The workflow cannot check that the files of the before run came from a build of that commit, which is why the line says that the session reports the before run.

Expand Down
Loading