Skip to content

ci: Fix running of only changed E2E tests #17551

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
mydea merged 1 commit into from
Sep 8, 2025
Merged

ci: Fix running of only changed E2E tests #17551

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions dev-packages/e2e-tests/lib/getTestMatrix.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -46,7 +46,7 @@ function run(): void {
},
});

const { base, head, optional } = values;
const { base, head = 'HEAD', optional } = values;

const testApplications = globSync('*/package.json', {
cwd: `${__dirname}/../test-applications`,
Expand Down Expand Up @@ -174,7 +174,7 @@ function getAffectedTestApplications(
}

function getChangedTestApps(base: string, head?: string): false | Set<string> {
const changedFiles = execSync(`git diff --name-only ${base}${head ? `..${head}` : ''} -- dev-packages/e2e-tests/`, {
const changedFiles = execSync(`git diff --name-only ${base}${head ? `..${head}` : ''} -- .`, {
encoding: 'utf-8',
Comment on lines +177 to 178

@seer-by-sentry seer-by-sentry Bot Sep 8, 2025

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Potential bug: The git diff scope was broadened, but the file filtering logic was not updated, causing all E2E tests to run unnecessarily for most infrastructure changes.

  • Description: The git diff command in getChangedTestApps was changed to scan for file changes across the entire repository (-- .) instead of being limited to the dev-packages/e2e-tests/ directory. However, the subsequent logic still checks if every changed file starts with the dev-packages/e2e-tests/test-applications/ prefix. If any file outside this specific subdirectory is changed (e.g., a change to the test infrastructure file getTestMatrix.ts itself), the function returns false. This causes the CI pipeline to run all E2E tests, defeating the purpose of the selective test matrix optimization and leading to significantly longer CI execution times.

  • Suggested fix: Revert the path specifier in the git diff command from -- . back to the more specific -- dev-packages/e2e-tests/. This ensures the command only considers file changes within the E2E tests package, which aligns with the existing filtering logic and restores the intended selective test execution.
    severity: 0.65, confidence: 0.98

Did we get this right? 👍 / 👎 to inform future reviews.

})
.toString()
Expand Down