Skip to content

fix(spark): parse_url uses one error for every invalid url - #340

Open
TechWizard9999 wants to merge 6 commits into
apache:mainfrom
TechWizard9999:fix/parse-url-error
Open

TechWizard9999 wants to merge 6 commits into
apache:mainfrom
TechWizard9999:fix/parse-url-error

Conversation

@TechWizard9999

Copy link
Copy Markdown
Contributor

Which issue does this PR close?

Rationale for this change

A bad port gave a raw error name instead of the invalid url message.

What changes are included in this PR?

Use the invalid url error for every parse failure. the tests now use a url that spark also rejects.

Are these changes tested?

Yes, updated the slt tests.

Are there any user-facing changes?

Yes, every invalid url gives the same error in ansi mode.

@TechWizard9999

Copy link
Copy Markdown
Contributor Author

@andygrove Ready for review. Thanks.

@andygrove andygrove left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for picking this up. Using one The url is invalid message for every parse failure reads much better, and the test no longer depends on a Debug string. I left one comment about Spark parity for the port case, and a small formatting note.

@TechWizard9999

Copy link
Copy Markdown
Contributor Author

Hi @andygrove , Have made changes could you check it once ?

@andygrove andygrove left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the update. Using invalid_url_err on both parse-failure paths looks good, and the .slt files no longer expect any raw error names. Keeping the :99999 case as a commented record with Spark's value is what I was after. The two blank lines are fixed too.

Locally at 3f083a1b, the url .slt files, the parse_url unit tests and cargo fmt --check all pass. I have approved the workflow runs.

Two things before this can merge. I left a comment about linking the tracking issue. Also, the branch now conflicts with main in crates/sqllogictest/test_files/url/parse_url.slt. Could you merge or rebase main and resolve it? I'm happy to look again after that.

# Conflicts:
#	crates/sqllogictest/test_files/url/parse_url.slt
@TechWizard9999

Copy link
Copy Markdown
Contributor Author

@andygrove Linked #365 and merged main. Thanks.

@andygrove andygrove left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the quick turnaround. The code change still looks good to me, and the url .slt files pass locally with main merged in, in both constant-folding modes. The only thing left is the :99999 records that the merge dropped. I left details in the thread.

@andygrove andygrove left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for restoring the :99999 records with the xfail markers. The url .slt files pass locally, and the PySpark runner reports 80 passes and 2 expected failures for #365. I've approved the workflow runs again.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants