[eugenegujing]

What changes were proposed in this PR?

Root cause. A PythonOperatorDescriptor whose code generation throws replaces its whole code with the line #EXCEPTION DURING CODE GENERATION: <message> (PythonOperatorDescriptor.scala:29), and WorkflowCompiler turns that marker back into a per-operator COMPILATION_ERROR. The compiler's regex was unanchored, so it matched the marker text anywhere in the code of every Python-based operator, including Python and R UDFs whose code is the user's own script. A UDF that mentioned the marker text in a comment or a string was rejected, the physical plan was dropped, and the reported error was the rest of the user's comment line.

Before:  "...\n# NOTE: #EXCEPTION DURING CODE GENERATION: only a comment\n..." -> COMPILATION_ERROR "Operator is not configured properly: only a comment", no physical plan
After:   same UDF                                                          -> compiles, physical plan kept
         "#EXCEPTION DURING CODE GENERATION: <msg>" (whole code)           -> COMPILATION_ERROR as before

Before (local main 92e86d4, workflow Text Input → Python UDF with the NOTE comment on line 3).

The run button reads "Invalid Workflow" and the Static Error tab reports COMPILATION_ERROR: Operator is not configured properly: this is only a comment.

The UDF code editor with the comment on line 3, next to the fabricated Static Error.

Control: a copy of the workflow with the same UDF minus the comment compiles, no Static Error tab, run button reads "Run".

After (this branch, the same workflow with the comment still on line 3).

The commented UDF compiles: run button reads "Run", the result panel has only the Console tab, no Static Error.

The code editor still shows the marker text in the line-3 comment while the run button reads "Run".

Changes.

• WorkflowCompiler.scala: the regex becomes (?s)^#EXCEPTION DURING CODE GENERATION:\s*(.*). ^ restricts the match to the start of the code, which is the only place the marker is ever written; (?s) lets the captured message include line breaks, so a multi-line generation failure is reported in full instead of its first line.
• PythonOperatorDescriptor.scala: an exception whose message is null or empty is rendered as ex.toString (its class name, plus : for the empty case) instead of the literal null or nothing.

Scope. The in-band marker design itself is left as is; moving the failure out of the code string would touch PhysicalOp and every consumer and is out of scope here.

Any related issues, documentation, discussions?

Closes #9213

How was this PR tested?

Added to WorkflowCompilerSpec (Python code-generation error path section):

• a PythonUDFOpDescV2 whose script contains the marker in a comment compiles with no error and keeps its physical plan (the test also asserts the marker text did reach the check);
• a test-only PythonOperatorDescriptor whose generated code mentions the marker mid-code compiles with no error;
• a generation failure with a null message is reported as Operator is not configured properly: java.lang.RuntimeException;
• a multi-line failure message is reported in full, with both \n and \r\n line endings;
• a Python UDF whose code starts with the marker line is still reported as a code-generation error with that message (the intended boundary of the anchor);
• an R UDF (RUDFOpDesc) whose code mentions the marker in a comment compiles with no error and keeps its physical plan.

Added to PythonOperatorDescriptorSpec: the embedded marker for an exception without a message and for one with an empty message (#EXCEPTION DURING CODE GENERATION: java.lang.RuntimeException: ). The existing tests for real generation failures (unconfigured Sort, padded message) are unchanged and still pass.

sbt "WorkflowOperator/testOnly org.apache.texera.amber.operator.PythonOperatorDescriptorSpec" "WorkflowCompiler/testOnly org.apache.texera.common.compiler.WorkflowCompilerSpec"

• Red (first batch of tests against unmodified source): PythonOperatorDescriptorSpec 3 passed, 1 failed; WorkflowCompilerSpec 34 passed, 4 failed, e.g. the UDF case produced Operator is not configured properly: this is only a comment with physicalPlan = None.
• Green (with this change): PythonOperatorDescriptorSpec 5 passed, WorkflowCompilerSpec 41 passed, 0 failed (2 and 7 new tests). The empty-message test was red first (4 passed, 1 failed) before the filter(_.nonEmpty) fallback was added.
• Mutation check (full suites): reverting both source lines fails 8 of the 9 new tests (PythonOperatorDescriptorSpec 3 passed / 2 failed, WorkflowCompilerSpec 35 passed / 6 failed); the UDF-starts-with-the-marker test passes either way by design. Removing only the filter(_.nonEmpty) fails exactly the empty-message test (4 passed / 1 failed, compiler spec 41 / 0). Restored afterwards.
• scalafmtCheck, Test/scalafmtCheck, scalafix --check and Test/scalafix --check on WorkflowCompiler and WorkflowOperator: all pass.

Was this PR authored or co-authored using generative AI tooling?

Yes. Generated-by: Claude Code (claude-fable-5-1). Co-authored with Claude; the author reviewed the change before submission.

[github-actions]

Automated Reviewer Suggestions

Based on the git blame history of the changed files, we recommend the following reviewers:

• Contributors with relevant context: @aglinxinyuan, @Yicong-Huang
  You can notify them by mentioning @aglinxinyuan, @Yicong-Huang in a comment.

[github-actions]

Backport auto-label report

This fix: PR was checked against each actively-supported release branch. A release/* label nominates a backport target; the branch's release manager approving this PR is what sends the fix there. The required Backport Approvals check stays red until every label below is approved, so each manager either approves or removes their own label — which is why the labels left on a merged PR are exactly the branches it reached.

Release branch	Analysis
✅ release/v1.3	Already labeled — this fix is queued to backport here. @mengw15 decides: approving sends the fix here, removing this label declines it. The merge waits on one or the other.
✅ release/v1.2	Already labeled — this fix is queued to backport here. @xuang7 decides: approving sends the fix here, removing this label declines it. The merge waits on one or the other.

Auto-label run.

[codecov-commenter]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 92.05%. Comparing base (92e86d4) to head (f4ef3da).
⚠️ Report is 2 commits behind head on main.
✅ All tests successful. No failed tests found.

Additional details and impacted files

@@             Coverage Diff              @@
##               main    #9215      +/-   ##
============================================
+ Coverage     91.98%   92.05%   +0.06%     
+ Complexity     5083     5040      -43     
============================================
  Files          1285     1286       +1     
  Lines         56094    55911     -183     
  Branches       7137     7137              
============================================
- Hits          51599    51467     -132     
+ Misses         2742     2693      -49     
+ Partials       1753     1751       -2     

Flag	Coverage Δ		*Carryforward flag
access-control-service	77.44% <ø> (ø)
agent-service	99.16% <ø> (ø)		Carriedforward from 92e86d4
amber	86.56% <100.00%> (+0.13%)	⬆️
computing-unit-managing-service	64.02% <ø> (ø)
config-service	87.61% <ø> (ø)
file-service	82.29% <ø> (ø)
frontend	96.47% <ø> (ø)		Carriedforward from 92e86d4
notebook-migration-service	83.77% <ø> (ø)
pyamber	98.54% <ø> (ø)		Carriedforward from 92e86d4
workflow-compiling-service	74.25% <ø> (ø)

*This pull request uses carry forward flags. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[github-actions]

⚠️ Benchmark changes need a look

🟢 0 better · 🔴 7 worse · ⚪ 8 noise (<±5%) · 0 without baseline

Compared against main f673554 benchmarked on this same runner, so the delta is largely free of cross-runner hardware noise. The "7d avg" column still reflects the gh-pages dashboard. Treat <±5% as noise unless repeated.

Dashboard · Run

	config	throughput	MB/s	latency	max Δ latest / 7d
🔴	bs=10 sw=10 sl=64	534	0.326	17,600/25,604/25,604 us	🔴 +20.5% / 🔴 +82.5%
⚪	bs=100 sw=10 sl=64	1,203	0.734	82,381/111,734/111,734 us	⚪ within ±5% / 🔴 +16.1%
🔴	bs=1000 sw=10 sl=64	1,451	0.885	675,201/803,224/803,224 us	🔴 +5.5% / 🟢 -23.1%

Baseline details

Latest main f673554 from same runner

config	metric	PR	latest main	7d avg	Δ latest	Δ 7d
bs=10 sw=10 sl=64	throughput	534 tuples/sec	631 tuples/sec	905.73 tuples/sec	-15.4%	-41.0%
bs=10 sw=10 sl=64	MB/s	0.326 MB/s	0.385 MB/s	0.553 MB/s	-15.3%	-41.0%
bs=10 sw=10 sl=64	p50	17,600 us	14,830 us	11,331 us	+18.7%	+55.3%
bs=10 sw=10 sl=64	p95	25,604 us	21,254 us	14,032 us	+20.5%	+82.5%
bs=10 sw=10 sl=64	p99	25,604 us	21,254 us	17,338 us	+20.5%	+47.7%
bs=100 sw=10 sl=64	throughput	1,203 tuples/sec	1,217 tuples/sec	1,205 tuples/sec	-1.2%	-0.1%
bs=100 sw=10 sl=64	MB/s	0.734 MB/s	0.743 MB/s	0.735 MB/s	-1.2%	-0.2%
bs=100 sw=10 sl=64	p50	82,381 us	79,622 us	89,237 us	+3.5%	-7.7%
bs=100 sw=10 sl=64	p95	111,734 us	112,982 us	96,280 us	-1.1%	+16.1%
bs=100 sw=10 sl=64	p99	111,734 us	112,982 us	111,344 us	-1.1%	+0.4%
bs=1000 sw=10 sl=64	throughput	1,451 tuples/sec	1,441 tuples/sec	1,239 tuples/sec	+0.7%	+17.1%
bs=1000 sw=10 sl=64	MB/s	0.885 MB/s	0.88 MB/s	0.756 MB/s	+0.6%	+17.1%
bs=1000 sw=10 sl=64	p50	675,201 us	685,007 us	877,826 us	-1.4%	-23.1%
bs=1000 sw=10 sl=64	p95	803,224 us	761,128 us	919,412 us	+5.5%	-12.6%
bs=1000 sw=10 sl=64	p99	803,224 us	761,128 us	963,000 us	+5.5%	-16.6%

Raw CSV

config_idx,batch_size,schema_width,string_len,num_batches,total_ms,total_tuples,total_bytes,tuples_per_sec,mb_per_sec,lat_p50_us,lat_p95_us,lat_p99_us
0,10,10,64,20,374.72,200,128000,534,0.326,17600.37,25603.95,25603.95
1,100,10,64,20,1663.16,2000,1280000,1203,0.734,82381.05,111734.27,111734.27
2,1000,10,64,20,13787.72,20000,12800000,1451,0.885,675201.38,803223.61,803223.61