Skip to content

Import time in _setup_trt_for_ablation so the TRT paths run - #107

Open
0z5a wants to merge 1 commit into
NVIDIA:mainfrom
0z5a:ablation-trt-time
Open

0z5a wants to merge 1 commit into
NVIDIA:mainfrom
0z5a:ablation-trt-time

Conversation

@0z5a

@0z5a 0z5a commented Nov 20, 2026

Copy link
Copy Markdown
Contributor

Summary

Refs #106.

_setup_trt_for_ablation now imports time as _time itself, so ONNX_WORKFLOW=3 loads the TensorRT engine and ONNX_WORKFLOW=2 builds and uses one in the decoder ablation, instead of falling back to PyTorch on NameError: name '_time' is not defined.

The mock TensorRT tests now reach the TensorRT path and check that it ran.

Changes

  • code/evaluation/failure_analysis.py: add import time as _time at the top of _setup_trt_for_ablation. This is the same local import decoder_ablation_study already uses.
  • code/tests/test_failure_analysis.py:
    • The mock _Logger takes the severity argument, as trt.Logger(trt.Logger.WARNING) passes one. Before, this failed first, so the TensorRT tests ran on the PyTorch fallback.
    • The mock counts execute_v2 calls. New tests test_trt_path_runs_the_engine and test_export_and_build_runs_the_engine check that the count is above zero. The fallback returns a well-formed result, so the existing structure checks could not tell the two apart.

How I found it

ruff check --select F821 reported _time as undefined in _setup_trt_for_ablation. Running the TensorRT tests with -s showed they used the PyTorch backend.

Where this can show up in practice

Decoder ablation runs with a TensorRT pre-decoder (ONNX_WORKFLOW=2 or 3 with WORKFLOW=decoder_ablation, as in the local_run.sh comments). Before this change they report PyTorch results and take PyTorch time, with only a "TRT load failed" line in the log. My estimate; I have not seen a field report.

Testing

On CPU, macOS arm64, Python 3.11, torch 2.15.0:

I did not run it with a real TensorRT engine on a GPU.

_setup_trt_for_ablation called _time.perf_counter() without importing
time, so ONNX_WORKFLOW=2 and 3 in the decoder ablation hit a NameError,
which the except blocks turned into a silent PyTorch fallback.

The mock TensorRT tests did not catch it: the mock Logger took no
arguments, so they also fell back. Let the mock Logger take the
severity, count execute_v2 calls, and check that the engine ran.

Refs NVIDIA#106

Signed-off-by: Dezhen Lu <dezhen.lu@student.uni-tuebingen.de>
@copy-pr-bot

copy-pr-bot Bot commented Nov 20, 2026

Copy link
Copy Markdown

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant