Repository navigation
Description
Summary
With WORKFLOW=decoder_ablation, ONNX_WORKFLOW=3 never uses the TensorRT engine, and ONNX_WORKFLOW=2 never uses the engine it builds. Both fall back to PyTorch.
_setup_trt_for_ablation in code/evaluation/failure_analysis.py calls _time.perf_counter() on both paths, but _time is only imported inside decoder_ablation_study (import time as _time), not in this helper or at module level. The NameError is caught by the except Exception around each path, so the run goes on with PyTorch and prints:
[Ablation] TRT load failed: name '_time' is not defined; using PyTorch.
[Decoder Ablation] Pre-decoder backend: PyTorch (ONNX_WORKFLOW=3)
For ONNX_WORKFLOW=2 the ONNX file is exported, then _time fails just before the engine build, so no engine is built or saved.
The mock TensorRT tests in code/tests/test_failure_analysis.py (TestDecoderAblationStudyTRTExecution, TestDecoderAblationStudyExportAndBuildTRT) do not catch this. Their mock _Logger takes no arguments, so trt.Logger(trt.Logger.WARNING) fails first, and the tests only check the result structure, which the PyTorch fallback also returns. With pytest -s they print TRT load failed: _Logger() takes no arguments; using PyTorch.
Steps to reproduce
Run from code/ with ONNX_WORKFLOW=3 python - < repro.py. It uses a stand-in tensorrt module and a dummy engine file, so no GPU is needed.
import os, sys, tempfile, types
from types import SimpleNamespace as NS
import torch
import evaluation.failure_analysis as fa
class _Engine:
def create_execution_context(self):
return "ctx"
trt = types.ModuleType("tensorrt")
trt.Logger = type("Logger", (), {"WARNING": 0, "__init__": lambda self, level: None})
trt.Runtime = type("Runtime", (), {
"__init__": lambda self, logger: None,
"deserialize_cuda_engine": lambda self, data: _Engine(),
})
sys.modules["tensorrt"] = trt
# Skip building the pre-decoder module; it is not needed to reach the engine load.
class _Module(torch.nn.Module):
def __init__(self, *args, **kwargs):
super().__init__()
def to(self, device):
return self
fa.PreDecoderMemoryEvalModule = _Module
fa._build_stab_maps = lambda D, rot: {}
cfg = NS(data=NS(), n_rounds=3, test=NS(n_rounds=3, dataloader=NS(batch_size=8)))
dist = NS(rank=0)
device = NS(type="cuda")
os.chdir(tempfile.mkdtemp())
open("predecoder_memory_d3_T3_X.engine", "wb").write(b"engine")
ctx, wf = fa._setup_trt_for_ablation(None, cfg, dist, device, "X", 3, 4, None)
print("workflow:", wf.name, "| trt_context:", ctx)Output on main (4e5ef40):
[Ablation] TRT load failed: name '_time' is not defined; using PyTorch.
workflow: USE_ENGINE_ONLY | trt_context: None
Expected behavior
The engine is loaded and trt_context is set, as the local_run.sh comments describe (ONNX_WORKFLOW=3 WORKFLOW=decoder_ablation bash code/scripts/local_run.sh # load existing engine, then ablation).
How I found it
ruff check --select F821 on the repo reported _time as undefined in this function. I then ran the mock TensorRT tests with -s to see which backend they used.
Where this can show up
Anyone who runs the decoder ablation with a TensorRT pre-decoder. The results are still correct, since they come from PyTorch, but they are not TensorRT results, and runs take the PyTorch time. The only hint is the "TRT load failed" line. My estimate; I have not seen a field report.
Environment
- macOS arm64, CPU only, Python 3.11, torch 2.15.0
- Ising-Decoding
mainat 4e5ef40
I have a small fix (import time in the helper, fix the mock _Logger, and check in the tests that the engine ran) and can open a PR.
Activity