Skip to content

Decoder ablation falls back to PyTorch with ONNX_WORKFLOW=2 or 3 (_time not defined in _setup_trt_for_ablation) #106

Open

Description

@0z5a

Summary

With WORKFLOW=decoder_ablation, ONNX_WORKFLOW=3 never uses the TensorRT engine, and ONNX_WORKFLOW=2 never uses the engine it builds. Both fall back to PyTorch.

_setup_trt_for_ablation in code/evaluation/failure_analysis.py calls _time.perf_counter() on both paths, but _time is only imported inside decoder_ablation_study (import time as _time), not in this helper or at module level. The NameError is caught by the except Exception around each path, so the run goes on with PyTorch and prints:

[Ablation] TRT load failed: name '_time' is not defined; using PyTorch.
[Decoder Ablation] Pre-decoder backend: PyTorch (ONNX_WORKFLOW=3)

For ONNX_WORKFLOW=2 the ONNX file is exported, then _time fails just before the engine build, so no engine is built or saved.

The mock TensorRT tests in code/tests/test_failure_analysis.py (TestDecoderAblationStudyTRTExecution, TestDecoderAblationStudyExportAndBuildTRT) do not catch this. Their mock _Logger takes no arguments, so trt.Logger(trt.Logger.WARNING) fails first, and the tests only check the result structure, which the PyTorch fallback also returns. With pytest -s they print TRT load failed: _Logger() takes no arguments; using PyTorch.

Steps to reproduce

Run from code/ with ONNX_WORKFLOW=3 python - < repro.py. It uses a stand-in tensorrt module and a dummy engine file, so no GPU is needed.

import os, sys, tempfile, types
from types import SimpleNamespace as NS

import torch
import evaluation.failure_analysis as fa

class _Engine:
    def create_execution_context(self):
        return "ctx"

trt = types.ModuleType("tensorrt")
trt.Logger = type("Logger", (), {"WARNING": 0, "__init__": lambda self, level: None})
trt.Runtime = type("Runtime", (), {
    "__init__": lambda self, logger: None,
    "deserialize_cuda_engine": lambda self, data: _Engine(),
})
sys.modules["tensorrt"] = trt

# Skip building the pre-decoder module; it is not needed to reach the engine load.
class _Module(torch.nn.Module):
    def __init__(self, *args, **kwargs):
        super().__init__()
    def to(self, device):
        return self
fa.PreDecoderMemoryEvalModule = _Module
fa._build_stab_maps = lambda D, rot: {}

cfg = NS(data=NS(), n_rounds=3, test=NS(n_rounds=3, dataloader=NS(batch_size=8)))
dist = NS(rank=0)
device = NS(type="cuda")

os.chdir(tempfile.mkdtemp())
open("predecoder_memory_d3_T3_X.engine", "wb").write(b"engine")

ctx, wf = fa._setup_trt_for_ablation(None, cfg, dist, device, "X", 3, 4, None)
print("workflow:", wf.name, "| trt_context:", ctx)

Output on main (4e5ef40):

[Ablation] TRT load failed: name '_time' is not defined; using PyTorch.
workflow: USE_ENGINE_ONLY | trt_context: None

Expected behavior

The engine is loaded and trt_context is set, as the local_run.sh comments describe (ONNX_WORKFLOW=3 WORKFLOW=decoder_ablation bash code/scripts/local_run.sh # load existing engine, then ablation).

How I found it

ruff check --select F821 on the repo reported _time as undefined in this function. I then ran the mock TensorRT tests with -s to see which backend they used.

Where this can show up

Anyone who runs the decoder ablation with a TensorRT pre-decoder. The results are still correct, since they come from PyTorch, but they are not TensorRT results, and runs take the PyTorch time. The only hint is the "TRT load failed" line. My estimate; I have not seen a field report.

Environment

  • macOS arm64, CPU only, Python 3.11, torch 2.15.0
  • Ising-Decoding main at 4e5ef40

I have a small fix (import time in the helper, fix the mock _Logger, and check in the tests that the engine ran) and can open a PR.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions