Repository navigation
Conversation
_setup_trt_for_ablation called _time.perf_counter() without importing time, so ONNX_WORKFLOW=2 and 3 in the decoder ablation hit a NameError, which the except blocks turned into a silent PyTorch fallback. The mock TensorRT tests did not catch it: the mock Logger took no arguments, so they also fell back. Let the mock Logger take the severity, count execute_v2 calls, and check that the engine ran. Refs NVIDIA#106 Signed-off-by: Dezhen Lu <dezhen.lu@student.uni-tuebingen.de>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Refs #106.
_setup_trt_for_ablationnow importstime as _timeitself, soONNX_WORKFLOW=3loads the TensorRT engine andONNX_WORKFLOW=2builds and uses one in the decoder ablation, instead of falling back to PyTorch onNameError: name '_time' is not defined.The mock TensorRT tests now reach the TensorRT path and check that it ran.
Changes
code/evaluation/failure_analysis.py: addimport time as _timeat the top of_setup_trt_for_ablation. This is the same local importdecoder_ablation_studyalready uses.code/tests/test_failure_analysis.py:_Loggertakes the severity argument, astrt.Logger(trt.Logger.WARNING)passes one. Before, this failed first, so the TensorRT tests ran on the PyTorch fallback.execute_v2calls. New teststest_trt_path_runs_the_engineandtest_export_and_build_runs_the_enginecheck that the count is above zero. The fallback returns a well-formed result, so the existing structure checks could not tell the two apart.How I found it
ruff check --select F821reported_timeas undefined in_setup_trt_for_ablation. Running the TensorRT tests with-sshowed they used the PyTorch backend.Where this can show up in practice
Decoder ablation runs with a TensorRT pre-decoder (
ONNX_WORKFLOW=2or3withWORKFLOW=decoder_ablation, as in thelocal_run.shcomments). Before this change they report PyTorch results and take PyTorch time, with only a "TRT load failed" line in the log. My estimate; I have not seen a field report.Testing
On CPU, macOS arm64, Python 3.11, torch 2.15.0:
pytest code/tests/test_failure_analysis.py: 79 passed.main), the two new tests fail and the other 77 pass.-s, the TensorRT tests now printPre-decoder backend: TRT (ONNX_WORKFLOW=3)(X and Z) andTRT (ONNX_WORKFLOW=2). Before, they printedPyTorch.trt_context: (...)instead ofNone.yapf --diffis clean on both files.I did not run it with a real TensorRT engine on a GPU.