Repository navigation
DeepStream tracker library causes OOM error when compiled with -DCMAKE_BUILD_TYPE=Release option #253
Description
Description
Build libByteTracker.so as per the instruction given in (README.md), but replace cmake .. with cmake -DCMAKE_BUILD_TYPE=Release to enable optimizations. Running a GStreamer pipeline with the built library file takes up all system memory immediately and crashes with OOM error.
To Reproduce
- Launch Docker container, set the memory limit to avoid screwing up the host system.
docker run --rm -it --gpus=all --memory=4G nvcr.io/nvidia/deepstream:6.1-devel- Install dependencies.
apt update && apt install -y libeigen3-dev- Clone repo and build
libByteTracker.so.
cd /root
git clone --depth 1 https://github.com/ifzhang/ByteTrack/
cd ByteTrack/deploy/DeepStream
mkdir build && cd build
# cmake -> /usr/local/bin/cmake version is 3.15.2
# /usr/bin/cmake version is 3.16.3
/usr/bin/cmake -DCMAKE_BUILD_TYPE=Release ..
make- Run GStreamer pipeline.
gst-launch-1.0 -v \
filesrc location=/opt/nvidia/deepstream/deepstream-6.1/samples/streams/sample_1080p_h264.mp4 ! qtdemux ! h264parse ! nvv4l2decoder \
! m.sink_0 nvstreammux name=m batch-size=1 width=1920 height=1080 \
! nvinfer config-file-path=/opt/nvidia/deepstream/deepstream-6.1/samples/configs/deepstream-app/config_infer_primary.txt \
! nvtracker \
ll-lib-file=/root/ByteTrack/deploy/DeepStream/lib/libByteTracker.so \
tracker-width=640 \
tracker-height=384 \
! nvvideoconvert ! nvdsosd ! fpsdisplaysink sync=0 video-sink=fakesink text-overlay=0- Observe memory usage. After a while, the program is killed and exit code is 137.
Additional Information
The issue is probably caused by missing return value in NvMOTContext::processFrame, which is an undefined behaviour that could lead to bad compiler optimizations.
https://github.com/ifzhang/ByteTrack/blob/72ca8b45d36caf5a39e949c6aa815d9abffd1ab5/deploy/DeepStream/src/NvMOTContext.cpp#L9-L56
Simply append return NvMOTStatus_OK; at the end of the function, rebuild the library, and the issue is fixed (tested on my local machine).
Activity
EmpireofKings commentedon Sep 27, 2022
FYI appending
return NvMOTStatus_OK;to the end ofNvMOTContext::processFramedoes not fix the memory leak.