Skip to content

BoT-SORT tracking, but code implements a custom IoU tracker (no KF/Hungarian/ReID/GMC) #2

Open

Description

@ArghyaChatterjee

Hello,

Thanks for your work.

The README mentions BoT-SORT for multi-object tracking, but the current implementation appears to be a custom IoU-based tracker with heuristic scoring. I couldn’t find BoT-SORT components (Kalman filter, Hungarian/global assignment, two-stage association like ByteTrack, camera motion compensation, or ReID embeddings).

What I expected

  • A BoT-SORT style tracker: constant-velocity Kalman filter state prediction, global (Hungarian) association on an IoU/appearance distance matrix, two-stage association (high-score then low-score detections), optional ReID (e.g., FastReID) and camera motion compensation (global 2×3 warp).

What I observed

  • Greedy IoU matching (best IoU > threshold) per detection.
  • No Kalman filter prediction/update, no linear/Hungarian assignment.
  • No two-stage association using low-score detections.
  • No camera motion compensation (GMC) and no ReID feature extraction/fusion.
  • A heuristic track “score” that blends probability/temporal/border/size/texture factors to start/stop publishing tracks.

How to reproduce / verify

  1. Search the repo for symbols like BoTSORT, bot_sort, Kalman, Hungarian, linear_assignment, FastReID, or GMC — none appear.
  2. Inspect the tracker module: matching is greedy IoU, track state is updated without a motion model, and tracks are started/stopped via heuristic thresholds.

Why this matters
Calling the current implementation “BoT-SORT” may mislead users about expected robustness (occlusions, camera motion, crowded scenes). BoT-SORT’s motion model + global assignment + two-stage matching + (optional) ReID/GMC are key to its performance.

Suggestions (any of these would resolve the mismatch)

  • Doc fix: Update the README to describe the tracker as a “lightweight IoU-based tracker with heuristic scoring,” and note BoT-SORT as a planned enhancement.

  • Feature addition: Integrate a true BoT-SORT pipeline:

    • Per-track constant-velocity Kalman filter.
    • Hungarian assignment over an IoU/appearance distance matrix (with gating).
    • Two-stage association (high-score then low-score detections).
    • Optional ReID (e.g., FastReID) and GMC (ECC/optical-flow affine).
    • Duplicate-track suppression (keep longer-lived track when IoU is high).

If I’m missing the BoT-SORT path or a compile-time flag that enables it, happy to be corrected—please point me to the relevant files/params. Thanks!

Activity

  1. alexlin2 commented on Aug 26, 2025

    @alexlin2
    Collaborator

    Hi, this repository is temporarily moved to another project, I have updated the perception system significantly and includes a lot more features such as grasp generation, 3D object detection and tracking etc. The new repository is called dimensional/dimos

  2. alexlin2 commented on Aug 26, 2025

    @alexlin2
    Collaborator

    I just emailed you

  3. ArghyaChatterjee commented on Aug 27, 2025

    @ArghyaChatterjee
    Author

    Hey Alex,

    Can you briefly describe why you used your own tracker in stead of BoTSORT Tracker ? I have tested your repo and the detection, segmentation looks pretty stable and fine to me.

    Is there a particular reason why you stayed with custom IoU based tracker with heuristic scoring over the original BoTSORT Tracker ?

  4. alexlin2 commented on Aug 27, 2025

    @alexlin2
    Collaborator

    Hey Alex,

    Can you briefly describe why you used your own tracker in stead of BoTSORT Tracker ? I have tested your repo and the detection, segmentation looks pretty stable and fine to me.

    Is there a particular reason why you stayed with custom IoU based tracker with heuristic scoring over the original BoTSORT Tracker ?

    Actually both are used. The FastSAM implementation I'm using is ultralytics, and when you have the model.track(persist=true) it uses the default tracker which is BoT-SORT (https://docs.ultralytics.com/datasets/track/#dataset-format-coming-soon). The reason why I implemented heuristics and scoring over it is because their implementation did not yield the results I wanted so I just added my own on top of it which is kind of hacky tbh.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions