Search code, repositories, users, issues, pull requests...

SAM2Long: Enhancing SAM 2 for Long Video Segmentation with a Training-Free Memory Tree Shuangrui Ding, Rui Qian, Xiaoyi Dong, Pan Zhang Yuhang Zang, Yuhang Cao, Yuwei Guo, Dahua Lin, Jiaqi Wang CUHK, Shanghai AI Lab

SAM2Long significantly improves upon SAM 2 by addressing error accumulation issue, particularly in challenging long-term video scenarios involving object occlusion and reappearance. With SAM2Long, the segmentation process becomes more resilient and accurate over time, maintaining strong performance even as objects are occluded or reappear in the video stream.

SAM2Long introduces a training-free memory tree that effectively reduces the risk of error propagation over time. By maintaining diverse segmentation hypotheses and dynamically pruning less optimal paths as the video progresses, this approach enhances segmentation without the need for additional parameters or further training. It maximizes the potential of SAM 2 to deliver better results in complex video scenarios.

SAM2Long pushes the performance limits of SAM 2 even further across various video object segmentation benchmarks, especially achieving an average improvement of 3 in J & F scores across all 24 head-to-head comparisons on long-term video datasets like SA-V and LVOS.

Leave a Comment