Article directory
2023
Segment and Track Anything
code : https://github.com/zx-yang/Segment-and-Track-Anything
Abstract : This report provides a framework named Segment and Track Anything (SAMTrack), which allows users to precisely and efficiently segment and Track any object in the video. In addition, SAM-Track employs a multimodal interaction approach that enables users to select multiple objects to track in a video, corresponding to their specific needs. These interaction methods include taps, strokes, and text, each with unique benefits and can be used in combination. As a result, SAM-Track can be applied in a range of fields, from drone technology, autonomous driving, medical imaging, augmented reality to bioanalysis. SAM-Track merges an interactive keyframe segmentation model (SAM) with our proposed AOT-based tracking model (DeAOT), which won first place in four tracks of the VOT 2022 challenge to promote objects in videos track. Furthermore, SAM-Track integrates Grounding-dino, which enables the framework to support text-based interactions. We have demonstrated the remarkable capability of SAM-Track on DAVIS-2016 Val (92.0%), DAVIS-2017 test (79.2%), and its utility in various applications.
Track Anything: Segment Anything Meets Videos
code: https://github.com/gaomingqi/Track-Anything
Abstract: In recent years, Segmentation Arbitrary Models (SAMs) have rapidly gained widespread attention due to their performance on image segmentation. Due to its strong image segmentation ability and high interactivity of different cues, we found it to be less effective in video-consistent segmentation. Therefore, in this report, we propose Track Anything Model (TAM), which enables high-performance interactive tracking and segmentation in videos. In detail, given a video sequence with only little human involvement, that is, a few clicks, people can track whatever they are interested in and get satisfying results in ontology inference. Without additional training, this interactive design performs impressive video object tracking and segmentation.
SAM-DA: UAV Tracks Anything at Night with SAM-Powered Domain Adaptation
code: https://github.com/vision4robotics/sam-da
Abstract : Domain Adaptation (DA) shows great promise for real-time nighttime Unmanned Aerial Vehicle (UAV) tracking. However, state-of-the-art (SOTA) DA still lacks latent objects with precise pixel-level locations and boundaries to generate high-quality training samples in the target domain. This critical issue limits transfer learning for real-time nighttime SOTA trackers to challenge nighttime UAV tracking. Recently, the well-known Segment Anything Model (SAM) has achieved remarkable zero-shot generalization ability in discovering a large number of potential objects due to its huge data-driven training method. To address the above issues, this work proposes a new DA framework for sam-driven real-time night-time drone tracking, namely SAM-DA. Specifically, an innovative SAM-driven target domain training sample inflation is devised to determine huge high-quality target domain training samples from each raw nighttime image. This new one-to-many approach significantly expands the high-quality target domain training samples for DA. Comprehensive experiments on a large number of nighttime drone videos demonstrate the robustness and domain adaptability of SAM-DA for nighttime drone tracking. In particular, SAM-DA can achieve better performance with fewer nighttime raw images than SOTA DA, i.e. less-better training.
