Play SAM (Segment Anything)
Official website link:
Segment Anything | Meta AI (segment-anything.com)
github link:
Paper link:
[2304.02643] Segment Anything (arxiv.org)
Dataset link:
Segment Anything | Meta AI (segment-anything.com)
Online demo link:
Segment Anything | Meta AI (segment-anything.com)
Does the emergence of SAM indicate the desolation of the traditional CV industry? With the emergence of products such as Chatgpt and diffusion models, prompt engineering , AIGC , etc. have become super hot topics today. Dividing everything, the SAM model trained by Facebook using a large data set has brought a huge impact to the CV field, making prompt engineering prompt engineering also developed and applied in the CV field, which has also inspired many of our computer vision researchers. Data can realize prompt models for various types of scene vision tasks, and even the ultimate large model that unifies the visual paradigm is getting closer and closer to us. Similarly, it also brings us concerns that artificial intelligence seems to be going further and further away in the use of big data for fixed-paradigm supervised learning, and is getting further and further away from the real future of artificial intelligence-spontaneous unsupervised learning. But we won't talk about these here, let's experience the delicious SAM!
1. Play online
If you just want to play for a couple of times, you can click on the online demo link above and start surfing!
The official website demo link gives three hint methods: point, box and complete segmentation.
2. API calls
If you don't want to be limited to the online demo, but want to call SAM's api interface to realize your various ideas and needs, and carry out secondary development, etc., just press us to start!
2.1 Install and configure SAM environment
# 安装相关依赖
pip install opencv-python pycocotools matplotlib onnxruntime onnx
安装SAM
方法一：
pip install git+https://github.com/facebookresearch/segment-anything.git
方法二：
git clone [email protected]:facebookresearch/segment-anything.git
cd segment-anything; pip install -e .
2.2 Operating instructions
First download a model checkpoint. The mask can then be obtained from a given hint with just a few lines of code:
from segment_anything import SamPredictor, sam_model_registry
sam = sam_model_registry["<model_type>"](checkpoint="<path/to/checkpoint>")
predictor = SamPredictor(sam)
predictor.set_image(<your_image>)
masks, _, _ = predictor.predict(<input_prompts>)
Or generate a mask for the whole image:
from segment_anything import SamAutomaticMaskGenerator, sam_model_registry
sam = sam_model_registry["<model_type>"](checkpoint="<path/to/checkpoint>")
mask_generator = SamAutomaticMaskGenerator(sam)
masks = mask_generator.generate(<your_image>)
In addition, the mask of the image can also be generated using the command line:
python scripts/amg.py --checkpoint <path/to/checkpoint> --model-type <model_type> --input <image_or_folder> --output <path/to/output>
Note: In subsequent articles, we will explain the SAM interface operation in more detail!