Segment Anything 2 model

Artificial intelligence - AI

Image segmentation is a technology used in computer vision to improve the classification of images. It allows for the analysis and identification of objects and their positions at the individual pixel level, providing more detailed information.

The process of image segmentation involves identifying object boundaries in order to define their position and shape, which then allows for the labeling of different regions in the image.

An example of segmentation that predicts class labels for each pixel in an image.

The AI Annotation feature is available on any project that has "Segment anything" selected in the ML model selection field in the Configuration tab. This feature makes it easy to segment an image in just a few clicks.

The Segment Anything algorithm is specifically designed to segment objects in images. When this feature is selected, the following tools are available in the AI annotation configuration window:

Foreground Points: Select the pixels in the image that belong to the object you want to highlight (foreground). This helps the algorithm understand which parts of the image should be marked as foreground.

When you define an annotation point, make sure to add more points until the object is annotated as accurately as possible. If needed, you can manually adjust the annotation using the Lasso and Auto Select tools.

Example of AI annotation while adding foreground points

To define the background points, select points in the image that belong to the background. These points indicate what should be marked as the background and what area has extra pixels that the system accepts as the object you want.

You can also adjust the image manually using the Lasso and Auto Select subtraction tools.

Using background points to adjust AI annotation

To select a Bounding Box around an object in the image, use the Containing Box tool. This rectangle also helps define the boundaries of the object. Once you have added AI annotations, evaluate the segmentation result. If you are not satisfied with the result, you can continue to manually tweak the annotation using the tools: add lasso, subtract lasso, add auto-selection, and subtract auto-selection.

Example of AI annotation while using Containing box tool

If you want to remove necessary parts of the annotation that belong to the annotated object, use the AI annotation subtraction tool. To do this, select the object, then select the "Subtract AI annotation" icon in the tool window that opens. Finally, select the tooltip: Foreground points, Background points, or Containing box, and mark the necessary section to be removed with a dot or frame.

Example of AI substruction using Foreground points

Tracking objects

Segments anything model also provides a tracking function.

The platform's object tracking function supports only specific annotation types: Bitmap and Bounding box. This enables the automatic platform's object tracking function to support only tracking of object movement across a sequence of frames and is applicable exclusively to these two types of annotation.

To track an object over multiple frames, first select the object, then click the “Track Object” button.

Tracking options toolbar will appear, that lets you select the number of frames on which the object needs to be tracked following the current frame by dragging a slider.