Main Content

Instance Segmentation

Perform instance segmentation using pretrained deep learning networks and train networks using transfer learning on custom data

Instance segmentation is a computer vision technique that plays a crucial role in tasks requiring precise object localization and the identification of individual object instances, such as in medical imaging and autonomous driving. By combining the principles of object detection and semantic segmentation, instance segmentation provides a more refined understanding of visual data by identifying individual object instances and delineating their boundaries pixel by pixel. Use instance segmentation to precisely identify, classify, and separate individual objects within an image.

You can perform inference on an image using a pretrained deep learning network, or train a network using perform transfer learning. Transfer learning is an approach in which you to start with a pretrained network and train it on a custom data set for your application. You can interactively generate ground truth data for training,by using the Image Labeler, Video Labeler, or Ground Truth Labeler (Automated Driving Toolbox) app to label pixels and export label data. Instance segmentation requires Deep Learning Toolbox™. Training and inference support CUDA® enabled GPU. Use of a GPU is recommended, and requires Parallel Computing Toolbox™. For more information, see Parallel Computing Support in MathWorks Products (Parallel Computing Toolbox).

Instance segmentation using SOLOv2: Left — segmented and labeled road scenario using a sample modified RGB image from the CamVid data set, Right — segmented image of PVC pipe connectors


expand all

Configure Instance Segmentation Network

solov2Segment objects using SOLOv2 instance segmentation network (Since R2023b)
maskrcnnDetect objects using Mask R-CNN instance segmentation (Since R2021b)

Perform Inference

segmentObjectsSegment objects using Mask R-CNN instance segmentation (Since R2021b)
segmentObjectsSegment objects using SOLOv2 instance segmentation (Since R2023b)

Load Training Data

boxLabelDatastoreDatastore for bounding box label data (Since R2019b)
groundTruthGround truth label data
imageDatastoreDatastore for image data
combineCombine data from multiple datastores

Train Instance Segmentation Networks

trainSOLOV2Train SOLOv2 network to perform instance segmentation (Since R2023b)
trainMaskRCNNTrain Mask R-CNN network to perform instance segmentation (Since R2022a)

Augment and Preprocess Training Data

poly2maskConvert region of interest (ROI) polygon to region mask
bwboundariesTrace object boundaries in binary image
balanceBoxLabelsBalance bounding box labels for object detection (Since R2020a)
bboxcropCrop bounding boxes (Since R2019b)
bboxeraseRemove bounding boxes (Since R2021a)
bboxresizeResize bounding boxes (Since R2019b)
bboxwarpApply geometric transformation to bounding boxes (Since R2019b)
bbox2pointsConvert rectangle to corner points list
imwarpApply geometric transformation to image
imcropCrop image
imresizeResize image
randomAffine2dCreate randomized 2-D affine transformation (Since R2019b)
centerCropWindow2dCreate rectangular center cropping window (Since R2019b)
randomWindow2dRandomly select rectangular region in image (Since R2021a)
insertObjectMask Insert masks in image or video stream (Since R2020b)
insertObjectAnnotationAnnotate truecolor or grayscale image or video
insertShapeInsert shapes in image or video
showShapeDisplay shapes on image, video, or point cloud (Since R2020b)
evaluateInstanceSegmentationEvaluate instance segmentation data set against ground truth (Since R2022b)
instanceSegmentationMetricsInstance segmentation quality metrics (Since R2022b)
metricsByAreaEvaluate instance segmentation across object mask size ranges (Since R2023b)
posemaskrcnnPredict object pose using Pose Mask R-CNN pose estimation (Since R2024a)
predictPoseEstimate object pose using Pose Mask R-CNN deep learning network (Since R2024a)
trainPoseMaskRCNNTrain Pose Mask R-CNN network to perform pose estimation (Since R2024a)


Get Started

Train Data for Instance Segmentation