How to generate mask R-CNN training set data from matlab's image labeler

I use Image labeler to mark images, Rectangle to mark object positions and Pixel to mark object categories. However, the generated gTruth file cannot be directly used for mask R CNN training, nor can it directly use the imageDatastore and boxLabelDatastore instructions to directly generate relevant data.

I followed the instructions on matlab's web page and used imageDatastore to grab the RGM image path from gTruth.

imageFilesA = gTruth.DataSource.Source;

imds = imageDatastore(imageFilesA);

Reorganize object position label data and use boxLabelDatastore to generate boxes and labels data.

rawBoxes = gTruth.LabelData{:, 1};

allBoxes = cell(size(rawBoxes,2));

for i = 1:numel(rawBoxes)

boxes = rawBoxes{i};

numBoxes = size(boxes, 1);

allBoxes{i,2} = string(repmat({'WWL'}, numBoxes, 1));

end

allBoxes = cell2table(allBoxes,'VariableNames',{'Boxes','Labels'});

blds = boxLabelDatastore(allBoxes);

But the final masks data cannot be generated. Because when I use imageDatastore to generate mask image file path data, I follow the instructions to generate H*W*M data (where H*W is the image size and M is the number of objects in the image).

Assuming that the first image has 4 markers and I generate a 1*4 cell image file path, an error will occur when using imageDatastore. If I store four image file paths into a 1*4 table, imageDatastore can be used, but when I need to combine three data later, a parameter error will be displayed.

Here is one of the methods I tried

maskPaths = gTruth.LabelData{:, 2};

numImages = numel(maskPaths);

logicalMasks = cell(numImages, 1);

tempdir='F:\MaskRCNN\ImageIMDS\';

targetMaskPaths = {};

for i = 1:numImages

mask = imread(maskPaths{i});

[H, W] = size(mask);

boxes = rawBoxes{i};

M = size(boxes, 1);

for j = 1:M

logicalMask = mask;

tempMaskFilePath = fullfile(tempdir, sprintf('mask_%d_target_%d.png', i, j));

imwrite(uint8(logicalMask), tempMaskFilePath);

targetMaskPaths{i,j} = tempMaskFilePath;

end

end

classes = ["WL"];

labelIDs = [0 1];

pxds = imageDatastore(targetMaskPaths);

Error message

Error using matlab.io.datastore.ImageDatastore (line 305)

Files must be a valid string scalar, character vector, cell array of character vectors,

string array, or a matlab.io.datastore.FileSet.

Error in imageDatastore (line 139)

ds = matlab.io.datastore.ImageDatastore(location, varargin{:});

^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

Error in MaskTest (line 77)

pxds = imageDatastore(targetMaskPaths);

^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

Here is one of the methods I tried

maskPaths = gTruth.LabelData{:, 2};

numImages = numel(maskPaths);

logicalMasks = cell(numImages, 1);

tempdir='F:\MaskRCNN\ImageIMDS\';

targetMaskPaths = {};

for i = 1:numImages

mask = imread(maskPaths{i});

[H, W] = size(mask);

boxes = rawBoxes{i};

M = size(boxes, 1);

targetMaskPaths = cell(M, 1);

for j = 1:M

logicalMask = mask;

tempMaskFilePath = fullfile(tempdir, sprintf('mask_%d_target_%d.png', i, j));

imwrite(uint8(logicalMask), tempMaskFilePath);

targetMaskPaths{i,j} = tempMaskFilePath;

end

logicalMasks{i} = targetMaskPaths';

end

classes = ["WL"];

labelIDs = [0 1];

pxds = imageDatastore(targetMaskPaths);

Error message

Error using trainMaskRCNN (line 181)

[trainedNet] = trainMaskRCNN(trainingData, net, options);

^^^^^^^^^^^^

Invalid argument at position 1. Expected input to be one of these types:

logical

Instead its type was uint8.

Error in MaskTest (line 99)

[trainedNet] = trainMaskRCNN(trainingData, net, options);

[Govind KM]

Hi @Huang Wei-Cher, could you share the data being used?

Answer 1

[Gayathri]

Hi @Huang Wei-Cher,

I understand that you are facing an error while creating the “ImageDatastore” object of the “tempMaskFilePath” in the first approach mentioned.

This is because “ImageDatastore” function expects the input to be a valid string scalar, character vector, cell array of character vectors / string arrays or a “matlab.io.datastore.FileSet”.

I created a sample code snippet to replicate the issue. To resolve this, we can make the “targetMaskPaths” variable to a one-dimensional array of string arrays using the line of code shown below.

targetMaskPaths{end+1} = tempMaskFilePath; 

With this code we can obtain a structure for “pxds” variable as shown below.

For more information on “ImageDatastore” function please refer to the following link.

https://www.mathworks.com/help/matlab/ref/matlab.io.datastore.imagedatastore.html

And for more information on Mask R-CNN, please refer to the below link.

https://www.mathworks.com/help/vision/ug/getting-started-with-mask-r-cnn-for-instance-segmentation.html

Hope you find this information helpful.