DMD ROI Image Registration Correct Procedure

Hi everyone,

I am making a system involving a DMD (digital micromirror device) to provide ROI selection capabilties for sample illumination. Everything happens under a custom-made system that magnifies the light 2x at the sample plane, which then gets imaged onto the camera. The same camera is used for everything.

What I need to do is broken into two parts:

1. Create a script which generates a transformation matrix that can be used on user-defined ROIs to select areas of the sample to illuminate
2. Create a script which allows the user to select an ROI of an image and and then uses the predefined transformation matrix to transform it into an image registered with the DMD.

I have a working script which completes both of these parts (provided below), but....it only works on some images. Only on images that are the same size as the original image that the tform matrix was generated with in part 1.

I understand that posting the full code of the project is discouraged, but I have an issue somewhere and I don't know where, so it is pertinent that all of the code is presented.

Part 1 code (plain english breakdown at the end):

%%
clear all; close all; clc; beep off; 
%% Step 1: get an image to register
image = snapshot(); %snapshot is a custom function which takes a single exposure of the camera and loads it into matlab using the image acqusition toolbox. 

%% Step 2: Draw ROI 
imshow(image)
% Allow the user to draw a freehand ROI
h = drawpolygon('Color', 'r'); % 'Color' specifies the color of the ROI
% Create a binary mask from the ROI
initial_mask = createMask(h); % Get the binary mask
% Resize the mask to 684x608 pixels
initial_mask_resized = imresize(initial_mask, [684, 608]);
close all; 

%% Step 3: export the inital mask
initial_mask_resized_flipped = flip(initial_mask_resized, 2); %mask should be flipped "2"
imwrite(initial_mask_resized_flipped, 'initial_test_mask_resized_flipped.bmp');

%% Step 4: Load to DMD 
% Do it manually 
input("Please upload the image to the lightcrafter manually. Press any key and then hit enter to continue.\n\n\n", "s")
%% Step 5: Take new snapshot to register 
image2 = snapshot();

%% Step 6: Register mask and image 
image2_resized = imresize(image2, [684, 608]); 

fixed_image = image2_resized;
moving_image = image2_resized;

[movingPoints, fixedPoints] = cpselect(moving_image, fixed_image, 'Wait', true);
tform = fitgeotform2d(movingPoints,fixedPoints,"similarity");
Roriginal = imref2d(size(fixed_image));
registered_mask = imwarp(initial_mask_resized,tform,OutputView=Roriginal);

montage({fixed_image, registered_mask})

%% Step 7: flip and resize 
final_mask_registered_flipped = flip(registered_mask, 2);
final_mask_registered_flipped_resized = imresize(final_mask_registered_flipped, [684, 608]);
imwrite(final_mask_registered_flipped_resized, 'final test registered mask.bmp');

%% Step 8: Save matrix
% build timestamp: HHmm_MMddyy  (hours, minutes _ month, day, 2-digit year)
ts = datetime('now','Format','HH_mm_._MM_dd_yy');
fname = "registration_matrix_" + char(ts) + ".mat";
% save
save(fname, 'tform');

Plain english breakdown:

Step 1: takes an image with the setup's camera. Image returns as 1920x1200.

Step 2: displays the image and allows the user to draw an ROI on the image. I usually use a triangle printed on paper under the setup. The mask is the resized to the DMD's size [684x608].

Step 3: The mask is flipped such that it is in the correct orientation for display, and then exported.

Step 4: You manually load the mask onto the DMD and the continue the code when ready.

Step 5: Another image is taken, this one with the DMD on and displaying the mask generated in step 2.

Step 6: The 2nd image is resized and passed into control point selection tool. The same image is displayed, yes, but the original printed triangle is selected for the fixed image and the triangle projected by the DMD is selected for the moving image. The registered mask is then displayed.

Step 7: The mask is flipped back for display (which is what the DMD requires due to how it is orinented).

Step 8: The tform matrix saved.

Part 2 code:

clear all; close all; clc; beep off;

%% Step 1: 
% Let user select an image file
[filename, pathname] = uigetfile( ...
    {'*.png;*.jpg;*.jpeg;*.tif;*.tiff;*.bmp;*.gif','Image Files (*.png,*.jpg,*.tif,...)';
    '*.*','All Files (*.*)'} , ...
    'Select an image file');
% If the user cancels, exit gracefully
if isequal(filename,0)
    disp('User canceled file selection.');
    return
end
% Read and display the selected image
imgPath = fullfile(pathname, filename);
image = imread(imgPath);
image = imresize(image, [684, 608]);

%% Step 2: Draw ROI
imshow(image)
title('Draw ROI on the image');
% Allow the user to draw a freehand ROI
h = drawpolygon('Color', 'r'); % 'Color' specifies the color of the ROI
% Create a binary mask from the ROI
initial_mask = createMask(h); % Get the binary mask
% Resize the mask to 684x608 pixels
initial_mask_resized = imresize(initial_mask, [684, 608]);
% Display the resized mask
figure;
imshow(initial_mask_resized);
title('Resized Binary Mask');
pause(2)
close all 

%% Step 3: Register new mask 
registration_matrix = dir("registration_matrix_*.mat")
load(registration_matrix(1).name)
Roriginal = imref2d(size(image));
registered_mask = imwarp(initial_mask_resized,tform,OutputView=Roriginal);
final_mask_registered_flipped = flip(registered_mask, 2);
final_mask_registered_flipped_resized = imresize(final_mask_registered_flipped, [684, 608]);
% build timestamp: HHmm_MMddyy  (hours, minutes _ month, day, 2-digit year)
ts = datetime('now','Format','HH_mm_._MM_dd_yy');
fname = "final_registered_ROI_mask_" + char(ts) + ".bmp";
imwrite(final_mask_registered_flipped_resized, fname);

Plain english breakdown:

Step 1: Allows the user to select an image file.

Step 2: allows the user to draw a desired ROI on the image selected in step 1.

Step 3: registers the image using the previously generated tform matrix.

The problem:

The code above works fine when the transform matrix is generated on an image the same size as the image used in part 2. But for our use case the image is not the same size. Example:

Part 1 uses images of size 1920x1200 as generated by the snapshot() function.

Part 2 has the user select an image of size 1194x994.

In this example the transform matrix is not correct.

I have been told by someone with more experience than me that I shouldn't be using imresize at all. I haven't worked out how to make that happen yet and this has me scratching my head.