Generate 3D model from a 2D image

Hi friends,

I would like to generate a 3D model from a 2D image but I don't have any clue.

From some good instruction, I have successfully generated a binary image with my defined masks. Here the reason why I want a binary image is that the 3D printer only accepts binary slices. I would like to just extrude my pixels into a 3D model (all numbers 1 need to be given the same height, but 0 don't need a height or a neglectable height), and slice them with my printer software. Here is the first section to generate a good binary image. I know there is some other ways we can use to reconstruct a 3D model from a 2D image, but I just want do it in matlab.

[DGM]

Is there a particular reason you're using this dither pattern? There are existing tools for ordered and error-diffusion dithering.

Is there a reason that you're doing dithering at all? At least for a FDM process, this is going to create something that may be a challenge for many printers due to the absurd number of retractions (~30k retractions per layer).

You'd also need to be combining the extruded image with some other solid geometry, otherwise you'd just end up with a bunch of disconnected plastic bits on the build plate. So obviously there's more to this than just extruding the binarized image.

If you're going to slice the generated model anyway, I don't see why it needs to be binarized or dithered. What is the actual goal? Lithophanes?

Answer 1

[DGM]

I've always found that one of the biggest troubles with resin processes is getting decent reproduction of small islands/holes. Small islands tend to break off and end up welding to the print elsewhere, and small holes are always undersized. If the goal were to just produce a lithophane, I'd convert the image into a 3D surface (a height map), and then let the slicer do its job instead of trying to do the slicer's job for it.

If you want to do it with dithering, well here's one attempt, but I'll warn you that it has problems.

% read and strip color
inpict = imread('MonaLisaSquare.jpeg');
inpict = im2gray(inpict);

xyscale = 0.102; % mm/px
zscale = 0.2; % mm
maxwidth = 300; % pixels

% resize
inpict = imresize(inpict,min(maxwidth./size(inpict,1:2)));

% binarize
%binarized = imbinarize(medfilt2(inpict,[1 1]*5)); % something simple
binarized = dither(inpict); % dithered

imshow(binarized,'border','tight')

% generate position data
szo = size(binarized,1:2);
x = xyscale/2:xyscale:xyscale*(szo(2)-0.5);
y = xyscale/2:xyscale:xyscale*(szo(1)-0.5);
z = zscale/4:zscale/2:zscale*0.75; % model is 1 voxel thick

% expand and reorient
out = repmat(flipud(binarized).',[1 1 2]);

% write as voxelized STL
outputname = 'mona.stl';
CONVERT_voxels_to_stl(outputname,out,x,y,z,'binary')

This uses this tool from the FEX

https://www.mathworks.com/matlabcentral/fileexchange/27733-converting-a-3d-logical-array-into-an-stl-surface

The problem here is that two pixels which are only diagonally connected will form a non-manifold edge. You can expect the STL file to be relatively large (a few tens of MB), and you can expect it to have thousands of defects. Whether or not your slicer will put up with that, I don't know.

[Haoqing Yang]

This is so great! seems easier than my script. But I have some error when generating the stl. Let me find some way to fix it!

Error using CONVERT_voxels_to_stl (line 68)

The dimensions of gridDATA do not match the dimensions of gridX, gridY, gridZ.

Error in untitled4 (line 22)

CONVERT_voxels_to_stl(outputname,out,x,y,z,'binary')

[Haoqing Yang]

I still don't know why I always get this error, I just used your script and everything else was correct, just the error was there.

[DGM]

I'm not sure what changes you made. Note that the conversion tool expects the output to be a 3D image whose dimensions are ordered X,Y,Z, so for a standard 2D image, we need to flip dim1, transpose, and replicate so that it has at least two layers on dim3. If the binary image is 75x100x1, the variable called out is 100x75x2. Likewise, 100, 75, and 2 should be the lengths of the x,y,z vectors.

[Haoqing Yang]

I made no change on your script. I understand this, that's why you call repmat. And I read the instruction under CONVERT_voxels_to_stl to get some idea, so far x and y are 1x300 double, and z is a 1x2 array, nothing confilct with the tool. Just cannot generate a stl.

[Haoqing Yang]

CONVERT_voxels_to_stl says (line 67)

if ~isequal(size(gridDATA),[numel(gridX),numel(gridY),numel(gridZ)])
  error(' The dimensions of gridDATA do not match the dimensions of gridX, gridY, gridZ.')
end

size(gridDATA)=1, numel (gridx) or numel (gridx)=300 for sure, numel (gridx)=2 for sure, this if fail, but I still get this error.

[DGM]

Why is size(gridDATA) 1? I'm running R2019b, but I don't think that makes sense. Even a scalar input would return size = [1 1].

Mind you, the way this tool is performing this test is easy to break, but it shouldn't. Let me see if I can try it in the online version.

[Haoqing Yang]

Yes, I trid again, size(gridDATA)=[1,1], [numel(gridX),numel(gridY),numel(gridZ)]=[10 10 10], and ~isequal(size(gridDATA),[numel(gridX),numel(gridY),numel(gridZ)]) gives

ans =

logical

1

It means this error will always show up.

[Haoqing Yang]

Okay I fixed the problem, now it works well.

I appreciate so much for your help!

You made my day!!!!

[DGM]

Oh. What was the problem? Somehow the image got collapsed to 1px?

I'm still trying to get MATLAB Online to even work for me. It's pretty broken in my browser now.

Answer 2

[DGM]

While I'm waiting for MATLAB to load, I'm going to throw this out there.

https://www.mathworks.com/matlabcentral/answers/1926360-plotting-3d-surface-using-depth-map-and-grey-scale-image#comment_3031456

This is just a height map. For a lithophane, you'd just invert the image with imcomplement() so that the brightest areas are thinnest.

% read and strip color
inpict = imread('peppers.png');
inpict = im2gray(inpict);

% invert if making a lithophane
inpict = imcomplement(inpict);

% resize if you want to change the mesh resolution
inpict = imresize(inpict,0.5);

% make sure the image is full-scale and floating point
outpict = mat2gray(inpict);

% object scale parameters (mm)
% this does not do any resampling
reliefheight = 10; % the variation in thickness
basethickness = 2; % the minimum thickness
xwidth = 150; % y-width is calculated to preserve aspect ratio

% construct coordinate arrays
szi = size(outpict);
szo = xwidth*[szi(1)/szi(2) 1];
x = linspace(0,szo(2),szi(2));
y = linspace(szo(1),0,szi(1)); % shift origin to SW corner
[X Y] = meshgrid(x,y);
Z = im2double(outpict)*reliefheight + basethickness; % scale z

% display the surface
figure
hs = surf(X,Y,Z); hold on
hs.EdgeColor = 'none';
hs.FaceColor = [1 1 1]*0.8;
followerlight(hs); % attached

% construct a closed surface
% surf2solid() outputs face/vertex lists, but the native stlwrite()
% only accepts triangulation objects, so convert it
[F V] = surf2solid(X,Y,Z,'elevation',0); % SEE FEX #42876
TR = triangulation(F,V); % complains, but i'm going to ignore that

Warning: Some input points are not referenced by the triangulation.

% display it again
figure
hs = trisurf(TR);
hs.EdgeColor = 'none';
hs.FaceColor = [1 1 1]*0.8;
followerlight(hs);

% write it to a STL
stlwrite(TR,'monarelief.stl')

The slicer does all the work of binarizing the layers.

[DGM]

Is that using this example or the other one? That looks like your original 3x3 custom dither pattern.