nonsensical imread behavior with multiframe gifs in R2019b

17 views (last 30 days)
DGM
DGM on 20 Nov 2020
Edited: DGM on 21 Apr 2023
I have been working primarily in R2009b and R2015b. When some users brought up some issues they were having with my tools in R2019a/b, I borrowed access to another computer with R2019b to straighten out those issues. Along the way I found out that I could no longer read muiltiframe gifs correctly in R2019b. I normally use my own gifread/gifwrite tools for this, but for the sake of certainty, I'm going to do most of these examples the long way using imread, imwrite, and imfinfo.
I found that certain simple images could be read correctly, but most images with unique LCTs couldn't. I will start with a simple gif which does work:
%% try an existing gif using mostly native tools
originalfile='d6.gif';
rewritefile1='d6rw2019-ind.gif';
rewritefile2='d6rw2019-rgb.gif';
% read the original image using imread directly
[A, ~]=imread(originalfile,'Frames','all');
s=size(A);
nframes=size(A,4);
Amap=zeros([256 3 1 nframes],'double');
infostruct=imfinfo(originalfile);
for f=1:nframes
thismap=infostruct(1,f).ColorTable;
Amap(1:size(thismap,1),:,:,f)=thismap; % LCT might not be full-length
end
% show frame 10 for inspection of indexed copy (look at SE corner with datatip tool)
% the background is solid and uniform, no artifacts. all pixels are index 7, mapped to [0.247 0.251 0.243]
imshow(A(:,:,:,10),Amap(:,:,:,10));
% make an RGB copy
Argb=zeros([s(1:2) 3 nframes]);
for f=1:nframes
Argb(:,:,:,f)=ind2rgb(A(:,:,:,f),Amap(:,:,:,f));
end
% show first 12 frames for simple web display
imshow2(imtile(addborder(Argb,2,[127 255 60]/255),[3 4]),'invert','tools')
Clearly, this image is read correctly. If the image is written directly using the indexed data and original CTs, the image can still be read back correctly:
%% continue
% rewrite the file directly using imwrite and original indexed image and maps
nframes=size(A,4);
for f=1:nframes
if f==1
imwrite(A(:,:,:,f),Amap(:,:,:,f),rewritefile1)
else
imwrite(A(:,:,:,f),Amap(:,:,:,f),rewritefile1,'writemode','append')
end
end
% read the second image using imread directly
[B, ~]=imread(rewritefile1,'Frames','all');
s=size(B);
nframes=size(B,4);
Bmap=zeros([256 3 1 nframes],'double');
infostruct=imfinfo(rewritefile1);
for f=1:nframes
thismap=infostruct(1,f).ColorTable;
Bmap(1:size(thismap,1),:,:,f)=thismap; % LCT might not be full-length
end
% show frame 10 for inspection of indexed copy (look at SE corner with datatip tool)
% the background is solid and uniform, no artifacts. all pixels are still index 7
imshow(B(:,:,:,10),Bmap(:,:,:,10));
% map is read exactly as written
imrange(Amap-Bmap) % returns global max and min of [0 0]
% make an RGB copy for display
Brgb=zeros([s(1:2) 3 12]);
for f=1:12
Brgb(:,:,:,f)=ind2rgb(B(:,:,:,f),Bmap(:,:,:,f));
end
% show first 12 frames for simple web display
imshow2(imtile(addborder(Brgb,2,[60 127 255]/255),[3 4]),'tools')
However, if the image is converted to RGB and back to indexed and written, the image can no longer be read correctly.
%% continue
% rewrite the file directly using imwrite, but use the RGB image instead
nframes=size(A,4);
Cmapwritten=zeros([256 3 1 nframes],'double');
for f=1:nframes
[thisframe thismap]=rgb2ind(Argb(:,:,:,f),256);
Cmapwritten(1:size(thismap,1),:,:,f)=thismap;
if f==10; writtensample=thisframe(130:140,130:140,:), end
if f==1
imwrite(thisframe,thismap,rewritefile2)
else
imwrite(thisframe,thismap,rewritefile2,'writemode','append')
end
end
% read the second image using imread directly
[C, ~]=imread(rewritefile2,'Frames','all');
s=size(C);
nframes=size(C,4);
Cmap=zeros([256 3 1 nframes],'double');
infostruct=imfinfo(rewritefile2);
for f=1:nframes
thismap=infostruct(1,f).ColorTable;
Cmap(1:size(thismap,1),:,:,f)=thismap; % LCT might not be full-length
end
% show frame 10 for inspection of indexed copy (look at SE corner with datatip tool)
% the background has worm artifacts, dominant index is 35 (not what was written)
imshow(C(:,:,:,10),Cmap(:,:,:,10));
readsample=C(130:140,130:140,:,10)
% map is read exactly as written
imrange(Cmapwritten-Cmap)
% make an RGB copy for display
Crgb=zeros([s(1:2) 3 12]);
for f=1:12
Crgb(:,:,:,f)=ind2rgb(C(:,:,:,f),Cmap(:,:,:,f));
end
% show first 12 frames for simple web display
imshow2(imtile(addborder(Crgb,2,[255 60 127]/255),[3 4]),'tools')
Inspecting the BG ROI of frame 10 reveals that the indices of the read data don't correspond to what was written.
writtensample =
11×11 uint8 matrix
32 32 32 32 32 32 32 32 32 32 32
32 32 32 32 32 32 32 32 32 32 32
32 32 32 32 32 32 32 32 32 32 32
32 32 32 32 32 32 32 32 32 32 32
32 32 32 32 32 32 32 32 32 32 32
32 32 32 32 32 32 32 32 32 32 32
32 32 32 32 32 32 32 32 32 32 32
32 32 32 32 32 32 32 32 32 32 32
32 32 32 32 32 32 32 32 32 32 32
32 32 32 32 32 32 32 32 32 32 32
32 32 32 32 32 32 32 32 32 32 32
readsample =
11×11 uint8 matrix
35 35 35 35 35 35 35 35 35 35 35
35 35 35 35 35 35 35 35 35 35 35
35 35 22 35 35 2 22 35 35 35 35
22 2 22 35 22 22 22 2 22 22 35
35 22 22 2 22 35 22 22 35 35 2
35 35 35 22 22 2 22 35 22 35 22
35 35 35 35 35 22 22 2 22 35 22
35 35 35 35 35 35 35 22 22 2 22
35 35 35 35 35 35 35 35 35 22 22
35 35 35 35 35 35 35 35 35 35 35
35 35 35 35 35 35 35 35 35 35 35
So imread isn't reading the image data block correctly, but the object content is still there Iit's still shaped like a d6). At first I thought this might be some LZW decoding bug, but then I accidentally found this baffling case:
%% try to "heal" the bad indexed image by using the wrong map
clc;
Crgbmagic=zeros([s(1:2) 3 12]);
for f=1:12
Crgbmagic(:,:,:,f)=ind2rgb(C(:,:,:,f),Cmap(:,:,:,1));
end
imshow2(imtile(addborder(Crgbmagic,2,[255 60 127]/255),[3 4]),'tools')
So if I use the wrong colormap, it almost looks right. The worm artifacts are still there, but they're very subtle. I have no idea how this is possible if the indexed image itself is being read incorrectly
So I guess I can sum this up into three core questions:
  1. Can this behavior be replicated by anyone else?
  2. Why can't imread read the image data for multiframe images correctly?
  3. Why do the incorrect images almost magically work with the wrong color table?
I've searched for bug reports and haven't found anything regarding this. I apologize for making this post so ridiculously long. If anyone tries to run the code blocks, just comment out the lines with the custom tools; either that or feel free to get my "Image Manipulation Toolbox" from the FEX.
EDIT:
As soon as I had posted that and went to rest, I had by a sinking suspicion. The color tables are correct, The image data is incorrect, but loosely matches when using the first color table. The incorrect image data has worm artifacts. Why would there be worm artifacts? Error diffusion dithering in flat image regions. That's why. Surely they aren't remapping all the frames to match CT1?
After a bit of digging, I found this (line 263 readgif.m)
% See geck:g1678142: Currently imread returns only the first local
% color table (if present).
% Using the first local color table to read all the frames later
% shows distorted frames of the image.
%
% In order to fix this issue, rescale the frames (starting from second
% frame) such that it works correctly when rendered using the first
% local color table.
So instead of supporting the output of multiple color tables, imread deliberately remaps it to fit the first color table instead. I guess I can scrub those prior three questions and ask some others.
  1. How is this not a bug? Even disregarding the needlessly destructive adulteration of the image data, I'm sure anyone can think of a scenario where presuming that all LCTs are similar would fall flat on its face. This turns imread's multiframe gif handling from "clumsy" to "broken". Should I just report this?
  2. What versions does this imapct? I only have tried R2015b and R2019b. Like I said, I haven't found any mention of this elsewhere. The number in readgif.m hasn't turned up any results, though maybe i'm looking in the wrong place. Has this been fixed already in newer versions?
  3. Is there any practical way to correctly read multiframe gifs anymore? I can set write permissions on readgif.m and comment out lines 304, 337, 366, 396 to fix it, but this is not something that any student in a lab can do. My own gifread tool has a workaround mode for an old bug (#813126), which also serves as a workaround for this one, but that requires external tools to split the file -- something which won't work for most people either.
  3 Comments
Show 1 older comment
Stephen23
Stephen23 on 20 Nov 2020
"Should I just report this?"
Yes. As you say, requiring destructive data changes is not a feature, it is a bug.
DGM
DGM on 21 Nov 2020
I went ahead and filed one, or at least tried. I suppose I should keep this post up, as Answers posts show up in web searches, whereas bug reports don't. I'll update with any new info I get.

Sign in to comment.

Sign in to answer this question.

Accepted Answer

DGM
DGM on 26 Nov 2020
Edited: DGM on 26 Nov 2020
I'm glad to have some traction on a bug report. As things develop further, any info or patches will be added/linked here.
If you're here from a web search or are otherwise curious, please let me be a little more thorough in my explanation of the issue as I understand it. I've been building this pile of notes to document the behavior, so I might as well share. Let's start with some context. Feel free to chime in if I'm misunderstanding something.
Color tables in a GIF file:
A GIF file may consist of one or more indexed images, the colors referenced by the indexes being defined in one or more color tables. A global color table (GCT) may optionally be defined following the image header. When defined, the GCT acts as a default color table within the scope of the file. If no GCT is defined, the decoder may fall back to some internal or system-level color table. Each image within the GIF file may also optionally include a local color table (LCT). When specified, a LCT takes precedence over the GCT within the scope of the corresponding image block. Long story short, a multiframe GIF may have multiple color tables, the precedence rules for which are defined in the format specification. See https://www.w3.org/Graphics/GIF/spec-gif89a.txt
Legacy behavior of imread():
Up until R2018b, the behavior of imread() when reading a multiframe gif with the syntax [myimage map]=imread('catpicture.gif') was to return all image frames as the 4D array 'myimage', and a single color map 'map'. In this case, 'map' is not necessarily the GCT. It is whichever color table is assigned to frame 1 as per the rules of color table precedence. The problem is that imread() ignores the existence of any color tables defined for subsequent frames. The remaining color tables must be read using imfinfo() separately. This has been a longstanding source of confusion for users who are either unaware that a GIF file may contain multiple color tables, or otherwise don't know about imfinfo(). I would know, since I fell for it myself in the past.
Behavior of imread() after R2018b:
After R2018b, the behavior of imread() for the same scenario is to return a 4D image 'myimage', and again, a single map corresponding to the calculated CT for frame 1. The issue is that the image data in 'myimage' for all frames >1 has been altered in an attempt to remap all frames to use the color table for frame 1. This change is intentional, as can be found in the support file readgif.m found at $matlabroot/toolbox/matlab/imagesci/private/readgif.m
% See geck:g1678142: Currently imread returns only the first local
% color table (if present).
% Using the first local color table to read all the frames later
% shows distorted frames of the image.
%
% In order to fix this issue, rescale the frames (starting from second
% frame) such that it works correctly when rendered using the first
% local color table.
%
% Algorithm:
% 1. Save the transparent pixels of the current frame.
% 2. Rescale the non-transparent current frame such that it renders
% properly with the first local color map (if one exist).
%
% To rescale just the non-transparent pixels, do the following:
% a. Rescale the current frame using the first local color map
% b. Restore the transparent pixels from step 1 and apply
% the background color or the previous frame's transparent
% color.
Correspondingly, the specific conditionals causing the behavior follow in each case section:
if hasLocalColorTable
% If the current frame's color table and previous's frame
% color is different then convert the current frame such
% that it can render correctly using the first color table.
% See geck g167814
current_frame = transformCurrentFrame(current_frame,current_info,map);
end
Truthfully enough, a better way to have handled this is already described in the same file:
% This only returns the color table for the first frame in the GIF file.
% There is an enhancement (g1055265) to make this return all color tables.
Long story short, it appears the issue history can be summarized:
  • A legitimate problem: imread() routinely confuses people because it doesn't directly return all necessary color tables
  • A problematic solution: instead of making imread() return all color tables, imread now forcibly remaps the image to use a single wrong color table
To be fair, the flexibility of the GIF format spec is going to force the developers of imread to make some compromises for the sake of how images are expected to be handled within Matlab (e.g. a 4D image requires the frames to have consistent geometry). Simply dumping all original LCTs may run into complications with how optimized GIFs need to be essentially composited into a 4D image (i.e. if imread is intended to coalesce the image).
This copypasta was from the version included in R2019b. As of R2020b, some comments have been removed and the conditional call to the remapping function moved outside the switch-case structure, but the behavior is essentially unchanged.
Demonstrate how this can fail:
If frame 1 uses a CT optimized only for the color content of frame 1, there is no good reason to presume that it can represent the color content throughout the remaining frames. For a demonstration of how this presumption will fail, consider the following test image:
This is a simple 100x100 GIF with 12 frames of solid color. The file has a unique GCT defined, but not used by any of the frames. Each frame has a unique LCT, each with only a single nonzero color tuple occupying a unique position in the LCT. The uniqueness of the GCT and LCTs help with identifying things if you decide to import it yourself. Even though your browser is already showing you the image, let's inspect it so we know that what we're seeing makes sense. For the sake of assurance and the elimination of unknowns, we can do this directly instead of trying to decode it with some other image viewer (hex with some notes, ignore syntax highlighting):
% header
% version info = 'GIF89a'
% size = [100 100]
% packed field 0xF3 = 0b11110011
% 1 = global color table flag (a GCT follows the header)
% 111 = color resolution (0b111+1) = 8 bits per CT entry
% 0 = sort flag
% 011 = size of GCT is 2^(0b011+1) = 2^4 = 16 entries
% background color index = 0 (zero-based indexing)
% pixel aspect ratio = 0 (no AR specified)
47 49 46 38 39 61 64 00 64 00 F3 00 00
% global color table (defined, but unused and distinct)
% none of these colors are present in any LCT
% and should not show up in the decoded image
11 00 00
00 11 00
00 00 11
22 00 00
00 22 00
00 00 22
33 00 00
00 33 00
00 00 33
44 00 00
00 44 00
00 00 44
55 00 00
00 55 00
00 00 55
66 00 00
% application extension
21 FF 0B 4E 45 54 53 43 41 50 45 32 2E 30 03 01 00 00 00
% FRAME 1
% graphical control extension
% 4 bytes in this GCE
% packed field 0x00 = 0b00000000
% 000 = reserved
% 000 = disposal method (do not specify)
% 0 = user input flag (no input)
% 0 = transparent index flag (none specified)
% delay time = 5/100 = 50ms
% transparent index = 0 (zero-based, unused)
21 F9 04 00 05 00 00 00
% logical image descriptor
% location = [0 0]
% size = [100 100]
% packed field 0x83 = 0b10000011
% 1 = local color table flag (this LCT supercedes the GCT for the scope of this frame)
% 0 = interlace flag
% 0 = sort flag
% 00 = reserved
% 011 = size of LCT is 2^(0b011+1) = 2^4 = 16 entries
2C 00 00 00 00 64 00 64 00 83
% local color table (only one nonzero tuple at position 1)
FF 15 E9 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
% image data
% 0x08 = 8 bit minimum code length
% 0xA1 = 161 bytes in this block
% 0x0001 = (this basically amounts to the relevant color index +1 bit, i.e. index 0)
% since these are all simple solid color frames, all blocks are identical
% except for that 4th byte containing the first relevant code in the sequence
08 A1 00 01 08 1C 48 B0 A0 C1 83 08 13 2A 5C C8 B0 A1 C3 87 10 23 4A 9C 48 B1 A2 C5 8B 18 33 6A DC C8 B1 A3 C7 8F 20 43 8A 1C 49 B2 A4 C9 93 28 53 AA 5C C9 B2 A5 CB 97 30 63 CA 9C 49 B3 A6 CD 9B 38 73 EA DC C9 B3 A7 CF 9F 40 83 0A 1D 4A B4 A8 D1 A3 48 93 2A 5D CA B4 A9 D3 A7 50 A3 4A 9D 4A B5 AA D5 AB 58 B3 6A DD CA B5 AB D7 AF 60 C3 8A 1D 4B B6 AC D9 B3 68 D3 AA 5D CB B6 AD DB B7 70 E3 CA 9D 4B B7 AE DD BB 78 F3 EA DD CB B7 AF DF BF 80 03 0B 1E 4C B8 B0 E1 C3 88 13 2B 5E CC 58 70 40 00
% FRAME 2
% graphical control extension (same as frame 1)
21 F9 04 00 05 00 00 00
% logical image descriptor (same as frame 1)
2C 00 00 00 00 64 00 64 00 83
% local color table (only one nonzero tuple at position 2)
00 00 00 FF 2A D4 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
% image data (same as frame 1, but index is incremented)
08 A1 00 03 08 1C 48 B0 A0 C1 83 08 13 2A 5C C8 B0 A1 C3 87 10 23 4A 9C 48 B1 A2 C5 8B 18 33 6A DC C8 B1 A3 C7 8F 20 43 8A 1C 49 B2 A4 C9 93 28 53 AA 5C C9 B2 A5 CB 97 30 63 CA 9C 49 B3 A6 CD 9B 38 73 EA DC C9 B3 A7 CF 9F 40 83 0A 1D 4A B4 A8 D1 A3 48 93 2A 5D CA B4 A9 D3 A7 50 A3 4A 9D 4A B5 AA D5 AB 58 B3 6A DD CA B5 AB D7 AF 60 C3 8A 1D 4B B6 AC D9 B3 68 D3 AA 5D CB B6 AD DB B7 70 E3 CA 9D 4B B7 AE DD BB 78 F3 EA DD CB B7 AF DF BF 80 03 0B 1E 4C B8 B0 E1 C3 88 13 2B 5E CC 58 70 40 00
% and so on...
Without completely decoding the image data blocks themselves, the image appears to correspond to what the browser is already showing us. The color tables are there as described, with flags properly set. Frame 1 is magenta(ish), subsequent frames sweeping toward yellow. The dominant index increments with each frame to correspond to the unique LCTs.
In pre-2018 versions, the behavior is as expected. Imread returns the entire image, but only the first calculated CT. We'd still need to get the remaining CTs with imfinfo().
[mypicture theonlymap]=imread('LCTdissimilarity_unused_gct.gif');
theonlymap
cross_section=permute(mypicture(1,1,:,:),[1 4 2 3])
theonlymap =
1.0000 0.0824 0.9137
0 0 0
0 0 0
0 0 0
0 0 0
0 0 0
0 0 0
0 0 0
0 0 0
0 0 0
0 0 0
0 0 0
0 0 0
0 0 0
0 0 0
0 0 0
cross_section =
0 1 2 3 4 5 6 7 8 9 10 11
In the post-2018 version, imread() will return an image in which all frames are remapped to use the CT calculated for frame 1. Obviously, LCT1 does not contain any entries that will allow yellow frames to be rendered. The result is that all frames are now magenta. This result is not the image represented by the file.
theonlymap =
1.0000 0.0824 0.9137
0 0 0
0 0 0
0 0 0
0 0 0
0 0 0
0 0 0
0 0 0
0 0 0
0 0 0
0 0 0
0 0 0
0 0 0
0 0 0
0 0 0
0 0 0
cross_section =
0 0 0 0 0 0 0 0 0 0 0 0
Attempting to use the correct color tables from the file (as correctly returned by imfinfo()) will also fail, as the indices in the image frames no longer point to the correct position in their respective tables. Simply put, image data has been lost in the import process.
Aren't there cases where the new behavior seems to work?
Yes. The damage evident in the prior example is obvious, but may be less severe for other images. Consider the following cases.
If the file has a GCT and all image frames have no LCTs defined, then the first calculated CT will be the GCT, which would logically have also been the calculated CT for all subsequent frames. In this case, the post-2018 behavior would produce correct output. As the legacy behavior would also produce correct output in this case, the change provides no benefit.
Consider an image such as the prior D6 example where all frames have similar color content. If the image frames have LCTs defined, and the LCTs are all similar, the image frames might be remapped to use the first calculated CT without severely noticeable degradation depending on the degree of similarity. While the results may appear passable in the most forgiving cases, I assert that silently altering image data needlessly is inappropriate for a technical environment like Matlab. I imagine that this second case and the ubiquity of working with grayscale images is a large part of why this behavior goes unnoticed.
Bear in mind that this remapping consists of int2rgb conversion and then a subsequent rgb2int conversion. The remapping process itself is likely to introduce new artifacts, even if the map contents are similar. To emphasize the "needlessly" part of my assertion, the remapping process as described is only possible because readgif.m already has the correct LCTs to begin with.
To demonstrate the variable impact of remapping, let's take the same file and change the content of LCT1. First, let's add all the color tuples to this one LCT. Leave the GCT and all other LCTs unchanged. Since LCT1 now represents the color content of all frames, the remapped image is perfectly fine.
But what if the overlap between LCT1 and the others is less complete? Instead of adding all the other color tuples to LCT1, add only the one for the last frame (yellow). The once-solid frames are now full of dithering and worm artifacts. You can only do so much to cover up the problems caused by using the wrong color table.
What can I do to read a multiframe GIF correctly?
Pending any patches, I can suggest two ways. One can comment out the offending conditionals in readgif.m and then restart Matlab, though this will require write permissions to the file. Multiframe GIF files could then be read using both imread() and imfinfo() as described. I am not certain that this simple fix won't allow problems with certain optimized GIFs though.
One other way would be to split the file into multiple single-frame images first and then build the arrays by reading the resulting images in a loop. One way would be to use ImageMagick:
convert mymultiframeimage.gif %03d_singleframeimage.gif
This splitting method is an option in my own gifread() tool (FEX submission #52514). I added the option as a workaround for an older bug (see https://www.mathworks.com/matlabcentral/answers/231702-reading-animated-gif-with-imread-produces-unexpected-results), though it works for this bug as well. Of course it won't work if you don't have ImageMagick, and the way I wrote it doesn't work on Windows.
If you're stuck on a lab computer without write permissions or the ability to install things, it's likely that neither of these options will work, leaving you to find some other way to split the file manually. If anyone knows of a better workaround, feel free to share it.
  2 Comments
Stephen23
Stephen23 on 21 Apr 2023
Edited: Stephen23 on 21 Apr 2023
"I assert that silently altering image data needlessly is inappropriate for a technical environment like Matlab."
Agreed.
Very impressive analysis, by the way. I would give this one hundred votes, if I could.
Just out of curiousity: what happens if the input idx is specified?:
https://www.mathworks.com/help/matlab/ref/imread.html#btnczv9-1-idx
Does IMREAD then use the idx-th frame and LCT, or does it still perform that ugly rigmarole involving the 1st LCT?
DGM
DGM on 21 Apr 2023
Edited: DGM on 21 Apr 2023
Ran in:
Since it's going to try to coalesce the image, it's going to have to read all the frames (or at least the frames up to the one requested).
% the first example with unused GCT and unique LCTs
fname = 'https://www.mathworks.com/matlabcentral/answers/uploaded_files/428998/image.gif';
% try to read frame 1 (magenta)
[frame map] = imread(fname,1);
frame(1,1) % index should be framenumber-1 = 0
ans = uint8 0
% try to read frame 12 (yellow)
[frame map] = imread(fname,12);
frame(1,1) % index should be framenumber-1 = 11
ans = uint8 0
So even when requesting a specific frame, the problem remains.

Sign in to comment.

More Answers (0)

Categories

Find more on Images in Help Center and File Exchange

Tags

Products


Release

R2019b

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!