MATLAB Answers

Video preview has growing delay when used in uifigure, but not in normal figure

22 views (last 30 days)
Hey there,
I am building an app which includes axes to display a preview of my camera (gentl adaptor). I created the axes with uiaxes.
My problem is, that the video is really laggy and the delay grows the longer the preview is running. When I stop the preview, it keeps running until it displays the last recorded frame. The memory usage of matlab is also steadily increasing while it is running.
If I try the example of a custom preview GUI everything runs smoothly. But that one does not create a axis object at all.
I also tried it this way:
ax = axes
im = image(ax, zeros(1542, 2064,3 ));
axis(ax, 'image')
vid = videoinput('gentl')
preview(vid, im)
And this also works fine.
I then tried replacing the uiaxes object in my app with an axes object (same parent), but then it does not show.
Is there a way to not use the uiaxes, because I think that thats the issue?
I also noticed, that the uiaxes do not use as much GPU as the axes, if thats useful.
This is my code (was created with app designer and then ported to normal matlab code in an attempt to fix the lag):
classdef Preview_Window < handle
% Properties that correspond to app components
properties (Access = public)
UIFigure matlab.ui.Figure
Menu matlab.ui.container.Menu
Menu2 matlab.ui.container.Menu
GridLayout matlab.ui.container.GridLayout
LeftPanel matlab.ui.container.Panel
GridLayout2 matlab.ui.container.GridLayout
ResetCameraInterfaceButton matlab.ui.control.Button
PreviewPanel matlab.ui.container.Panel
GridLayout4 matlab.ui.container.GridLayout
StartButton matlab.ui.control.Button
StopButton matlab.ui.control.Button
RightPanel matlab.ui.container.Panel
UIAxes matlab.ui.control.UIAxes
Timestamp matlab.ui.control.Label
% Properties that correspond to apps with auto-reflow
properties (Access = private)
onePanelWidth = 576;
properties (Access = private)
feed; % camera feed
src; %getsource(feed)
canvas; %image handle to show the preview on
methods (Access = private)
function preview_fcn(app, ~, event, himage)
% callback of preview()
% display some additional information
% Get timestamp for frame.
tstampstr = event.Timestamp;
framerate = event.FrameRate;
%status = event.Status;
resolution = event.Resolution;
% Set the value of the text label.
app.Timestamp.Text = strcat(tstampstr, " | " , num2str(framerate), " | ", resolution);
%app.Status.Text = strcat( "| ", status, " | :", resolution);
% Display image data.
himage.CData = event.Data;
function reset_camera(app)
% setup camera and interface
app.ResetCameraInterfaceButton.Enable = false;
% preset axes with black picture
app.canvas = image(app.UIAxes, zeros(1542, 2064, 3) );
colormap(app.UIAxes, gray);
axis(app.UIAxes, 'image');
%reset camera interface and try to open camera feed
app.feed = videoinput("gentl", 1 , 'Mono8');
warndlg("Can't open Camera feed. Fix and reset interface!", "Warning!")
%Settings for camera
triggerconfig(app.feed, 'manual');
app.feed.FramesPerTrigger = 1;
app.src = getselectedsource(app.feed);
app.src.ReverseY = 'True';
app.src.ExposureTime = 60000;
% Setup preview container
frame = getsnapshot(app.feed);
vidRes = app.feed.VideoResolution;
nBands = app.feed.NumberOfBands;
app.canvas = image(app.UIAxes, zeros(vidRes(2), vidRes(1), nBands));
app.UIAxes.XLim = [0, double(vidRes(1))];
app.UIAxes.YLim = [0, double(vidRes(2))];
pbaspect(app.UIAxes, [vidRes(1),vidRes(2),1]);
app.canvas.CData = frame;
% activate custom preview callback
setappdata(app.canvas, 'UpdatePreviewWindowFcn', @app.preview_fcn);
% enable buttons
app.StartButton.Enable = true;
app.ResetCameraInterfaceButton.Enable = true;
% Callbacks that handle component events
methods (Access = private)
% Code that executes after component creation
function startupFcn(app)
app.UIAxes.Title.String = '';
app.UIAxes.XLabel.String = '';
app.UIAxes.YLabel.String = '';
app.UIAxes.Visible = 'off';
app.UIAxes.XTick = [];
app.UIAxes.YTick = [];
app.UIAxes.PlotBoxAspectRatioMode = 'manual';
% Changes arrangement of the app based on UIFigure width
function updateAppLayout(app, event)
currentFigureWidth = app.UIFigure.Position(3);
if(currentFigureWidth <= app.onePanelWidth)
% Change to a 2x1 grid
app.GridLayout.RowHeight = {480, 480};
app.GridLayout.ColumnWidth = {'1x'};
app.RightPanel.Layout.Row = 2;
app.RightPanel.Layout.Column = 1;
% Change to a 1x2 grid
app.GridLayout.RowHeight = {'1x'};
app.GridLayout.ColumnWidth = {220, '1x'};
app.RightPanel.Layout.Row = 1;
app.RightPanel.Layout.Column = 2;
% Button pushed function: StartButton
function StartButtonPushed(app, event)
app.StopButton.Enable = true;
app.StartButton.Enable = false;
%app.src.ExposureAuto = 'Continuous';
preview(app.feed, app.canvas);
% Close request function: UIFigure
function UIFigureCloseRequest(app, event)
% Button pushed function: StopButton
function StopButtonPushed(app, event)
app.StopButton.Enable = false;
app.StartButton.Enable = true;
% Button pushed function: ResetCameraInterfaceButton
function ResetCameraInterfaceButtonPushed(app, event)
% Component initialization
methods (Access = private)
% Create UIFigure and components
function createComponents(app)
% Create UIFigure and hide until all components are created
app.UIFigure = uifigure('Visible', 'off');
app.UIFigure.AutoResizeChildren = 'off';
app.UIFigure.Position = [100 100 640 480];
app.UIFigure.Name = 'MATLAB App';
app.UIFigure.CloseRequestFcn = @(obj, event)app.UIFigureCloseRequest;
app.UIFigure.SizeChangedFcn = @(obj, event)app.updateAppLayout;
% Create Menu
app.Menu = uimenu(app.UIFigure);
app.Menu.Text = 'Menu';
% Create Menu2
app.Menu2 = uimenu(app.UIFigure);
app.Menu2.Text = 'Menu2';
% Create GridLayout
app.GridLayout = uigridlayout(app.UIFigure);
app.GridLayout.ColumnWidth = {220, '1x'};
app.GridLayout.RowHeight = {'1x'};
app.GridLayout.ColumnSpacing = 0;
app.GridLayout.RowSpacing = 0;
app.GridLayout.Padding = [0 0 0 0];
app.GridLayout.Scrollable = 'on';
% Create LeftPanel
app.LeftPanel = uipanel(app.GridLayout);
app.LeftPanel.Layout.Row = 1;
app.LeftPanel.Layout.Column = 1;
% Create GridLayout2
app.GridLayout2 = uigridlayout(app.LeftPanel);
app.GridLayout2.RowHeight = {'1x', '1x', '1x', '1x', '1x', '1x', '1x', '1x'};
app.GridLayout2.RowSpacing = 25;
% Create ResetCameraInterfaceButton
app.ResetCameraInterfaceButton = uibutton(app.GridLayout2, 'push');
app.ResetCameraInterfaceButton.ButtonPushedFcn = @(obj, event)app.ResetCameraInterfaceButtonPushed;
app.ResetCameraInterfaceButton.Enable = 'off';
app.ResetCameraInterfaceButton.Layout.Row = 6;
app.ResetCameraInterfaceButton.Layout.Column = 1;
app.ResetCameraInterfaceButton.Text = {'Reset Camera'; 'Interface'};
% Create PreviewPanel
app.PreviewPanel = uipanel(app.GridLayout2);
app.PreviewPanel.Title = 'Preview';
app.PreviewPanel.Layout.Row = [1 3];
app.PreviewPanel.Layout.Column = 1;
% Create GridLayout4
app.GridLayout4 = uigridlayout(app.PreviewPanel);
app.GridLayout4.ColumnWidth = {'1x'};
% Create StartButton
app.StartButton = uibutton(app.GridLayout4, 'push');
app.StartButton.ButtonPushedFcn = @(obj, event)app.StartButtonPushed;
app.StartButton.Enable = 'off';
app.StartButton.Layout.Row = 1;
app.StartButton.Layout.Column = 1;
app.StartButton.Text = 'Start';
% Create StopButton
app.StopButton = uibutton(app.GridLayout4, 'push');
app.StopButton.ButtonPushedFcn = @(obj, event)app.StopButtonPushed;
app.StopButton.Enable = 'off';
app.StopButton.Layout.Row = 2;
app.StopButton.Layout.Column = 1;
app.StopButton.Text = 'Stop';
% Create RightPanel
app.RightPanel = uipanel(app.GridLayout);
app.RightPanel.Layout.Row = 1;
app.RightPanel.Layout.Column = 2;
% Create UIAxes
app.UIAxes = uiaxes(app.RightPanel);
title(app.UIAxes, 'Title')
xlabel(app.UIAxes, 'X')
ylabel(app.UIAxes, 'Y')
app.UIAxes.PlotBoxAspectRatio = [1 1.02770083102493 1];
app.UIAxes.Box = 'on';
app.UIAxes.Position = [6 28 408 425];
% Create Timestamp
app.Timestamp = uilabel(app.RightPanel);
app.Timestamp.Position = [6 1 408 22];
app.Timestamp.Text = '';
% Show the figure after all components are created
app.UIFigure.Visible = 'on';
% App creation and deletion
methods (Access = public)
% Construct app
function app = Preview_Window
% Create UIFigure and components
% Execute the startup function
if nargout == 0
clear app
% Code that executes before app deletion
function delete(app)
% Delete UIFigure when app is deleted
Thanks in advance for any help!
  1 Comment
Raphael Kriegl
Raphael Kriegl on 20 Aug 2020
After further trial and error, I could replace the uiaxes with axes, but that did not fix the issue with the delay.

Sign in to comment.

Accepted Answer

Raphael Kriegl
Raphael Kriegl on 25 Aug 2020
Edited: Raphael Kriegl on 4 Sep 2020
I got an answer from the MATLAB support team:
After collaborating with my colleagues I figured that this is a known limitation with the "uiaxes" performance in terms of rendering the images. Currently, "uiaxes"/"uifigure" (javascript based) does not support high frame rates like the traditional MATLAB figure window (java based) and the frames passed to the "uiaxes" are queued for rendering, this is causing the memory increase they noticed. For the same reason, it will cause a lag which worsens with larger images and high acquisition frame rates if the acquisition is running for a long duration .
Until the performance bottleneck in the "uiaxes" is addressed, there is little we can do to help you, if you goal is to have the live preview running in the app.
Having said that, for the time being, I would suggest you to open the preview in traditional MATLAB figure window outside of the app and i believe you have already tried this.
Another thing to experiment is to turn the imaqmex's "slowpreview" flag which currently sets a max limit of 10 frames for previewing. However, with the "uiaxes" rendering performance even 10 FPS might be high. Following is the code to enable slow preview.
>> imaqreset
>> imaqmex('feature', '-slowpreview', 'true')
>> vid = videoinput(...)Please note that this is an undocumented feature and it could change and/or be removed in the future.
Additionally, lowering the resolution and/or ROI can help as this decreases the load on the "uifigure".

More Answers (1)

Evan Bates
Evan Bates on 4 Sep 2020
Raphael, thank you for posting that response from Matworks. That is a huge help. I just ran into this issue as well. I’m glad the problem is not on my end. They should add functionality in the preview function to set the playback framerate.
I found a workaround to this issue. I made a UpdatePreivewWindowFcn and inside the callback function I added a pause(1/60). The preview function will dump frames if the UpdatePreivewWindowFcn has not finished executing, therefore slowing the video to the desired framerate (frame rate is set by the time in the pause).
Hope this helps

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!