Parfor waitbar : How to do this more cleanly?

I'm a frequent casual user of parfor loops. Getting a waitbar (progress meter) from parallel code used to involve having to have each thread append to a file to record progress, and there are a number of contributions on the file exchange that do this. It was kinda silly, but it was necessary.

Now that the DataQueue system exists, it should be possible to do this more sensibly. So I went looking for a good waitbar on the File Exchange. And I couldn't find one, so I tried to make my own.

The code below works nicely, but it depends on using three globals - which isn't great, because if those variable names are being used by something else, it's gonna break stuff. Can anybody suggest a better way to do this, preferably not involving globals? Persistent variables should work within update.m, but it's getting the info (waitbar handle and expected number of interations) from create.m to update.m that's stumped me.

There are three functions, all part of the ParWaitbar package. Usage is explained in the first one:

create.m:

function [ q ] = create( n, txt )
%FNPARWAITBAR Produces a waitbar object that works with parfor.
% Relies on globals. Not sure how to avoid this.
%   Input:  
%           n: Number of iterations expected
%           txt: Text for the waitbar dialog.
%   Output: q: handle for the dataqueue
%
%   Usage:  [ q ] = ParWaitbar.create( 10, "Predicting earthquakes..." );
%           parfor i=1:10
%               pause(rand*5); (or do useful stuff)
%               send( q, i ); % doesn't matter what's sent (i here). We're just
%                               counting the items rcvd.
%           end
%           ParWaitbar.cleanup(q);

global wb; %waitbar handle.
global wb_N; % number of iterations completed
global wb_max; %number of iterations expected

q = parallel.pool.DataQueue;
wb = waitbar( 0, txt );
wb_max = n;
wb_N = 0;

afterEach( q, @ParWaitbar.update );

end

update.m:

function update(~)

global wb; %shoudl have waitbar handle.
global wb_N; % number of iterations completed
global wb_max; %number of iterations expected

wb_N = wb_N + 1;
waitbar( wb_N / wb_max, wb );

end

cleanup.m

function cleanup( q )
% FIXME we're left with q afterwards. Presumably a fn can't delete
% something outside its own scope

global wb; %shoudl have waitbar handle.
global wb_N; % number of iterations completed
global wb_max; %number of iterations expected

close(wb);
delete(q);
clear wb wb_N wb_max q

end

(as an aside, the fact that we're left with q leftover as a "handle to deleted DataQueue" isn't ideal. But that's far less of a problem than the use of globals with fixed names.)

Answer 1

[Edric Ellis]

Hm, I've been meaning to tidy up my work-in-progress parallel.pool.DataQueue pool waitbar for quite a while. Here's roughly what it would look like. This uses a handle object that gets copied to the workers, so there's no need for global state.

classdef PoolWaitbar < handle
    properties (SetAccess = immutable, GetAccess = private)
        Queue
        N
    end
    properties (Access = private, Transient)
        ClientHandle = []
        Count = 0
    end
    properties (SetAccess = immutable, GetAccess = private, Transient)
        Listener = []
    end

    methods (Access = private)
        function localIncrement(obj)
            obj.Count = 1 + obj.Count;
            waitbar(obj.Count / obj.N, obj.ClientHandle);
        end
    end
    methods
        function obj = PoolWaitbar(N, message)
            if nargin < 2
                message = 'PoolWaitbar';
            end
            obj.N = N;
            obj.ClientHandle = waitbar(0, message);
            obj.Queue = parallel.pool.DataQueue;
            obj.Listener = afterEach(obj.Queue, @(~) localIncrement(obj));
        end
        function increment(obj)
            send(obj.Queue, true);
        end
        function delete(obj)
            delete(obj.ClientHandle);
            delete(obj.Queue);
        end
    end
end

This works with parfor, spmd, and parfeval, like this:

pw = PoolWaitbar(100, 'Example');
parfor ii = 1:20
    increment(pw)
end
spmd
    for ii = 21:40
        if labindex == 1
            increment(pw);
        end
    end
end
for ii = 41:100
    parfeval(@() increment(pw), 0);
end

[Simon]

Fabulous, thank you! I don't fully understand Matlab's OO facilities, but this seems to do nicely.

Is there any reason to do increment(pw) instead of pw.increment? The latter feels more intuitive to me for using an object's method, but I don't know what normal is in Matlab for this

Are you planning to put this on File Exchange?

[Edric Ellis]

This PoolWaitbar does use a slightly subtle trick to make things work - the use of Transient properties stops the client-side handle being sent to the workers.

There's no real difference between increment(pw) and pw.increment - but the former is the usual function-call style that tends to be used by MathWorks documentation and example code.

I would like to put this on File Exchange - but it's part of a larger suite of similar utilities, and there's some polishing required...

[Julia]

Thank you so much for this solution! Works really nicely! Even in 2025, there doesn't seem to be an in-built Matlab function for waitbars in parfor yet (?). So Edric's code is all the more valuable!

I attempted a slight extension of the code, adding a starting and an end value for the bar plus a publically available handle. The publically available handle allows to access the bar from outside the parfor loop, in case it's needed before or after. I include a testing script at the end that illustrates how this works. It's not entirely beautiful (sorry for that - have never worked with classes before!) but seems to work and adds functionality.

On a side note: Personally, I think the in-built waitbar() in general lacks that capacity to programmatically access the current bar length, which would allow to increase the bar length incrementally (i.e. get current value, add something and set again).

This is the modified code, based on the one posted by Edric Ellis above:

classdef PoolWaitbar < handle
    % SOURCE: Posted by Edric Ellis on 10 June 2019 on
    % https://uk.mathworks.com/matlabcentral/answers/465911-parfor-waitbar-how-to-do-this-more-cleanly
    % Accessed, copied and modified by JB 11/04/2025
    
    % Syntax: pw = PoolWaitbar(parfor_increments, 'text of waitbar', start_value_before_parfor, end_value_after_parfor);
    % The added public handle 'ClientHandlePublic' allows accessing the object
    % from outside a parfor loop as well.
    
    properties (SetAccess = immutable, GetAccess = private)
        Queue
        N
        startvalue      % JB 11/04/2025
        endvalue        % JB 11/04/2025
    end
    
    properties (Access = private, Transient)
        ClientHandle = []
        Count = 0
    end
    
    properties (SetAccess = immutable, GetAccess = private, Transient)
        Listener = []
    end
    
    properties
        ClientHandlePublic
    end
    
    methods (Access = private)
        function localIncrement(obj)
            obj.Count = 1 + obj.Count;
            current_value = obj.startvalue + (obj.Count/obj.N)*(obj.endvalue-obj.startvalue);   % JB 11/04/2025
            waitbar(current_value, obj.ClientHandle);                                           % modified JB 11/04/2025
        end
    end
    
    methods
        function obj = PoolWaitbar(N, message, startvalue, endvalue)
            if nargin < 2
                message = 'PoolWaitbar';
            end
            obj.N = N;
            obj.startvalue = startvalue;    % JB 11/04/2025
            obj.endvalue = endvalue;        % JB 11/04/2025
            obj.ClientHandle = waitbar(startvalue, message);
            obj.ClientHandlePublic = obj.ClientHandle;
            obj.Queue = parallel.pool.DataQueue;
            obj.Listener = afterEach(obj.Queue, @(~) localIncrement(obj));
        end
        
        function increment(obj)
            send(obj.Queue, true);
        end
        
        function delete(obj)
            delete(obj.ClientHandle);
            delete(obj.Queue);
        end
    end
    
end

Use this to see how it works:

%% Initialise normal waitbar
num_iter_parfor = 10;               % Expected iterations during parfor loop.
start_barlength_parfor = 0.2;
end_barlength_parfor = 0.7;

% Just initialising the bar
pw = PoolWaitbar(num_iter_parfor, 'This could have a title but we reassign it below', start_barlength_parfor, end_barlength_parfor);

%% Make it a generally accessible wait bar for a while
pw1 =  pw.ClientHandlePublic;
waitbar(0, pw1, 'Normal waitbar - wait for while')
pause(1)
waitbar(0.1, pw1, 'Wait some more.')
pause(1)
waitbar(0.2, pw1, 'About to change to parfor.')
pause(1)
waitbar(0.2, pw1, 'Now title required during parfor loop.')

%% During parfor loop, use 'increment' to increase the bar as intially defined
parfor ii = 1:num_iter_parfor
    pause(rand()*2)
    increment(pw)
end

%% Return to normal waitbar
waitbar(0.7, pw1, 'Back to normal waitbar.')
pause(2)
waitbar(0.8, pw1, 'Further progress...')
pause(0.5)
waitbar(0.85, pw1, 'Further progress...')
pause(0.5)
waitbar(0.9, pw1, 'Further progress...')
pause(0.5)
waitbar(0.95, pw1, 'Further progress...')
pause(0.5)
waitbar(1, pw1, 'Finished.')

%% Close the waitbar window
pause(2)
close(pw1)
disp('done')