parfor chuncks: are loop iterations running in parallel WITHIN a chunck on local machine?

I know that parfor does load-balancing to break the loop iterations into chuncks which are then sent to the workers.

Demo example:

parpool;
% parpool.NumWorkers = 5
parfor i=1:100
    %do something
end

Say, the 100 iterations are divided into 5 chuncks of size 20. Then, 1:20 are maybe sent to worker 1, 21:40 are sent to worker 2, etc.

I do not take about the order in which the iterations are worked off within a chunck. So if worker 1 executes in the order

[20,19,18,....2,1] or in the random order [15,18,2,7,...] does not matter,

but what matters is that there is only one index running at a time WITHIN a worker. So what is not allowed in my case is that worker 1 runs, e.g., indices 20 and 19 simultaneously.

Can I be sure that parpool / parfor respects this or is there uncertainty?

[Harald]

Hi,

I'd say that you can count on a single worker not running iterations simultaneously.

I am curious about why this would be causing problems, though.

Best wishes,

Harald

[SA-W]

Hi Harald,

this is not a problem in general, but only for my problem setting: I perform a multistart parameter optimization and use the taskID (getCurrentTask.Id) as a file identifier. So if I run the optimization corresponding to my ith start vector on task 1, I have to be sure that there is no second start vector running at the sime time on task 1. This would mean they both write their data in the same file and the everything is unreliable.

Answer 1

[Abhijeet Singh]

Hi,

I understand that you are concerned about whether parfor guarantees that each worker will only handle one iteration at a time within its assigned chunk of iterations, irrespective of the order in which each worker processes its assigned iterations.

parfor in MATLAB is designed to ensure that each worker in the parallel pool executes only one iteration at a time. When parfor is used, it distributes the work among different workers so that each worker handles a specific part of the task. However, within each worker, the iterations are processed serially, not simultaneously. This way, each worker respects the rule of only working on one task at a time.

Please refer to the following documentation for more information on parfor.

https://www.mathworks.com/help/parallel-computing/parallel-for-loops-parfor.html

I hope this helps !!

Thanks,

Abhijeet