parforOptions

Options set for parfor

collapse all in page

Syntax

opts = parforOptions(cluster)
opts = parforOptions(pool)
opts = parforOptions(___,Name,Value)

Description

opts = parforOptions(cluster) creates a set of options for parfor using the cluster object cluster. To specify options for a parfor-loop, use the parfor (loopVar=initVal:endval, opts); statements; end syntax.

example

opts = parforOptions(pool) creates a set of options for parfor using the pool object pool.

When you create multiple pools, use this syntax to specify which pool to run the parfor-loop on.

Tip

When you run a parfor-loop, MATLAB® automatically uses a parallel pool to run the loop, if one is available.

If you only need to run a parfor-loop using your default cluster profile or an available parallel pool, consider using the parfor loopVar=initVal:endval; statements; end syntax instead of using parforOptions.

example

opts = parforOptions(___,Name,Value) creates a set of options for parfor using one or more name-value arguments. For example, use parforOptions(pool,"MaxNumWorkers",M) to run a parfor-loop using the pool object pool and a maximum of M workers. Specify name-value arguments after all other input arguments.

example

Examples

collapse all

Open Live Script

Create a cluster object using the parcluster function, and create a set of parfor options with it. By default, parcluster uses your default cluster profile. Check your default profile on the MATLAB® Home tab, in Parallel > Select Parallel Environment.

cluster = parcluster;
opts = parforOptions(cluster);

To run parfor computations directly in the cluster, pass the parfor options as the second input argument to parfor.

When you use this approach, parfor can use all the available workers in the cluster, and workers become available as soon as the loop completes. This approach is also useful if your cluster does not support parallel pools.

values = [3 3 3 7 3 3 3];
parfor (i=1:numel(values),opts)
    out(i) = norm(pinv(rand(values(i)*1e3)));
end

Use this syntax to run parfor on a large cluster without consuming workers for longer than necessary.

Open Live Script

You can control how parfor divides iterations into subranges for the workers with parforOptions. Controlling the range partitioning can optimize performance of a parfor-loop. For best performance, try to split into subranges that are:

  • Large enough that the computation time is large compared to the overhead of scheduling the subrange

  • Small enough that there are enough subranges to keep all workers busy

To partition iterations into subranges of fixed size, create a set of parfor options, set 'RangePartitionMethod' to 'fixed', and specify a subrange size with 'SubrangeSize'.

opts = parforOptions(parcluster,'RangePartitionMethod','fixed','SubrangeSize',2);

Pass the parfor options as the second input argument to parfor. In this case, parfor divides iterations into three groups of 2 iterations.

values = [3 3 3 3 3 3];
parfor (i=1:numel(values),opts)
    out(i) = norm(pinv(rand(values(i)*1e3)));
end

To partition iterations into subranges of varying size, pass a function handle to the 'RangePartitionMethod' name-value pair. This function must return a vector of subrange sizes, and their sum must be equal to the number of iterations. For more information on this syntax, see RangePartitionMethod.

opts = parforOptions(parcluster,'RangePartitionMethod', @(n,nw) [2 1 1 2]);

Pass the parfor options as the second input argument to parfor. In this case, parfor divides iterations into four groups of 2, 1, 1, and 2 iterations.

values = [3 3 7 7 3 3];
parfor (i=1:numel(values),opts)
    out(i) = norm(pinv(rand(values(i)*1e3)));
end
Open Live Script

You can use parforOptions to run parfor on the workers of a parallel pool. Use this approach when you want to reserve a fixed number of workers for the parfor-loop. You can also have finer control on how parfor divides iterations for workers.

Create a parallel pool using the parpool function. By default, parpool uses your default profile. Check your default profile on the MATLAB Home tab, in Parallel > Select Parallel Environment. Create a set of parfor options with the parallel pool object, and specify options. For example, specify subranges of fixed size 2 as the partitioning method.

p = parpool;
Starting parallel pool (parpool) using the 'Processes' profile ...
Connected to the parallel pool (number of workers: 6).

opts = parforOptions(p,'RangePartitionMethod','fixed','SubrangeSize',2);

Pass the parfor options as the second input argument to the parfor function. parfor runs the loop body on the parallel pool and divides iterations according to opts.

values = [3 3 3 3 3 3];
parfor (i=1:numel(values),opts)
    out(i) = norm(pinv(rand(values(i)*1e3)));
end
Open Live Script

When you run parfor with or without a parallel pool, by default, MATLAB performs an automatic dependency analysis on the loop body. MATLAB transfers required files to the workers before running the statements. In some cases, you must explicitly transfer those files to the workers. For more information, see Identify Program Dependencies.

If you are using parfor without a parallel pool, use parforOptions to transfer files. Create a cluster object using the parcluster option. Create a set of parfor options with the cluster object using the parforOptions function. To transfer files to the workers, use the 'AttachedFiles' name-value pair.

cluster = parcluster;
opts = parforOptions(cluster,'AttachedFiles',{'myFile.dat'});

Pass the parfor options as the second input argument to the parfor function. The workers can access the required files in the loop body.

parfor (i=1:2,opts)
    M = csvread('myFile.dat',0,2*(i-1),[0,2*(i-1),1,1+2*(i-1)]);
    out(i) = norm(rand(ceil(norm(M))*1e3));
end

Input Arguments

collapse all

Cluster, specified as a parallel.Cluster object. To create a cluster object, use parcluster.

Example: parcluster('Processes');

Pool, specified as a parallel.Pool object.

Example: parpool('Processes');

Example: backgroundPool;

Name-Value Arguments

expand all

Specify optional pairs of arguments as Name1=Value1,...,NameN=ValueN, where Name is the argument name and Value is the corresponding value. Name-value arguments must appear after other arguments, but the order of the pairs does not matter.

Before R2021a, use commas to separate each name and value, and enclose Name in quotes.

Example: opts = parforOptions(cluster,"AttachedFiles","myFile.dat");

All Object Types

expand all

Method for partitioning iterations into subranges, specified as "auto", "fixed", or a function handle. A subrange is a contiguous block of loop iterations that parfor runs as a group on a worker. Use this argument to optimize the performance of your parfor-loop by specifying how iterations are distributed across workers.

  • If RangePartitionMethod is "auto" or if you do not specify a value, parforOptions divides the parfor-loop iterations into subranges of varying sizes to seek good performance for a variety of parfor-loops.

  • If RangePartitionMethod is "fixed", parforOptions divides the parfor-loop iterations into subranges of fixed sizes. When you use this method, you must also use the SubrangeSize name-value argument to specify the subrange sizes.

  • If RangePartitionMethod is a function handle, parforOptions uses the function handle to divide parfor-loop iterations into subranges of fixed sizes.

    The function runs the function handle as sizes = customFcn(n,nw).

    • n is the number of iterations in the parfor-loop.

    • nw is the number of workers available to run the loop.

      When you use a pool to run the loop, nw is the number of workers in the parallel pool. When you use a cluster to run the loop without a pool, nw is the NumWorkers property of the cluster.

    • sizes is an integer vector of subrange sizes. For any value of n and nw, the sum of the vector sizes must be equal to n.

Example: parforOptions(cluster,"RangePartitionMethod","auto")

Example: parforOptions(cluster,"RangePartitionMethod",@(n,nw) ones(1,n))

Maximum number of iterations in a subrange, specified as a positive integer scalar. A subrange is a contiguous block of loop iterations that parfor runs as a group on a worker.

When you use this argument, you must specify the RangePartitionMethod argument as "fixed".

Example: parforOptions(cluster,"RangePartitionMethod","fixed","SubrangeSize",5)

Cluster Name-Value Arguments

expand all

Folders to add to the MATLAB search path of each worker running the parfor-loop, specified as a character vector, string scalar, string array, or cell array of character vectors.

The default value is an empty cell array.

The folders are added to the search path of the workers when you run the parfor-loop. When the parfor-loop finishes, these folders are removed from the search path of the workers.

If the client and workers have different paths to the same folder, specify the folder using the path on the workers. For example, if the path to the folder is /shared/data on the client and /organization/shared/data on the workers, specify "/organization/shared/data".

If you specify relative paths such as "../myFolder", the paths are resolved relative to the current working directory on the workers.

Specify AdditionalPaths to avoid copying files unnecessarily from the client to workers. Specify AdditionalPaths only when the files are available on the workers. If the files are not available, use AttachedFiles to send files to the workers.

Example: opts = parforOptions(cluster,"AdditionalPaths",["/additional/path1","/additional/path2"])

Files and folders to send to each worker running the parfor-loop, specified as a character vector, string scalar, string array, or cell array of character vectors.

The default value is an empty cell array.

The files and folders are sent to workers when you run the parfor-loop. When the parfor-loop finishes, these files and folders are removed from the file system of each worker.

If you specify relative paths such as "../myFolder", the paths are resolved relative to the current working directory on the client.

If the files are available on the workers, specify AdditionalPaths instead. When you specify AdditionalPaths, you avoid copying files unnecessarily from the client to workers.

Flag to send client path to workers, specified as true or false.

If you specify AutoAddClientPath as true, the user-added entries are added to the path of each worker when you run the parfor-loop. When the parfor-loop finishes, these entries are removed from the path of each worker.

Flag to copy files to workers automatically, specified as true or false.

When you offload computations to workers, any files that are required for computations on the client must also be available on the workers. If you specify AutoAttachFiles as true, the client attempts to automatically detect and attach such files. If you specify AutoAttachFiles as false, you turn off automatic detection on the client. If automatic detection cannot find all the files, or if sending files from client to worker is slow, use the following arguments.

  • If the files are in a folder that is not accessible on the workers, specify the files using the AttachedFiles argument. The cluster copies each file you specify from the client to workers.

  • If the files are in a folder that is accessible on the workers, you can use the AdditionalPaths argument instead. Use the AdditionalPaths argument to add paths to the MATLAB search path of each worker and avoid copying files unnecessarily from the client to workers.

Automatically detected files are sent to workers when you run the parfor-loop. When the parfor-loop finishes, these files and folders are removed from the file system of each worker.

Pool Name-Value Arguments

expand all

Maximum number of workers, specified as a positive integer scalar.

The default value is Inf.

  • If you specify MaxNumWorkers as a finite positive integer, your parfor-loop will run with a maximum of MaxNumWorkers workers.

  • If you specify MaxNumWorkers as Inf, your parfor-loop will run with as many workers as are available.

Extended Capabilities

expand all

Version History

Introduced in R2019a

See Also

| |

Topics