Using parfor on a cluster

I have 400 cores split across 20 nodes on my cluster and I am running the following command for parfor: `parfor(ifile2=ifile1:500,np2)` where np2=400. Instead of connecting to 400 workers, I get this in the output: connected to 12 workers.

Is there something wrong that is going on? I read somewhere that the 12 workers limit was only for R2013a and earlier; I am running R2016a (9.0.0.341360) 64-bit (glnxa64).

Answer 1

[Kojiro Saito]

By default, preferred number of workers are defined as 12, so you need to change this value to much larger by "Parallel" -> "Parallel Preferences" -> "Preferred number of workers in a parallel pool".

[Karthik Ganeshan]

Thanks for the reply. I have to submit this job to the cluster, so I don't get the matlab GUI. I use this command to execute the program: matlab -r -nodesktop -nosplash "programName". Is there a way I can set the preferences through some command in the script instead?

[Kojiro Saito]

You can change this value by editing parallel.settings in prefdir. In Linux machine, prefdir of R2016a would be /home/USERNAME/.matlab/R2016a. You can check this directory by do "prefdir" command in MATLAB.

It is editable by vi, vi ~/.matlab/R2016a/parallel.settings

then, change the value of PreferredNumWorkers from 12 to higher in the following part.

<key name="PreferredNumWorkers">
     <double>
         <value>12</value>
     </double>
</key>