Variables Unrecognized in parfor but Work in for Loop

I have a struct variable with which I extract each variable using 'eval()'. I then call those variables in a parfor loop and I recieve an error (provided below). Notably, if I define the variable explicitly and don't use 'eval()', the parfor loop works fine... Why is this the case and are there any fixes? I would rather not extract each variable from the struct manually in my code because there are many variables and it would make the code look unorderly.

I would appreciate any suggestions. Thank you!

"Error using *FUNCTION NAME* (line 26)

The source code (*FILE PATH*) for the

parfor-loop that is trying to execute on the worker could not be found.

Caused by:

Unrecognized function or variable 'x0'.

Worker unable to find file.

Unrecognized function or variable 'x0'."

The simplified code is provided as follows:

function [output] = *FUNCTION NAME*(params)
% Start Parallel processing
parpool('local',8)
% Extract variables from the struct
fields = fieldnames(params); 
for i = 1:length(fields)
    eval([fields{i} ' = params.' fields{i} ';']); % Extract variable from struct (i.e. attain x0)
end
parfor j = 1:NumTrials
    [x, z] = truth_gen(x0);
    *ADDITIONAL CODE*
end
*ADDITIONAL CODE*
end

EDIT: Upon a more rigorous search, it appears that using 'eval' is bad practice! Instead I will save my variables to a mat file and reload them: https://www.mathworks.com/matlabcentral/answers/299135-is-it-possible-to-extract-all-fields-from-a-structure-automatically#answer_231447

However, for my own knowledge, I am still curious why the creation of variables (via 'eval') does not work in parfor.

EDIT 2: It seems saving the variables in a mat file and loading them still causes the same error. Can someone explain why?

Answer 1

[Matt J]

You should not be using eval to create new variables, regardless of whether the loop is parfor or a regular for loop, see also,

TUTORIAL: Why Variables Should Not Be Named Dynamically (eval)

Use dot-indexing expressions params.fieldname to reference the variables. If you prefer numeric indices instead of field names, move the data to a different container, like a cell array, e.g.,

params.a=1:3; params.b=4:5;
s=struct2cell(params),

s = 2x1 cell array
    {[1 2 3]}
    {[  4 5]}

[Matt J]

However, for my own knowledge, I am still curious why the creation of variables (via 'eval') does not work in parfor.

If you use eval instead of explicit code to create variables, the parfor parser cannot see how the variable was created, which means it cannot know how to classify the variables into the necessary categories decribed here,

https://www.mathworks.com/help/coder/ug/classification-of-variables-in-parfor-loops.html

[Aneesh]

"Use dot-indexing expressions params.fieldname to reference the variables"

Thank you for the response and explanations. I am aware of this, but I was hoping to avoid writing 'params.' to reduce clutter in my code. Though, I understand now there is a classifcation issue when using 'eval'. I wish there was a way to notice this in my workspace/command window when I was debugging.

[Matt J]

I wish there was a way to notice this in my workspace/command window when I was debugging.

For me, your code produces the Code Analyzer warning below.

[Aneesh]

Matt, the eval is NOT within the parfor in the code I provided, so that is not the issue.

[Edric Ellis]

Matt's explanation is exactly correct. When you create variables using eval ahead of the parfor-loop, those variables are not present in the text of your program. To ensure efficient operation, parfor sends to the workers only those variables it knows for sure are used in the body of the loop. It works this out using static analysis. So, because there is no statement inside your function assigning a value directly to x0, parfor assumes that it does not need to send that variable into the body of the loop. (Further, it assumes that x0 will turn out to be a function).

Any means of dynamically creating variables without explicit assignment statements to those variables (i.e. using load without an output argument, or assignin, or eval, or evalin, ...) will hit the same restriction.

This section of the documentation https://uk.mathworks.com/help/parallel-computing/troubleshoot-variables-in-parfor-loops.html#bvlk718-1 is referring to this limitation.

[Mohammad Sami]

Is there a reason why you can't directly reference the params variable as follows ?

    [x, z] = truth_gen(params.x0);

[Matt J]

Matt, the eval is NOT within the parfor in the code I provided, so that is not the issue.

Yes, I see now that you do not get a warning, but that is one of the reasons eval is discouraged. The Code Analyzer's ability to analyze what eval is doing is limited.

EDIT 2: It seems saving the variables in a mat file and loading them still causes the same error. Can someone explain why?

It depends how you are using load(). If you are returning an output from load(), there should be no problem,

params=load(___)

Otherwise, you are creating variables non-explicitly in the same manner as eval() and run into the same problem. For that reason, using load() without an output argument is generally discouraged, for the same reasons as eval is discouraged.

[Aneesh]

I did not know that by doing 'eval' or 'load' I was dynamically creating variables, I simply thought this was a normal operation of assigning variables in my workspace. Thank you Matt and Edric, your replies have been very informative!

[Stephen23]

"I did not know that by doing 'eval' or 'load' I was dynamically creating variables, I simply thought this was a normal operation of assigning variables in my workspace."

For an interpreted language there might not be much difference. But because the MATLAB Engine uses static code analysis to optimize and compile code before it is run there is a very big difference between them:

https://www.mathworks.com/products/matlab/performance.html

https://blogs.mathworks.com/loren/2016/02/12/run-code-faster-with-the-new-matlab-execution-engine/

Performance is only one reason to avoid dynamically creating/accessing variables:

https://www.mathworks.com/matlabcentral/discussions/tips/849901-tutorial-why-variables-should-not-be-named-dynamically-eval

Answer 2

[Matt J]

I am aware of this, but I was hoping to avoid writing 'params.' to reduce clutter in my code.

You might consider this File Exchange download,

https://www.mathworks.com/matlabcentral/fileexchange/26216-structure-fields-to-variables

It doesn't avoid the need to explicitly unpack the struct, but it reduces the manual effort of writing code to do so, and can also automatically layout the statements in a way that condenses the code. Example, suppose you have a 30-field struct,

args=cell(2,30); 
args(1,:)=cellstr("x"+(1:30));
p=struct(args{:}) %struct with 30 fields

p = struct with fields:
     x1: []
     x2: []
     x3: []
     x4: []
     x5: []
     x6: []
     x7: []
     x8: []
     x9: []
    x10: []
    x11: []
    x12: []
    x13: []
    x14: []
    x15: []
    x16: []
    x17: []
    x18: []
    x19: []
    x20: []
    x21: []
    x22: []
    x23: []
    x24: []
    x25: []
    x26: []
    x27: []
    x28: []
    x29: []
    x30: []

Then you can auto-generate the unpacking statements by doing as below and copy/pasting the output text into your code.

columns=6;
structvars(columns,p) %copy-paste the output into your code

 
x1 = p.x1;     x6 = p.x6;       x11 = p.x11;     x16 = p.x16;     x21 = p.x21;     x26 = p.x26;     
x2 = p.x2;     x7 = p.x7;       x12 = p.x12;     x17 = p.x17;     x22 = p.x22;     x27 = p.x27;     
x3 = p.x3;     x8 = p.x8;       x13 = p.x13;     x18 = p.x18;     x23 = p.x23;     x28 = p.x28;     
x4 = p.x4;     x9 = p.x9;       x14 = p.x14;     x19 = p.x19;     x24 = p.x24;     x29 = p.x29;     
x5 = p.x5;     x10 = p.x10;     x15 = p.x15;     x20 = p.x20;     x25 = p.x25;     x30 = p.x30;     
 

[Aneesh]

This is a very interesting alternative, thank you!

[Matt J]

You are welcome, but if you now consider your questions resolved, please Accept-click the answer that best addresses your question.