Save and load variables of different samples

I do experiments with different samples (let's say 10 different materials). After measuring all samples seperatly, I apply a Matlab Code(the same code for all samples) to the data of each sample. The code does some calculations and in the end I have some variables. These variables I would like to load into another Matlab Code which is suppose to plot some of the variables (e.g. comparison plot for different samples, etc.)

Now the question: How can I save and load the variables in a way to access the variables of different samples (e.g. plot variable 1 of sample 1 with variable 1 of sample 2)?

My idea was to add all the variables to a structure named after the sample and save the structure (e.g water.density, water.color, etc. and then save('filename.mat','-struct','water') ). But the problem is that I am not able to name the structure efficiently (I need to change the code every time). Is there a better approach for this problem?

[Stephen23]

"But the problem is that I am not able to name the structure efficiently (I need to change the code every time). Is there a better approach for this problem?"

Yes: do not force meta-data (e.g. sample numbers, parameter values, etc) into variable names. Meta-data is data, and data should be stored in variables. Once you do that, then you can easily loop over your saved files and process them.

[Marcel345614]

Many thanks!

This means, I should include the sample name in the first field of the structure (e.g. structure.name, structure.variable1, etc.)? Then I save this structure in the first code.

How can I proceed in the second code after I have load all the structures (now all named 'structure')?

[Stephen23]

"This means, I should include the sample name in the first field of the structure (e.g. structure.name, structure.variable1, etc.)? Then I save this structure in the first code."

I don't understand how "first" relates to anything, but yes, in general you should save any meta-data together with all of the other data for that sample/test, e.g.:

P = 'absolute or relative path to where the files are saved';
S.density = 2.3
S.color = [255,192,203];
S.name = 'water';
F = sprintf('myfile_%03d.mat',k);
save(fullfile(P,F),'-struct','S')

"How can I proceed in the second code after I have load all the structures (now all named 'structure')?"

If all of the mat files contain exactly the same variable names (which they should) then this is easy:

https://www.mathworks.com/help/matlab/import_export/process-a-sequence-of-files.html

For example, something like this:

S = dir(fullfile(P,'*.mat'));
for k = 1:numel(S)
    F = fullfile(P,S(k).name);
    S(k).data = load(F);
end
A = [S.data]; % 1xN structure of all file data

You can trivially access the content of the structure array A, for example:

A(1).color
A(1).name
A(3).density

https://www.mathworks.com/help/matlab/matlab_prog/access-multiple-elements-of-a-nonscalar-struct-array.html

[Marcel345614]

Many thanks for this good answer!

But one point is not fully clear to me: Do I need to "remember" which index number belongs to which sample? Suppose I want to plot density of water and density of milk in one plot, I need to remember that water was index k=3 and milk was k=5?

What happens when I have some more samples the other day? Is milk still k=5 or suddenly k=6?

[Marcel345614]

Note: I need to call A(1).S.name instead of A(1).name...

[Stephen23]

"Do I need to "remember" which index number belongs to which sample? Suppose I want to plot density of water and density of milk in one plot, I need to remember that water was index k=3 and milk was k=5? "

It would be more reliable to use the (meta-)data that you have saved in the file. For example:

X = strcmpi('water',{A.name});
A(X).color
A(X).density

and the same for 'milk'.

"I need to call A(1).S.name instead of A(1).name... "

Yes, we need to use the syntax that saves the structure as separate variables. The correct syntax for saving is:

save(fullfile(P,F),'-struct','S')
%                  ^^^^^^^^^ you need this!

[Marcel345614]

I still not see a handy way with the indices: I have hunderds of samples and it is difficult to assign a fixed index number to every sample in a logical way because sometimes new samples are measured.

The samples belong somehow to different groups: To proceed with the allegory let's say, I have water and milk and for some reason I want to measure some variables for different salt concentrations in the two fluids, e.g. I have

-milk 0g/l

-milk 10g/l

-milk 15g/l

-water 0g/l

-water 10g/l

(the above lines are the samples and for each of these samples I have the same variables, e.g. conductivity, color etc.)

Now in the code in which I read in all the data, I would like to access the conductivity of 'milk 0g/l' in a easy way and plot it. This is is easy as long as I have the same samples. But suppose I measure the next week apple juice for different salt concentration. Now all the index numbers are shifted and it would be very time consumming to change the code to again simply plot the conductivity of 'milk 0g/l'.

Your solution with strcmpi is not very handy for many hunderd samples and a lot of plots...

Do you see my problem? Is there an easier solution for this?

[Stephen23]

"I would like to access the conductivity of 'milk 0g/l' in a easy way and plot it. This is is easy as long as I have the same samples. But suppose I measure the next week apple juice for different salt concentration."

Sure, just store all meta-data as data in their own right, which will make processing your data much easier.

"Now all the index numbers are shifted and it would be very time consumming to change the code to again simply plot the conductivity of 'milk 0g/l'."

"Very time consuming" is exactly what I am trying to help you to avoid.

Keep your name data and concentration data separate (names should be character/categorical, concentration should be numeric, etc). Then you can perform simple loops, generate permutations, match data in vectors (i.e. get indices), or however you want to pick the samples and groups that you want, e.g.

https://www.mathworks.com/help/matlab/ref/discretize.html

and if you used a table:

https://www.mathworks.com/help/matlab/matlab_prog/split-table-data-variables-and-apply-functions.html

"Is there an easier solution for this?"

Probably you should be using a table, which is specifically designed for working with groups within sets of data:

https://www.mathworks.com/help/matlab/matlab_prog/create-a-table.html

[Marcel345614]

Thank you very much for your help! But I still have some problems in understanding how to apply this to practice.

Could you give me a short code sniplet of how I could solve the following problem:

I would like to plot the conductivity for the sample 'milk 10g/l' over time (Suppose I have measured the conductivity for each sample over time).

In the simple case that I know that the sample 'milk 10g/l' corresponds to the index k=3, it is very easy:

plot(A(3).time,A(3).conductivity)

But how can I plot the same thing in the case I do not now which index belongs to the sample 'milk 10g/l' or in the case in which I add samples to the structure A again and again and the indices are thus shiftet each time (and I would need to change the number in the plot command each time)? How can I easily do this without writing a lot of code(In the real experiment I have a lot of samples and want to plot way more than simply the conductivity for one sample)?

[Marcel345614]

The only idea I came up with is to use something like this:

%%%% Figures of Project %%%%%
%% Figure 1
X1 = strcmpi('milk 10g/l',{A.name});
plot(A(X1).time,A(X1).conductivity)

%% Figure 2: Comparison plot of conductivity in milk for different salt concentrations
X1=strcmpi('milk 0g/l',{A.name});
X2=strcmpi('milk 10g/l',{A.name});
X3=strcmpi('milk 20g/l',{A.name});
plot(A(X1).time,A(X1).conductivity)
plot(A(X2).time,A(X2).conductivity)
plot(A(X3).time,A(X3).conductivity)

%% Figure 3: Comparison of conductivity in different materials
X1=strcmpi('milk 0g/l',{A.name});
X2=strcmpi('apple juice 0g/l',{A.name});
X3=strcmpi('water 0g/l',{A.name});
plot(A(X1).time,A(X1).conductivity)
plot(A(X2).time,A(X2).conductivity)
plot(A(X3).time,A(X3).conductivity)

Is this the most elegant way?

[Stephen23]

"Is this the most elegant way"

It looks like a good start to me. The real questions are:

• does it do what you want?
• is it easy for you to understand and maintain?

As mentioned in a previous comment, and for the reasons explained in more detail here

https://www.mathworks.com/matlabcentral/answers/225435-save-variable-as-string-from-user-input

https://www.mathworks.com/matlabcentral/answers/56124-eval-is-evil-using-variables-created-dynamically-info-retrieval#answer_67978

it would be best to not put lots of (particularly numeric) meta-data together into one string. You will regret that later.

"But how can I plot the same thing in the case I do not now which index belongs to the sample 'milk 10g/l' or in the case in which I add samples to the structure A again and again and the indices are thus shiftet each time (and I would need to change the number in the plot command each time)?. How can I easily do this without writing a lot of code(In the real experiment I have a lot of samples and want to plot way more than simply the conductivity for one sample)?"

Take a deep breath. Let the code do these things for you. Do not write lots of code (that is what loops are for).

In essence, everything boils down to indexing.

So you want to plot milk vs coffee: then use strcmp (or whatever) to get the indices of those names. And then loop over all concentrations (stored as numeric, of course) and plot each pair of corresponding data sets. Make sure your code can cope with exceptions (missing pairs, no matches, etc). How you define those combinations and sets of data is up to you: e.g. if you want to do the same for milk vs a whole set of other things (coffee, agave syrup, diesel) then loop over those names. Or perhaps use ndgrid to generate all combinations.

Use indexing to select the relevant data, plot, and save (you will find sprintf and fullfile useful). Done.

Answer 1

[Xingwang Yong]

You could try cell(). You do not have to save the variable names.

c = cell(1,1); % each col contains variables of a sample, each row is a variable in diffent samples
c{1,1}='water';
c{1,2}='oil';
c{1,3}='salt';

for k = 1:numSamples
    c{end+1,k} = var1;
    c{end+1,k} = var2;
end

[Marcel345614]

This is not exactly what I want. The problem is that I apply the first code on different samples. I want to store the resulting variables and load them into another code for plotting.

[Xingwang Yong]

You can save the cell array then. The key idea here is that you can use index of cell, i.e. c{1,1}, to get variables instead of structure.filed. So you can get rid of the name of samples.