Loop only storing last run in nested for loop ... should be easy indixing easy for the experienced eye

This is a bit complex to explain without an essay but in point form, I have:

A large matrix with two parameters for 6 heights I want to bin the data by one parameter and then store this data in a struct for each bin I then was to do this for each height

The data is arranged in the following order date x y x2 y2 x3 y3 ... where the different levels represent the parameters at different heights.

Here is the code I have thus for but the loop is making the structure of the struct (names etc) perfectly but not iteratively saving the data:

inpdata = tower_wind_GF_allyrs;

inpdata = inpdata(~any(isnan(inpdata),2),:);
inpdata = inpdata(~any(isinf(inpdata),2),:);

bin = 0:0.51444:40;%maxws; 
names = strtrim(cellstr(num2str([1:(length(bin)-1)]'))');
names = strrep(strcat('bin',names),'.','');
names2 = {'height10m' 'height20m' 'height40m' 'height80m' 'height120m' 'height200m'};
ind = cell(6,(length(bin)-1));

for k = linspace(2,12,6)
    for l = linspace(3,13,6)
        for i = 1:(length(bin)-1)
            for varname = 1:length(names2)
                search = find(inpdata(:,k) >= bin(i) & inpdata(:,k) < bin(i+1)); 
                ind{varname,i} = search
                binned_wind_allyrs.(names2{1,varname}).(names{1,i}) = cell(1,(length(bin)-1)); 
                binned_wind_allyrs.(names2{1,varname}).(names{1,i}) = [inpdata(ind{varname,i}(:,1),k),inpdata(ind{varname,i}(:,1),l)];
            end
        end
    end
end

Would LOVE to figure this out. Thanks in advance

Answer 1

[Kelly Kearney]

You could simplify the syntax quite a bit using histc and accumarray:

inpdata = [datenum(2014,1,1)+(1:100)' rand(100,12)*39];
nheight = (size(inpdata,2)-1)/2;

bin = 0:0.51444:40;
nbin = length(bin);

xdata = inpdata(:,2:2:end);
ydata = inpdata(:,3:2:end);

% Bin the data

[n, idx] = histc(xdata, bin);
idx(idx == nbin+1) = nbin;

bindata = cell(nbin, nheight);
for ii = 1:nheight
    x = accumarray(idx(:,ii), xdata(:,ii), [nbin 1], @(x) {x}); 
    y = accumarray(idx(:,ii), ydata(:,ii), [nbin 1], @(x) {x}); 
    bindata(:,ii) = cellfun(@(x,y) [x y], x, y, 'uni', 0);
end

% Structure output

bname = cellstr(num2str((1:nbin)', 'bin%02d'));
hname = {'height10m' 'height20m' 'height40m' 'height80m' 'height120m' 'height200m'};

for ib = 1:nbin
    for ih = 1:nheight
        S.(bname{ib}).(hname{ih}) = bindata{ib,ih};
    end
end

[mashtine]

Thank you Kelly! This works great with the dataset you constructed but when I implement the script with mine I get an error with accumarry:

Error using accumarray
First input SUBS must contain positive integer
subscripts.

Something is wrong with the site and it won't let me upload a sample file although it is way smaller than 5 MB. There aren't any negative values in the dataset.

[mashtine]

Also, idx nor xdata or data have any negative values, there mins are all positive.

[Kelly Kearney]

Are there any 0's in the idx array? This will happen if any of your data points fall outside your bins. I notice that the bins as you define them don't quite span the 0-40 range that I think you wanted (there's a gap from 39.612 to 40), so perhaps that's the cause?

[mashtine]

Hi Kelly,

Sorry for the late reply, I was away. You are correct, there will be gaps in the bins and some bins will be empty for instance (arbitrary numbers) bin 5-6 may have members but 7-8 and 9-10 won't but 11-12 will. This will leaving some bins as empty arrays. Is there some way around this problem? I am not familiar with accumarray in this regard. Your code clearly works but my dataset seems to be the issue.

[mashtine]

Here is a link to download my dataset as I cannot upload here. Really appreciate the help!

https://drive.google.com/file/d/0B1W1LHdueUapaHJ0eEdwWDl4Vk0/edit?usp=sharing

[Kelly Kearney]

Empty bins are not a problem. But data points that don't fit into a bin are. You have NaNs in your dataset, and histc assigns those data points to bin zero. So you'll have to decide what you want to do with those data points. This modification to the code above throws out any points where the x-data is NaN (it seems you have several data points with a valid x but NaN y; those points are left alone).

bindata = cell(nbin, nheight);
for ii = 1:nheight

      isn = isnan(xdata(:,ii));
      xtmp = xdata(~isn,ii);
      ytmp = ydata(~isn,ii);

      x = accumarray(idx(~isn,ii), xtmp, [nbin 1], @(x) {x}); 
      y = accumarray(idx(~isn,ii), ytmp, [nbin 1], @(x) {x}); 
      bindata(:,ii) = cellfun(@(x,y) [x y], x, y, 'uni', 0);
  end

[mashtine]

Sorry for the late reply Kelly. Thank you very much for this!

Have a great week

Answer 2

[Iain]

Simple answer is on the left hand side, you need to index with k, l, i and varname.

So, this ought to work:

 binned_wind_allyrs(k,l,i,varname) = {[inpdata(ind{varname,i}(:,1),k),inpdata(ind{varname,i}(:,1),l)]};

But you might need

 binned_wind_allyrs(k,l,i,varname, 1:(numel(bin)-1)) = {[inpdata(ind{varname,i}(:,1),k),inpdata(ind{varname,i}(:,1),l)]};

[mashtine]

Like this you mean:

binned_wind_allyrs(k,l,i,varname, 1:(numel(bin)-1)).(names2{1,varname}).(names{1,i}) = [inpdata(ind{varname,i},k),inpdata(ind{varname,i},l)];

When I put this, I am getting the following:

Scalar index required for this type of multi-level indexing.