Daily values to monthly sums

Very silly issue I'm having, clearly it's Friday and my brain is dead!

I have yearly data (column 1 below is year, col 2 is DoY, col 3 is what I want, col 4 is a running ref #)

1999  88  -0.153775166948929  88
  1999  89  0.0645182041737496  89
  1999  90  0.305720459521316  90
  1999  91  0.608267678784863  91
  1999  92  -0.105377378067329  92
  1999  93  0.164670217562409  93
  1999  94  0.486056906589846  94
  1999  95  -0.153775166948929  95
  1999  96  0.0645182041737496  96
  1999  97  0.305720459521316  97
  1999  98  0.608267678784863  98
  1999  99  -0.0388209349360338  99
  1999  100  -0.0388209349360338  100
  1999  101  -0.220461941261943  101
  1999  102  -0.0576523900004068  102
  1999  103  0.896360903826591  103
  1999  104  0.842649481758915  104
  1999  105  -0.347869165344918  105

What I want to do is pull out monthly totals of column 3. I wrote some indices:

%Monthly indexes
iJan=(1:1:31);
iFeb=(32:1:59);
iFebl=horzcat(iFeb,29);
iMar=(60:1:90);
iApr=(91:1:120);
iMay=(121:1:151);
iJun=(152:1:181);
iJul=(182:1:212);
iAug=(213:1:243);
iSep=(244:1:273);
iOct=(274:1:304);
iNov=(305:1:334);
iDec=(335:1:365);

I need monthly values for every month in my dataset. I keep on writing loops that do very silly things, please save my brain from friday frustration!

[Cedric]

What is the purpose of iFeb1? Day 29 is clearly in January ..

[Kate]

iFebl is leap year february :)

Answer 1

[Cedric]

Well, TGIF anyway! Assuming that iFeb1 is irrelevant (if not, the solution can be updated).

 % - Vector of months start days.
 monthsStart = [1,32,60,91,121,152,182,213,244,274,305,335] ;

 % - Build lookup table day/month: e.g. LT(47) is month ID for day 47.
 LT = zeros( 365, 1 ) ;
 LT(monthsStart) = 1 ;
 LT = cumsum( LT ) ;

 % - Build vector of month ID for data entries.
 monthId = LT(data(:,2)) ;

 % - Accumulate using month ID as index.
 sums = accumarray( monthId, data(:,3), [12,1] ) ;

PS: did you have any more questions about my answer to your question here?

[Kate]

Thanks Cedric, you're right about Fridays!

We're really close, but one small issue. I'm essentially using this code to process hundreds of different sites, all with different lengths of data, and each month for each year must have it's own monthly summed value. (see my code below for how I deal with loading variable data). I really like accumarray and didn't know about it before. Is there a way to use my 'years' variable to accumulate monthly values for each year?

Also thanks for your help on the other question. I sometimes make life too complicated and appreciate your help. Sorry that I forgot to accept your answer!

function [site, numyr, year_start, year_end, years, sitedata]=call_up_append
% Flux Anom Extraction
%Step #1
%Loading in one site-year and appending data to single matrix for multiple
% years.

%load files
clc;
clear all;
pwd
fuf(pwd, 'detail');
fn=fuf(['/Users/kate/Documents/Research/MATLAB/flux_sensitivity_analysis/flux_data/EBF/Br-Cax/*daily*.mat'],'detail');
load(char(fn{1}));
%Extract variables names
pieces = regexp(fn, '\/', 'split');
p=pieces{1};
site = p{12};
q = regexp(site, '\.', 'split');
sitename = q{1};
clear p
clear q
%set up length of data
numyr=length(fn);

year_start=str2num(q{2});

year_end=year_start+numyr-1;

years=year_start:1:year_end;

load first year

v=([sitename,'.',int2str(years(1)),'.synth.daily.mat']);

eval(['load ' v]);

sitedata=data_d;
%append data
clear v
for i=2:numyr
 %  Create file names to load
    v=([sitename,'.',int2str(years(i)),'.synth.daily.mat']);
    eval(['load ' v]);

    % Append data to full matrix

      sitedata=[sitedata;data_d];
      clear v
  end
%save 
save('Br-Cax', 'sitedata')
end

[Cedric]

If you understood my answer with ACCUMARRAY (let me know if it's not fully the case) the update for managing years will be really simple, because the first argument of indices can have as many columns as there are dimensions in the accumulator.

This means that you can do something like..

 yearId = data(:,1) - min( data(:,1) ) + 1 ;
 nYear  = range( data(:,1) ) + 1 ;

for creating year IDs starting at 1, and then

 sums = accumarray( [monthId,yearId], data(:,3), [12,nYear] ) ;

for building a 2D array with one row per month and one column per year.

Note that if you had huge gaps between years, e.g. 1920, 1960, 2000, you could build a look up table for mapping the 81 years over year IDs 1,2,3 instead of 1:81.

Let me know if you need more information.

Answer 2

[Chad Greene]

Alternatively, downsample_ts does this quite simply.