How do I define HANTS matlab parameters?

I am using HANTS MATLAB code for the NDVI time series. I have defined the parameters as follows;

function [yOut, amp, phi]=ApplyHants(y,nb,nf,fet,dod,HiLo,low,high,delta)

nb = 22 ; total number of images

nf = 3 frequency above zero frequency

fet = 5

dod = 5

HiLo = 'none'

low = -1.0 ; minimum value of NDVI

high = 1.0 ; maximum value of NDVI

delta = 0.1

but it is giving output as zero values.

Please suggest me how to define these above mentioned parameters for my NDVI time series of one year time period having 22 images.

I would appreciate your kind help.

Gauri

[Manikanta Aditya]

If you’re getting zero values as output, it might be due to the fet and dod parameters. These parameters control the fit of the harmonic function to the time series data. If the fit error tolerance (fet) is too high or the degree of overdeterminedness (dod) is too low, the function might not fit the data well, resulting in zero values.

Here are some suggestions:

1. Try reducing the fet value. A smaller fet means the function will try to fit the data more closely.
2. Increase the dod value. A higher dod means the function will use more observations to fit the harmonic function, which might result in a better fit.

[gauri]

Hi Aditya,

Thanks for suggestions. I tried it with fet as 1 and 0.1 and dod as 9 but still the output is zero. Can you please give me some more suggestions?

Gauri

[Manikanta Aditya]

nb = 22; % This should be set to the total number of images/observations in your time series, which is 22 in your case.
nf = 4; % This parameter determines the number of harmonic components used to fit the time series. For an annual time series with 22 images, a value of 3 or 4 is typically recommended. 
fet = 1.5; % This controls how closely the model fits the data. A smaller value will lead to a tighter fit, but may also overfit the noise. Start with a value around 1 or 2 and adjust based on the results.
dod = 8; % This determines the number of observations used for each fitted value. A higher value will make the fit more robust to outliers but may also smooth out real variations. For an annual time series with 22 images, a value between 6 and 10 is generally recommended.
HiLo = 'both'; % This parameter determines whether the HANTS algorithm should force the fitted values to stay within the specified range. Since NDVI values are bounded between -1 and 1, you can set HiLo = 'both' to ensure that the fitted values remain within this range.
low = -1; % Set these to -1 and 1, respectively, to match the expected range of NDVI values.
high = 1;
delta = 0.1; % This parameter controls the amount of smoothing applied to the fitted values. A value of 0.1 is a reasonable starting point.

[yOut, amp, phi] = ApplyHants(y, nb, nf, fet, dod, HiLo, low, high, delta);

Try these parameters can check once.. Tried to explain in the code snippet.

If you still get zero values as output, it may be an issue with the input data (y) or the implementation of the HANTS algorithm. In that case, you may need to double-check your input data and the HANTS code for any potential issues.

[gauri]

Thank you Aditya for your support. It did not work. The header file generated by applyHants is as follows

ENVI

description = {Exported from MATLAB}

samples = 22

lines = 1315

bands = 1153

data type = 4

interleave = bsq

So here bands and samples are interchanged. I am puuting a part of applyHants matlab which reads the parameters as follows

function [yOut, amp, phi]=ApplyHants(y,nb,nf,fet,dod,HiLo,low,high,delta)

if (max(size(size(y)))~=3)

error('Input data must be three dimensional [time,lat,lon]')

end

[ni,ny,nx]=size(y);

yOut= zeros(ni,ny,nx,'single');

amp = zeros(nf+1,ny,nx,'single');

phi = zeros(nf+1,ny,nx,'single');

ts=1:ni;

I layer stacked the 22 files in image as follows

image = (22, 1315, 1153)

and passed it to applyHants as follows

[yOut, amp, phi] = ApplyHants(image, nb, nf, fet, dod, HiLo, low, high, delta);

I request you to please suggest me how to make them compatable to each other so that code works properly.

Once again thank you so much for your valuable comments.

Gauri

[gauri]

Can I once again request you to please have a look on it and suggest me some solution?

Thanks a lot for your valuable comments.

Gauri

Answer 1

[Manikanta Aditya]

Hi @gauri

Please consider these as my suggestations from my research and exploration and try to see if you can resolve the issue. Might not completely fix the issue.

From your description, it seems like the dimensions of your input data might not be matching the expected dimensions in the 'ApplyHants' function. The function expects the input data to be three-dimensional in the form [time, lat, lon], but your data seems to be in the form [bands, lines, samples].

You can try transposing your data to match the expected dimensions.

Here’s how you can do it:

% Transpose the dimensions
image = permute(image, [3 2 1]);

% Now pass the transposed image to the function
[yOut, amp, phi] = ApplyHants(image, nb, nf, fet, dod, HiLo, low, high, delta);

This will rearrange your data dimensions to match the expected [time, lat, lon] format. Please try this and let me know if it helps.

By using permute(image, [1, 3, 2]), you are rearranging the dimensions of the image array to [time, samples, lines], which should match the expected order [time, lat, lon] for the ApplyHants function.

After making this change, the ApplyHants function should work as expected, and you should get the desired output instead of zero values.

[gauri]

Thank you very much for your help. I incorporated your suggested code and it worked very well. However, the first band of output image is non zero values while rest of the bands contains zero values.

I am very grateful to you for your kind suggestions and help.

Gauri

[gauri]

The first band of output image is non zero values while rest of the bands contains zero values. Therefore, I need to work further to make it workable.

Gauri

[Manikanta Aditya]

Okay, hope the issue gets fixed sooner.

[gauri]

I am getting following error;

Error using HANTS (line 45)

Not enough data points.

part of code which generate this error is as follows;

if nout > noutmax

if any(abs(y - fill_val) < eps)

ready = true;

yr = y;

outliers(:) = fill_val;

else

error('Not enough data points.');

end

else

ready = false(size(y));

end

I request you to please suggest me how to overcome this error.

Gauri

[Manikanta Aditya]

The error "Not enough data points." is likely occurring because the HANTS algorithm is not able to find enough valid data points (i.e., non-outliers) to fit the model. This can happen when there are too many missing values or outliers in the input data.

To overcome this error, you may need to adjust the input parameters or preprocess your data to reduce the number of missing values or outliers.

1. Check your input data
2. Adjust the fet and dod parameters
3. Preprocess your data
4. Check the fill_val value

if nout > noutmax
    if any(abs(y - fill_val) < eps)
        % If all data points are equal to fill_val (missing/outlier value)
        ready = true(size(y));
        yr = fill_val; % Set output to fill_val
        outliers(:) = fill_val; % Mark all as outliers
    else
        % Check if there are enough non-missing/non-outlier data points
        valid_points = abs(y - fill_val) >= eps;
        if sum(valid_points(:)) < min_valid_points
            % Not enough valid data points
            error('Not enough data points. Consider adjusting the parameters or preprocessing your data.');
        else
            % Proceed with the algorithm
            ready = false(size(y));
        end
    end
else
    ready = false(size(y));
end

[gauri]

I set min_valid_points =15

Then following error occured.

Operands to the logical AND (&&) and OR (||) operators must be convertible to logical scalar values. Use the ANY or ALL functions to reduce operands to logical scalar values.

Error in HANTS (line 75)

while any(~ready) && (nloop < nloopmax)

Gauri

[Manikanta Aditya]

The error you're encountering is related to the way MATLAB handles logical operations with arrays. In MATLAB, the && and || operators expect scalar inputs or arrays of the same size. However, the condition any(~ready) returns a single logical value (true or false), which is incompatible with the array ready in the expression any(~ready) && (nloop < nloopmax).

To fix this issue, you can replace the logical AND operation (&&) with the element-wise AND operation (&), and wrap the condition with the any function, like this:

while any(~ready & (nloop < nloopmax)) 
    % ... rest of the loop code ...
end

This way, the element-wise AND operation (&) is performed between the negated ready array (~ready) and the condition nloop < nloopmax, and then the any function checks if any element in the resulting logical array is true.

Alternatively, you can restructure the condition using && and any separately, like this:

while (any(~ready)) && (nloop < nloopmax)  
    % ... rest of the loop code ...  
end

In this case, the any(~ready) expression returns a single logical value, which can be used with the && operator along with the scalar condition nloop < nloopmax.

[gauri]

Thank you very much. It has overcome that error. However, a new error has occured from the following code;

% Initialize matrices

mat = zeros(min(2*nf+1, ni), ni);

p = ones(size(y));

bool_out = (y < low) | (y > high);

p(bool_out) = 0;

outliers(bool_out) = 1;

nout = sum(p == 0);

Arrays have incompatible sizes for this operation.

Error in HANTS (line 80)

za = mat .* (p .* y);

I request you to please suggest me how to overcome this.

Gauri

Answer 2

[John]

The error "Arrays have incompatible sizes for this operation" is likely due to a mismatch in the dimensions of the arrays involved in the element-wise multiplication operation

mat .* (p .* y)

In the HANTS code, the line

mat = zeros(min(2*nf+1, ni), ni);

initializes a matrix mat with dimensions

(min(2*nf+1, ni), ni)

. However, the dimensions of the arrays p and y are not explicitly specified in the code snippet you provided. To resolve this issue, you need to ensure that the dimensions of mat , p , and y are compatible for element-wise operations.

Here are a few steps you can try: Check the dimensions of your input data y by using the size function:

[ni, ny, nx] = size(y); 

Ensure that the dimensions of mat match the expected dimensions based on the input data y .

The first dimension of mat should be

min(2*nf+1, ni)

, and the second and third dimensions should match the dimensions of y . You can modify the initialization of mat as follows:

mat = zeros(min(2*nf+1, ni), ny, nx);

Initialize the array p with the same dimensions as y :

p = ones(ni, ny, nx);

Modify the line that creates the logical array bool_out to ensure that the dimensions match:

bool_out = (y < low) | (y > high); 

Update the line that assigns values to p based on bool_out :

p(bool_out) = 0; 

Update the line that calculates nout :

nout = sum(p == 0, 'all');

After making these changes, the dimensions of mat , p , and y should be compatible, and the element-wise multiplication operation

mat .* (p .* y) 

should work correctly. If you still encounter issues, provide more information about the dimensions of your input data y and the specific error message you receive.