Minimizing/optimizing a function containing multiple arrays?

Hello all,

I'm relatively new to MATLAB as my lab uses it for various data analysis. As such please bear with me . I have data that has waterlines in it. See figure 1 for the unaltered data.

Figure 1

To acheive this data, I simply subtract solvent signal from the signal I measured with InP nanocrystals in it.

SSUP_Y(:,n)=FTIR_y(:,n)-Solvent_y(:,n);

The Jagged stuff above are from waterlines see Figure 2.

My research advisor simply suggested doing water line removal by eye, which is incredibly tedious, and results in a lot of lines of code that look like this if I am removing water lines from multiple files.

 fiftyfivenowaterup=SSUP_Y_b(:,4)-0.35*waterline_d(:,2);

This is an issue because I collect as many as 50 data files a day and I am not about to try and sit around all day changing scalars on all of them.

The result is a cleaner spectra here is an example in figure 3

figure 3

I would like to mitigate the tediousness of changing the scalar infront of the waterline array and adjusting by eye, by writing a script.

Someone in my lab suggested using fminsearch. However, I don't think fminsearch works with functions that contain arrays, and I have gotten an error stating

Here is an outline of the code I would like to write

1. load InP nanocrystals data, load solvent data, load waterline data convert to arrays
2. Set up initial conditions in row vector v , write list of temperatures in the row vector temperature array, choose the temperature of the data I want to analyze
3. Import function waterlinesubtraction(heres some pseudocode)

 function [waterlinesubtraction]=optimizeWaterlines(FTIR_heating_y,Solvent_heating_y,Waterline_path,temperature,temperaturearray, v )
 collumn=find(temperaturearray==temperature);
 waterlinesubtraction=FTIR_heating_y(:,collumn)+v(1)*Solvent_heating_y(:,collumn)+v(2)*Waterline_y;
 end

4. Use some magical optimize command that probably doesn't exist to find coefficients v(1) and v(2) for waterline subtraction.

5. Shove those coefficients into an array where they will be easily searchable.

I'm sorry if this is impossible or if this question is beyong the course of this website.

[Mathieu NOE]

hello

I wonder if the figure 3 curve can be obtained by simply smoothing out the curve of fig 1 with movmean or something similar ?

Answer 1

[UDAYA PEDDIRAJU]

Hi cbisted,

1. Define a function: This function subtracts a scaled version of the solvent and waterline signals from your data (FTIR_heating_y) based on two scaling factors.
2. Write an error function (optional): This function calculates how "bad" the subtraction is. A common approach is to find the average squared difference between the result and your desired outcome (e.g., smoother data). You can skip this step if you have a clear idea of the good outcome visually.
3. Optimize with fminsearch: Use "fminsearch" to find the two scaling factors that minimize the error function (or make the subtraction visually better if you skipped step 2). Provide an initial guess for the scaling factors (e.g., [0.5; 1]).
4. Apply optimized factors: Use the scaling factors found by "fminsearch" to subtract the scaled solvent and waterline signals from your original data.

This approach automates waterline removal by finding the best scaling factors for subtraction. You can adjust the initial guess and error function (if used) based on your specific data

Here's the template of the code, Try working on it.

% Function to perform waterline subtraction with scaling factors
function [subtractedData] = waterlineSubtraction(FTIR_data, solventData, waterlineData, scalingFactors)
  % Apply scaling factors and subtract
  subtractedData = FTIR_data + scalingFactors(1) * solventData + scalingFactors(2) * waterlineData;
end

% (Assuming you have loaded your data into variables)
% Initial guess for scaling factors (adjust as needed)
initialGuess = [0.5; 1];

% Define an error function (optional, replace with visual inspection if preferred)
function [error] = errorFunction(scalingFactors)
  subtractedData = waterlineSubtraction(FTIR_data, solventData, waterlineData, scalingFactors);
  % Replace with your desired error metric (e.g., mean squared difference)
  error = mean((desiredData - subtractedData).^2);
end

% Optimize scaling factors (or comment out if using error function)
% Uncomment the following lines to use error function
% options = optimset('Display','off');  % Suppress fminsearch output
% [optimizedFactors, errorValue] = fminsearch(@errorFunction, initialGuess, options);

% Comment out the above lines and uncomment the following if using visual inspection
optimizedFactors = fminsearch(@(x) visuallyEvaluateSubtraction(waterlineSubtraction(FTIR_data, solventData, waterlineData, x)), initialGuess);

% Apply optimized factors
subtractedDataFinal = waterlineSubtraction(FTIR_data, solventData, waterlineData, optimizedFactors);

% Use the subtractedDataFinal for further analysis