Importing data from a text file into MATLAB as multiple variables of differing sizes

13 views (last 30 days)
Hello,
I have an nx12 matrix in a text file with tab delimiters and would like to import certain chunks of information. I need columns 9&10 from rows 3 through n, so long as column 4 has a value of '16'. Column 4 will output '177' occasionally, which is a signal to create a new variable. The number of times column 4 outputs '177' will determine the number of variables output and each variable will be a 2-column matrix with a different number of rows. Here's what I have so far using the import tool to manually select data:
%
% This is from using the import tool to manually determine start and end rows from the data
% I would like to automate this process for larger files so I don't have to manually select the data and copy-paste.
%
filename = 'C:\Program Files (x86)\Mission Planner\WP Files\Constraints\Constraints_0m.waypoints';
delimiter = '\t';
startRow = [3,10,29,42,49,58,67,76,86,95,110]; % always start on row 3
endRow = [8,27,40,47,56,65,74,84,93,108,113]; % ends at 'n' which is 113 here.
% rows [9,28,41,48,57,66,75,85,94,109] had '177' in column 4... skip.
% The rest had '16' in column 4. This leaves 'm' chunks of data.
% 'm' and 'n' will vary for each file.
%
formatSpec = '%*s%*s%*s%*s%*s%*s%*s%*s%f%f%*s%*s%[^\n\r]';
fileID = fopen(filename,'r');
%
dataArray = textscan(fileID, formatSpec, endRow(1)-startRow(1)+1, 'Delimiter', delimiter, 'TextType', 'string', 'HeaderLines', startRow(1)-1, 'ReturnOnError', false, 'EndOfLine', '\r\n');
%
for block=2:length(startRow)
frewind(fileID);
dataArrayBlock = textscan(fileID, formatSpec, endRow(block)-startRow(block)+1, 'Delimiter', delimiter, 'TextType', 'string', 'HeaderLines', startRow(block)-1, 'ReturnOnError', false, 'EndOfLine', '\r\n');
for col=1:length(dataArray)
dataArray{col} = [dataArray{col};dataArrayBlock{col}];
end
end
%
fclose(fileID);
%
Constraints0m = table(dataArray{1:end-1}, 'VariableNames', {'VarName9','VarName10'});
% This only outputs one single 2-column table when I need 'm' 2-column variables
%
clearvars filename delimiter startRow endRow formatSpec fileID block dataArrayBlock col dataArray ans;
I'm guessing 'starRow' and 'endRow' will need to be generated first based on column 4 with a 'textscan' loop, then I need to create a new variable or index for each chunk of information. I will need to know 'm' as an output because the rest of my code runs loops based on the size of m (which I've hard coded for the time being but will need to automate) .

Accepted Answer

Siddharth Bhutiya
Siddharth Bhutiya on 22 Oct 2018
Yes, you are right. You can start by reading the entire data using textscan. After that, you can use something similar to the code snippet below to obtain the row indices where the output is '177'.
signalValues = mydata{:,4}; % here we get the 4th column from the data
idx = find(ismember(signalValues, '177'));
You can get more information about "ismember" and "find" functions using the links below
Once you have these you can generate the vectors for start and end row indices and divide your data. Also, you can index the entire data to get the subparts instead of using multiple textscans inside the for loop. For example, to get rows from start_index to end_index and columns m to n, you can index the data as follows,
mydata(start_index:end_index, m:n);

More Answers (0)

Categories

Find more on Large Files and Big Data in Help Center and File Exchange

Products


Release

R2017b

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!