CSV file reading and applying loop to plot graphs

4 views (last 30 days)
How can I do it for multiple csv file name of "WT_201119.csv" ,"WT_201120.csv" contain 52 main header with column. All headers are constant in csv files. Above csv read data correctly using readtable command but only for few files. where I have to do changes? similar to above one csv file Because I got error- Where I have to do changes always so that it can detect the total number of header and rows which we wan b "Error using matlab.io.internal.shared.TextInputs/set.DataLines (line 101) 'DataLines' must be specified as a positive integer or a 2-column matrix of positive integers."y
clear all
d=dir('WT_201119.csv'); % "*" will detect the sequence of your file
opts = detectImportOptions(d(1).name,"Delimiter",","); % already know it's csv, avoid detecting it
T = {};
for i=1:numel(d)
fid = fopen(d(i).name,'r');
nhdr = 8 + cell2mat(textscan(fid,'Total Channels:%f',1,'HeaderLines',5,'Delimiter',','));
fid = fclose(fid);
if nhdr == 8 % skip no data in file cases if that occurs (dpb conjecture)
warning(['No data. Skipped file: ' d(i).name])
opts.DataLines = [nhdr+2 inf];
opts.VariableNamesLine = nhdr+1;
T{i} = readtable(d(i).name,opts)
See more (1)
Arshey Dhangekar
Arshey Dhangekar on 2 Jun 2021
"Error using matlab.io.internal.shared.TextInputs/set.DataLines (line 101) 'DataLines' must be specified as a positive integer or a 2-column matrix of positive integers."
getting error below the line
opts.DataLines = [nhdr+2 inf];

Sign in to comment.

Accepted Answer

Jeremy Hughes
Jeremy Hughes on 3 Jun 2021
Without an example file, it's impossible to say why the code doesn't work as expected. I'm just working backwards from the error message and what I know about the returns of textscan.
I see the example uploaded in a comment for a different answer, but that file works as expected with the code you included.
Arshey Dhangekar
Arshey Dhangekar on 15 Jun 2021
Hello Jeremy,
I have two csv file WT and both have fixed have 52 header. I tried to use below command to read however header is as Var1,Var2..to Var52 instead of Store no,Date and Time. How can I fix the header problem with importing all the datas from those files?
table = readtable('WT_201120.csv');
table1 = readtable('WT_.csv');

Sign in to comment.

More Answers (2)

LO on 2 Jun 2021
Edited: LO on 3 Jun 2021
first import one single CSV file and select coumns and rows of what you want usually to import.
Generate the code by clicking the 'generate code' button, in the import menu (right side of the screen, up)
Then use this code and put it where IMPORT CODE is written here below
[logfname, pathname] = uigetfile('*.csv','Pick a log file'); %select a csv file
cd(pathname); % move to that folder
file_list = dir ('*.csv') ; % create a list of csv files based on folder content
conversion = struct2cell(file_list);
conversion1 = transpose(conversion);
name_list = (conversion1(:,1));
% these two limits give you the option to select only a certain file range
% but you could actually remove them and simply set the k in the for loop
% from 1 to Nr of files
first= 1; % set first file to analyze
last= 10; % set last file to analyze
for k= first:last
filename = [pathname name_list{k}];
delimiter = ',';
startRow = 5;
endRow = 11831; % you can remove this or set it to a level that would fit all your data logs
formatSpec = '%s%s%s%s%s%s%s%s%s%s%s%s%s%s%s%s%s%[^\n\r]';
fileID = fopen(filename,'r','n','UTF-8');
fseek(fileID, 2, 'bof');
dataArray = textscan(fileID, formatSpec, endRow-startRow+1, 'Delimiter', delimiter, 'TextType', 'string', 'EmptyValue', NaN, 'HeaderLines', startRow-1, 'ReturnOnError', false, 'EndOfLine', '\r\n');
data_table = table(dataArray{1:end-1}, 'VariableNames', {'Name','x1','x2','x3','x4','x5','x6','x7','x8','x9','x10','x11','x12','x13','x14','x15','x16'});
data_table = data_table(22:end,:); % this crops away the header part of your file (somehow the file format is not properly detected by matlab and there is some problem with formatting
clearvars filename delimiter startRow endRow formatSpec fileID dataArray ans;
size_t = size(data_table);
plot_size = sqrt(size_t(2));
if plot_size > floor(plot_size)
for i = 1:size_t(2)
% pre allocate var space
time_vector = zeros(numel(data_table(:,i)),1);
var_vector = zeros(numel(data_table(:,i)),1);
for j = 1:numel(data_table(:,i))
time = table2array(data_table(j,1));
time(time == ' ') = [];
time_vector(j) = str2double(time);
var = table2array(data_table(j,i));
var=strrep(var, ' ', '');
var_vector(j) = str2double(var);
title('title here')
LO on 2 Jun 2021
I have updated the script, it works. However there is an issue with the text formatting of your data: due to the presence of whitespaces between characters the variables are converted into NaNs so you cannot see the data. I have tried to eliminate them using the function (var is one of the values in your data array)
strip(var,char(133)) or the other char codes you can see below but it does not remove them.
char(133) Next line
char(160) Nonbreaking space
char(8199) Figure space
char(8239) Narrow no-break space
the strip function could replace the line "var=strrep(var, ' ', '');", which is meant to have analogous results (although strrep does not affect special whitespace characters I suppose).
The problem may be in the formatting of the text and how MATLAB reads it. Do you have by chance the older file you posted ?
Apart from this, it should work.

Sign in to comment.

Jeremy Hughes
Jeremy Hughes on 3 Jun 2021
You should look at what nhdr is when you see the error. It's likely an empty value which makes nhdr+2 empty, and [nhdr+2 inf] just [inf].

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!