[Eyetracker] How can I extract specific lines of text data with multiple headers

Hello,

I have a text file which is in this format :

Table Header for Fixations:
Event Type  Trial  Number  Start  End  Duration  Location X  Location Y  Dispersion X  Dispersion Y  Plane  Avg. Pupil Size X  Avg. Pupil Size Y

Table Header for Saccades:
Event Type  Trial  Number  Start  End  Duration  Start Loc.X  Start Loc.Y  End Loc.X  End Loc.Y  Amplitude  Peak Speed  Peak Speed At  Average Speed  Peak Accel.  Peak Decel.  Average Accel.

Table Header for Blinks:
Event Type  Trial  Number  Start  End  Duration

Table Header for User Events:
Event Type  Trial  Number  Start  Description

Table Header for Trigger Line Events:
Event Type  Trial  Number  Start  Duration  Port Status

TriggerLine  1  1  632707887  17402790  120
TriggerLine  1  2  650127313  22044803  120
TriggerLine  1  3  672188819  373945665  120
TriggerLine  1  4  1046151042  58214740  120
TriggerLine  1  5  1104382361  325613598  120
TriggerLine  1  6  1430012651  24057981  120
TriggerLine  1  7  1454087118  192346927  120
TriggerLine  1  8  1646450655  13027109  120
TriggerLine  1  9  1659511020  320755270  120
TriggerLine  1  10  1980283097  149920874  120
Fixation B  1  1  623756870  624388984  632114  451.56  294.47  2  8  -1  87.16  75.92
Blink B  1  1  624388984  624621974  232990
Fixation B  1  2  624621974  625553703  931729  578.77  312.47  5  15  -1  90.41  78.13
Blink B  1  2  625553703  625819981  266278
Fixation B  1  3  625819981  626485566  665585  567.75  305.17  5  6  -1  94.03  80.65
Saccade B  1  1  626485566  626535288  49722  566.27  306.11  454.79  309.45  4.96  260.04  0.67  140.33  9071.39  410.15  5348.79
Fixation B  1  4  626535288  627683395  1148107  435.34  305.19  21  11  -1  96.94  82.01
Blink B  1  3  627683395  628232281  548886
Fixation B  1  5  628232281  628731643  499362  458.17  292.09  3  5  -1  101.20  85.20
Saccade B  1  2  628731643  628764768  33125  455.68  292.43  446.05  294.35  0.54  36.53  0.50  19.82  2024.70  1.80  1013.25
Fixation B  1  6  628764768  629779743  1014975  426.41  304.12  6  7  -1  101.57  85.30
Blink B  1  4  629779743  646500490  16720747
Fixation B  1  7  646500490  646966141  465651  519.89  274.86  16  44  -1  90.11  69.54
Saccade B  1  3  646966141  647016189  50048  517.23  263.95  545.84  273.14  1.87  80.40  0.67  38.92  3058.65  586.94  1890.55
Fixation B  1  8  647016189  647415411  399222  549.07  279.40  6  4  -1  88.42  79.42
Saccade B  1  4  647415411  647448660  33249  547.67  281.06  559.92  280.36  0.84  45.32  0.50  26.62  2250.69  495.81  1373.25
Fixation B  1  9  647448660  648047684  599024  562.61  282.73  4  7  -1  88.56  79.50
Blink B  1  5  648047684  648347193  299509
Fixation B  1  10  648347193  648929475  582282  387.15  283.03  4  6  -1  91.51  79.60
Saccade B  1  5  648929475  648979394  49919  386.72  285.17  328.18  271.67  2.98  116.87  0.67  74.50  6179.34  228.32  2462.09
Fixation B  1  11  648979394  649378721  399327  326.44  269.82  5  4  -1  89.08  79.46
Saccade B  1  6  649378721  649412104  33383  329.41  269.51  336.87  269.52  0.52  27.20  0.50  16.46  1698.78  406.03  1052.41
Fixation B  1  12  649412104  649961159  549055  345.87  272.08  4  3  -1  91.73  80.39

And my goal would be to extract events of the same types in separate arrays with the respective headers. Do you guys have any idea how I should proceed ?

Thanks a lot, and sorry for asking such a vague question.

[Agnes Palit]

Have you solve this?

[Paolo]

@Agnes you should open a new question for your query. It's unlikely that OP will get back to you soon.

[Agnes Palit]

Hi, @Paolo could you help me to solve this? https://uk.mathworks.com/matlabcentral/answers/410596-how-import-file-csv-with-vertical-header

Thank you

Answer 1

[Saurabh Gupta]

Your file format is a bit mixed up, so you will have to do some processing to separate the related data rows and match them with corresponding headers.

1) It may make more sense to simply use Low-level I/O to import the data and move them into respective data structures based on the concerned logic. The following doc page demonstrates the use of Low-level I/O.

https://www.mathworks.com/help/releases/R2017a/matlab/import_export/import-text-data-files-with-low-level-io.html

2) Another method would involve 'detectImportOptions' to detect and then set the corresponding DelimitedTextImportOptions, followed by using 'readtable' to read the contents, and finally post-processing the table to separate the data rows based on logic. You should find the following pages helpful.

readtable https://www.mathworks.com/help/releases/R2017a/matlab/ref/readtable.html#bvghccx

DelimitedTextImportOptions https://www.mathworks.com/help/releases/R2017a/matlab/ref/delimitedtextimportoptions-object.html

detectImportOptions https://www.mathworks.com/help/releases/R2017a/matlab/ref/detectimportoptions.html

To give you one example, the following lines of code will give you the header with 13 variables and all data rows upto 13 values (with blanks filled with NaN's)

>> opts = detectImportOptions('test.txt', 'Delimiter', '\t', 'NumVariables', 13);
>> T = readtable('test.txt',opts);

Since you have multiple headers of varying lengths and conflicting header names at some positions, you may need to play around with the properties and work through different iterations to work out the correct logic for you file.

All the best!