Feedback re: Using Tall Arrays with Big Data Demo Video

If you have questions, comments, and/or feedback regarding the Tall Arrays in Action video, or using MATLAB with big data in general, please post them here, and I will respond to you (or find someone to respond =P) as soon as I can!

Best regards, Gabriel

P.S. Heather Gorr has a very similar video located here. This Answers entry is intended for the first video, but feel free to ask away about tall arrays in general =)

[michel hillen]

Is there a way to add new rows of data to an existing tall array? Suppose I have a large data set, stored as a tall array with the write function, and new data are coming in. Can I then add the new data to the existing data set? If so, how?

Best regards, Michel

[Rick Amos]

Hi Michel, at this time the only way to add new data to a tall array is to recreate the tall array from a datastore containing both the existing data and new data. You could use the tall array write method to achieve this:

% Suppose at the start you have initial data, write it out as so:
write('MyFolder01', tX1);

% Suppose you have some new data at a later time, write it out like so:
write('MyFolder02', tX2);

% Then to analyse all the data thus far, specify both folders as
% the input to datastore:
tX = tall(datastore({'/my/folder/01', '/my/folder/02'}));

Kind Regards, Rick

[mohaideen]

Is there any test sensor data set is available for big data analytics?.

[Gabriel Ha]

We may have some coming out with our next release. Be sure to check back when 17a goes out!

Answer 1

[Hector Tejeda]

At the time of performing the calculations I get an error saying that indexing using tall arrays as subscripts for non-tall arrays is not supported.