Memory Pre allocation, Dataset Array
2 views (last 30 days)
Show older comments
Hi All,
How Can I Pre allocate memory for a DATASET ARRAY which should have 69 Rows and 740 Columns. There is another dataset array of the same size in my workspace. How Can I do NewArray=dataset(size(OldArray)) ?,,,to use size arguments from already built DATASET Array?
Regards,
AMD.
0 Comments
Accepted Answer
Daniel Shub
on 3 May 2012
The dataset class is basically a container holding pointers to other variables/memory locations. Even if you can preallocate the dataset array, I am not sure it will improve performance by much.
8 Comments
Daniel Shub
on 3 May 2012
"What does the above error mean" and "how to convert my dataset array to a cell array" are new questions. I will not attempt to answer them in a comment to an answer to an unrelated question. I have never used the dataset class so have little insight into converting it to other classes, or even what it is good for. You will be much better asking these as new questions.
More Answers (4)
Oleg Komarov
on 3 May 2012
One way:
n = 300;
names = arrayfun(@(x) sprintf('C%d',x),1:n,'un',0);
dataset([{zeros(30,n)},names])
Or
dataset(zeros(size(OldArray)));
2 Comments
Oleg Komarov
on 3 May 2012
The first solution creates 300 columns (you can set 'n' to be anything) and the second solution is slightly different. Depends what you need to do.
I am not sure but the second might be more efficient.
Peter Perkins
on 3 May 2012
Ahmad, there are several things going on in this thread. Let me try to answer them one by one.
A dataset array can hold just about any type of variable, so just specifying what size you want is not sufficient to create one. As you noted, Oleg's suggestion will create a dataset with a single variable, and Jeff's suggestion will create a numeric array. You need to create a dataset array. I can't say exactly what you need to do to preallocate, because you have not provided any specific information about types. In the simplest case, if you want the new array to have the same data types in each variable as the old array, all you need to do to preallocate is
dsNew = dsOld;
and then just overwrite the existing values with the new ones. There's no particular reason why you would need to start with zeros, or empty strings or whatever, unless you would only be overwriting some elements. But there are ways to do that too. If the new array is to contain different types of data than the old array, there are ways to do that too.
You must be running a version of MATLAB older than R2012a, which is why dataset2cell is not found.
As for your comments about dataset vs. cell:
Elsewhere you've asked a question or two about manipulating data in a dataset array. One of them involved a nested loop with some string comparison. Not sure if your comments here are related to that or not, but it is often possible to avoid the kind of loop you had there by appropriate use of vectorized operations. So in that case, strcmp. Yes, dataset can be much slower than cell for scalar access, but it is often possible to write code that is both faster to write and to run, and easier to read, by using vectorized operations. In this case, I can't say what that would be without more information.
I can't tell what you are doing with your data, but you may ultimately find that a cell array is not the best way to store it. dataset provides a convenient way to wrap mixed types of data into one container, and still be able to access individual variables as their "native" type using dot subscripting. For example, with a dataset array it's easy to say mean(data.Var1). From what I can tell from your other questions, you have strings and numbers. You can put all that in a cell array, but you'll likely end up disappointed if you are storing scalar numeric data that way, because they are too general and lack, for example, the ability to do simple math. That will also not scale well, again because cell arrays are too general -- internally a cell array will store each scalar value as a separate MATLAB array. On the other hand, if what you have is all string data, then yes, a cell array is the right container.
Hope this helps.
2 Comments
avantika
on 29 Aug 2013
Hi!
I am trying to convert a dataset to cell array of strings to be able to use the unique command in matlab version 2009b. However when I use the command C = dataset2cell(ds3);
I get the following error message:
??? Undefined variable "dataset2cell" or class "dataset2cell".
Is there any solution to tihs problem in matlab v2009.
See Also
Categories
Find more on Large Files and Big Data in Help Center and File Exchange
Products
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!