parallelize

Class: matlab.compiler.mlspark.SparkContext
Namespace: matlab.compiler.mlspark

Create an RDD from a collection of local MATLAB values

Syntax

rdd = parallelize(sc,cellArray)
rdd = parallelize(sc,cellArray,numSlices)


Description

rdd = parallelize(sc,cellArray) creates an RDD from a collection of local MATLAB^® values grouped as a cell array.

rdd = parallelize(sc,cellArray,numSlices) creates an RDD with the number of partitions specified by numSlices.

Input Arguments

sc — SparkContext to use

SparkContext object

The SparkContext to use, specified as a SparkContext object.

cellArray — Collection of values

cell array

A collection of values, specified as a MATLAB cell array.

Data Types: cell

numSlices — Number of partitions to create

scalar

Number of partitions to create, specified as a scalar.

Data Types: double

Output Arguments

rdd — Output RDD created from the collection of values

RDD object

An output RDD, created from the collection of values, and returned as an RDD object.

Examples

Create an RDD From MATLAB Values

Create an RDD from local MATLAB values.

%% Connect to Spark
sparkProp = containers.Map({'spark.executor.cores'}, {'1'});
conf = matlab.compiler.mlspark.SparkConf('AppName','myApp', ...
                        'Master','local[1]','SparkProperties',sparkProp);
sc = matlab.compiler.mlspark.SparkContext(conf);

%% parallelize
x = sc.parallelize({1, 2, 3, 4, 5});
y = x.count()

Version History

Introduced in R2016b