streamingDataCompiler

Package stream processing function into deployable archive

Since R2022b

This function requires Streaming Data Framework for MATLAB^® Production Server™ and MATLAB Compiler SDK™.

Syntax

filePath = streamingDataCompiler(streamFcn,inStream,outStream)

filePath = streamingDataCompiler(streamFcn,inStream,outStream,Name=Value)

Description

filePath = streamingDataCompiler(streamFcn,inStream,outStream) creates a MATLAB Compiler SDK project file and opens the Production Server Archive Compiler (MATLAB Compiler SDK) app. Use this app to package the streaming function into a deployable archive for MATLAB Production Server.

streamingDataCompiler returns the full path to the project file.

filePath = streamingDataCompiler(streamFcn,inStream,outStream,Name=Value) sets additional options for packaging the function.

For example, if you specify OutputType="Archive", the streamingDataCompiler function returns a deployable archive (CTF file) instead of a project file.

Examples

Package Streaming Analytic Function for Deployment to MATLAB Production Server

Assume that you have a Kafka^® server running at the network address kafka.host.com:9092 that has a topic RecamanSequence.

Also assume that you have a streaming analytic function recamanSum and a function initRecamanSum to initialize persistent state.

Create a KafkaStream object connected to the RecamanSequence topic.

ks = kafkaStream("kafka.host.com",9092,"RecamanSequence");

Create another KafkaStream object to write the results of the streaming analytic function to a different topic called RecamanSequenceResults.

outks = kafkaStream("kafka.host.com",9092,"RecamanSequenceResults");

Package the streaming analytic function recamanSum into a deployable archive. Since the analytic function uses the state initialization function initRecamanSum, also specify the StateStore property as an input argument.

file = streamingDataCompiler(@recamanSum,ks,outKS, ...
    StateStore="KVStore",InitialState=@initRecamanSum)

file = 

    "J:\recamanSum.prj"

The package function generates a MATLAB project file based on the eventStreamProcessor object, returns the path to this file, and opens this project file with the Production Server Archive Compiler (MATLAB Compiler SDK) app. The Production Server Archive Compiler task contains values for:

• The streaming analytic function, recamanSum.m

• The entry point function, streamfcn.m

• The deployable archive, RecamanSum.ctf

For more information on using the Production Server Archive Compiler, for instance, to modify the list of deployed functions or the name of the generated archive, see Create MATLAB Production Server Archive Using Production Server Archive Compiler App (MATLAB Compiler SDK).

In the Production Server Compiler, click Package to generate the deployable archive. You can deploy the generated archive to MATLAB Production Server. For more information on deploying to MATLAB Production Server, see Deploy Archive to MATLAB Production Server.

Input Arguments

streamFcn — Streaming analytic function

function handle | string | character vector

Streaming analytic function, specified as a function handle, string, or character vector.

Data Types: function_handle | string | char

inStream — Event stream from which streaming analytic function reads events

stream connector object

Event stream from which the streaming analytic function reads events, specified as a stream connector object, such as KafkaStream or TestStream.

outStream — Event stream to which streaming analytic function writes events

stream connector object

Event stream from which the streaming analytic function reads events, specified as a stream connector object, such as KafkaStream or TestStream. InMemoryStream objects are not supported.

Name-Value Arguments

Specify optional pairs of arguments as Name1=Value1,...,NameN=ValueN, where Name is the argument name and Value is the corresponding value. Name-value arguments must appear after other arguments, but the order of the pairs does not matter.

Example: streamingDataCompiler(streamFcn,inStream,outStream,OutputFolder='J:\')

Compile Configuration Options

ArchiveName — Name of generated deployable archive

streamFcn (default) | string | character vector

Name of the generated deployable archive, specified as a string or character vector.

Data Types: char | string

OutputType — File type

Project (default) | Archive

Type of file created by the streamingDataCompiler function, specified as one of these values.

• Project — Creates a productionServerCompiler project file and launches the Production Server Archive Compiler app. You can create a CTF file using the Production Server Archive Compiler.

• Archive — Creates a CTF file.

Data Types: char | string

ExtraFiles — Additional files to include in archive

character vector | string scalar | string array

Additional files to include in the generated archive, specified as a character vector or string scalar for a single file or as a string array for multiple files.

Example: ExtraFiles=["data.mat","/schema/registry/schema.json"] includes the files data.mat and schema.json in the generated deployable archive.

Data Types: char | string

OutputFolder — Location of generated file

current folder (default) | string | character vector

Location of the generated file, specified as a string or character vector.

Example: OutputFolder='J:\' saves the generated file in J:\.

Data Types: string | char

OpenProject — Flag to automatically open project in MATLAB

true (default) | false

Flag to automatically open the project in MATLAB, specified as logical true or false. This property is incompatible with OutputType="Archive".

Data Types: logical

Execution Configuration Options

InitialState — Function that creates initial state for streaming analytic function

function handle | string | character vector

Function that creates the initial state for the streaming analytic function, specified as either a function handle, string, or character vector.

Data Types: function_handle | char | string

StateStore — Persistent storage connection name

string | character vector

Persistent storage connection name, specified as a string or character vector. You must specify a StateStore when using InitialState or when using a stateful stream function. The connection name must be known to the MATLAB Production Server instance to which the archive will be deployed. For more information on using a data cache for persistent storage, see Data Caching Basics.

Data Types: char | string

GroupByKey — Flag to call streaming function with events that have same key

false (default) | true

Flag to call the streaming function with events that have same key, specified as a logical true or false. Setting this flag to true splits an event window into subwindows with homogeneous keys and calls the streaming function once per subwindows.

Data Types: logical

Version History

Introduced in R2022b