ExportOptions

Export options for event stream

Since R2022b

expand all in page

    This object requires Streaming Data Framework for MATLAB® Production Server™.

    Description

    An ExportOptions object specifies how MATLAB exports data from timetables to external event streams, such as Kafka® streams. The object contains properties that control the data export process, including the transformation of event data to the specified type.

    Creation

    Create an ExportOptions object by using the detectExportOptions function. This function detects and populates the export rules based on the configuration of the event stream specified by stream and the variables in the timetable row specified by row.

    opts = detectExportOptions(stream,row)

    After creating opts, use setvartype to change the types of the variables as they are exported from MATLAB to the stream. To export stream data, use the writetimetable function.

    Properties

    expand all

    Scope of the schema, specified as one of these options:

    • "None" — Schema has no scope.

    • "Event" — Schema applies to a single event.

    • "Window" — Schema applies to all events in the window.

    • "Stream" — Schema applies to all windows in the stream.

    Use Scope to optimize how often to refresh the schema.

    Data Types: string | char

    Schema describing the MATLAB variables being exported to the event stream, specified as one of these values:

    • JSON-formatted text (string or character vector)

    • Function handle that evaluates to JSON-formatted text

    • Schema object that produces JSON-formatted text

    This schema defines the rules for converting variables to the appropriate data type.

    Example: '[{"name":"N","type":"int64","size":[1,1],"missingValue":0,"categorical":false},{"name":"X","type":"double","size":[1,1],"missingValue":"","categorical":false}]' describes a schema with two variables: N (int64) and X (double).

    Data Types: char | string | function_handle

    Location of the schema, specified as one of these options:

    • "Event" — Schema is embedded in a single event.

    • "Window" — Schema is embedded in an event window.

    Data Types: char | string

    Structure of the text representation of the schema, specified as one of these options:

    • "Event" — Native, default even stream format

    • "InfluxDB" — Format specifically used by InfluxDB

    Data Types: char | string

    Content of the schema as it is stored in OutputLocation, specified as one of these options:

    • "None" — No schema specified

    • "Literal" — The literal schema (a struct or JSON text)

    • "GeneratorFunction" — A function handle that generates the schema

    • "RegistryID" — A string that identifies the schema

    Data Types: char | string

    Object Functions

    setvartypeSet data types used to import and export variables to stream

    Examples

    collapse all

    Assume that you have a Kafka server running at the network address kafka.host.com:9092 that has the topics Triangles and numericTriangles.

    Create a KafkaStream object connected to the Triangles topic.

    inKS = kafkaStream("kafka.host.com",9092,"Triangles");

    Read events from the Triangles topic into a timetable. Preview the data by viewing the first row. The a, b, and c triangle side lengths are stored as strings.

    tt = readtimetable(inKS);
row = tt(1,:)
    row =

  1×3 timetable

     timestamp      a       b       c  
    ___________    ____    ____    ____

    03-Sep-2022    "15"    "31"    "36"

    Use detectExportOptions to generate an ExportOptions object from the Kafka stream object. The function obtains the types used to export the variables from the first row of the timetable.

    opts = detectExportOptions(inKS,row);

    Use getvartype to confirm that the side length variables are currently exported to the stream as strings.

    type = getvartype(opts,["a" "b" "c"]);

    type = 

  1×3 string array

    "string"    "string"    "string"

    Update the export options so that the side lengths are exported as double values. Confirm the updated options by using getvartype.

    opts = setvartype(opts,["a","b","c"],"double");

[name,type] = getvartype(opts);
fprintf("%s: %s\n", [name; type])
    a: double
b: double
c: double

    Connect to the stream to export data to numericTriangles.

    outKS = kafkaStream("kafka.host.com",9092,"numericTriangles", ...
    ExportOptions=opts)

    outKS = 

  KafkaStream with properties:

                  Topic: "numericTriangles"
                  Group: "85c42e39-695d-467a-86f0-f0095792e7de"
                  Order: EventTime
                   Host: "kafka.host.com"
                   Port: 9092
      ConnectionTimeout: 30
         RequestTimeout: 61
          ImportOptions: "None"
          ExportOptions: "Source: string"
          PublishSchema: "true"
             WindowSize: 50
            KeyVariable: "key"
            KeyEncoding: "utf16"
                KeyType: "text"
           KeyByteOrder: "BigEndian"
           BodyEncoding: "utf8"
             BodyFormat: "JSON"
              ReadLimit: "Size"
    TimestampResolution: "Milliseconds"

    Export the timetable to the new stream. The triangle side lengths in this stream are of type double.

    writetimetable(outKS,tt);

    Version History

    Introduced in R2022b

    See Also

    | |