TubeMQ
Overview
Apache InLong TubeMQ is a distributed, open source pub-sub messaging and steaming platform for real-time workloads, trillions of massive data precipitation.
Version
Extract Node | Version |
---|---|
TubeMQ | TubeMQ: >=0.1.0 |
Dependencies
In order to set up the TubeMQ Extract Node
, the following provides dependency information for both projects using a build automation tool (such as Maven or SBT) and SQL Client with Sort Connectors JAR bundles.
Maven dependency
<dependency>
<groupId>org.apache.inlong</groupId>
<artifactId>sort-connector-tubemq</artifactId>
<version>1.13.0-SNAPSHOT</version>
</dependency>
How to create a TubeMQ Extract Node
Usage for SQL API
The example below shows how to create a TubeMQ Extract Node with Flink SQL Cli
:
-- Create a TubeMQ table 'tube_extract_node' in Flink SQL Cli
Flink SQL> CREATE TABLE tube_extract_node (
id INT,
name STRING,
age INT,
salary FLOAT
) WITH (
'connector' = 'tubemq',
'topic' = 'topicName',
'master.rpc' = 'rpcUrl', -- 127.0.0.1:8715
'format' = 'json',
'group.name' = 'groupName');
-- Read data from tube_extract_node
Flink SQL> SELECT * FROM tube_extract_node;
Usage for InLong Dashboard
TODO
Usage for InLong Manager Client
TODO
TubeMQ Extract Node Options
Parameter | Required | Default value | Type | Description |
---|---|---|---|---|
connector | required | tubemq | String | Set the connector type. Available options are tubemq . |
topic | required | (none) | String | Set the input or output topic |
masterRpc | required | (none) | String | Set the TubeMQ master service address. |
format | required | (none) | String | TubeMQ message value serialization format, support JSON, Avro, etc. For more information, see the Flink format. |
groupId | required | (none) | String | Consumer group in TubeMQ |