TubeMQ

Overview

Apache InLong TubeMQ is a distributed, open source pub-sub messaging and steaming platform for real-time workloads, trillions of massive data precipitation.

Version

Extract NodeVersion
TubeMQTubeMQ: >=0.1.0

Dependencies

In order to set up the TubeMQ Extract Node, the following provides dependency information for both projects using a build automation tool (such as Maven or SBT) and SQL Client with Sort Connectors JAR bundles.

Maven dependency

  1. <dependency>
  2. <groupId>org.apache.inlong</groupId>
  3. <artifactId>sort-connector-tubemq</artifactId>
  4. <version>1.4.0-SNAPSHOT</version>
  5. </dependency>

How to create a TubeMQ Extract Node

Usage for SQL API

The example below shows how to create a TubeMQ Extract Node with Flink SQL Cli :

  1. -- Create a TubeMQ table 'tube_extract_node' in Flink SQL Cli
  2. Flink SQL> CREATE TABLE tube_extract_node (
  3. id INT,
  4. name STRING,
  5. age INT,
  6. salary FLOAT
  7. ) WITH (
  8. 'connector' = 'tubemq',
  9. 'topic' = 'topicName',
  10. 'masterRpc' = 'rpcUrl', -- 127.0.0.1:8715
  11. 'format' = 'json',
  12. 'groupId' = 'groupName');
  13. -- Read data from tube_extract_node
  14. Flink SQL> SELECT * FROM tube_extract_node;

Usage for InLong Dashboard

TODO

Usage for InLong Manager Client

TODO

TubeMQ Extract Node Options

ParameterRequiredDefault valueTypeDescription
connectorrequiredtubemqStringSet the connector type. Available options are tubemq.
topicrequired(none)StringSet the input or output topic
masterRpcrequired(none)StringSet the TubeMQ master service address.
formatrequired(none)StringTubeMQ message value serialization format, support JSON, Avro, etc. For more information, see the Flink format.
groupIdrequired(none)StringConsumer group in TubeMQ