TubeMQ

Overview

Apache InLong TubeMQ is a distributed, open source pub-sub messaging and steaming platform for real-time workloads, trillions of massive data precipitation.

Version

Load NodeVersion
TubeMQTubeMQ: >=0.1.0

Dependencies

In order to set up the TubeMQ Load Node, the following provides dependency information for both projects using a build automation tool (such as Maven or SBT) and SQL Client with Sort Connectors JAR bundles.

Maven dependency

  1. <dependency>
  2. <groupId>org.apache.inlong</groupId>
  3. <artifactId>sort-connector-tubemq</artifactId>
  4. <version>2.1.0-SNAPSHOT</version>
  5. </dependency>

How to create a TubeMQ Load Node

Usage for SQL API

The example below shows how to create a TubeMQ Load Node with Flink SQL Cli :

  1. -- Create a TubeMQ table 'tube_load_node' in Flink SQL Cli
  2. Flink SQL>
  3. CREATE TABLE tube_load_node
  4. (
  5. id INT,
  6. name STRING,
  7. age INT,
  8. salary FLOAT
  9. ) WITH (
  10. 'connector' = 'tubemq',
  11. 'topic' = 'topicName',
  12. 'master.rpc' = 'rpcUrl', -- 127.0.0.1:8715
  13. 'format' = 'json',
  14. 'group.name' = 'groupName');
  15. -- Read data from tube_load_node
  16. Flink SQL>
  17. SELECT *
  18. FROM tube_load_node;

Usage for InLong Dashboard

TODO

Usage for InLong Manager Client

TODO

TubeMQ Load Node Options

ParameterRequiredDefault valueTypeDescription
connectorrequiredtubemqStringSet the connector type. Available options are tubemq.
topicrequired(none)StringSet the input or output topic
masterRpcrequired(none)StringSet the TubeMQ master service address.
formatrequired(none)StringTubeMQ message value serialization format, support JSON, Avro, etc. For more information, see the Flink format.
groupIdrequired(none)StringConsumer group in TubeMQ