Doris

Overview

Doris Extract node supports reading data from Doris. This chapter describes how to set up a Doris Extract node to run SQL queries against the Doris database.

Supported Version

Dependencies

In order to set up the Doris Extract node, the dependency information needed to use build automation tools such as Maven or SBT is provided below.

Maven dependency

<dependency>
    <groupId>org.apache.inlong</groupId>
    <artifactId>sort-connector-doris</artifactId>
    <version>1.11.0-SNAPSHOT</version>
</dependency>

Prepare

Create a Doris Extract table

First create a table doris_extract_node in the Doris database, the command is as follows:

[root@fe001 ~]# mysql -u root -h localhost -P 9030 -p000000
mysql> use test;
Reading table information for completion of table and column names
You can turn off this feature to get a quicker startup with -A
Database changed
mysql> CREATE TABLE `doris_extract_node` (
       `id` int(11) NOT NULL COMMENT "user id",
       `name` varchar(50) NOT NULL COMMENT "user name",
       `dr` tinyint(4) NULL COMMENT "delete tag"
       ) ENGINE=OLAP
       UNIQUE KEY(`id`)
       COMMENT "OLAP"
       DISTRIBUTED BY HASH(`id`) BUCKETS 1
       PROPERTIES (
       "replication_allocation" = "tag.location.default: 1",
       "in_memory" = "false",
       "storage_format" = "V2"
       );
Query OK, 0 rows affected (0.03 sec)
mysql> insert into doris_extract_node values(1, 'zhangsan', 0),(2, 'lisi', 0),(3, 'wangwu', 0);
Query OK, 3 rows affected (0.07 sec)
{'label':'insert_29d973e9509a48d4-a20e9f0e2d510605', 'status':'VISIBLE', 'txnId':'1032'}
mysql> select * from doris_extract_node;
+------+---------+------+
| id   | name    | dr   |
+------+---------+------+
|    1 | zhansan |    0 |
|    2 | lisi    |    0 |
|    3 | wangwu  |    0 |
+------+---------+------+
3 rows in set (0.02 sec)

How to create a Doris Extract Node

Usage for SQL API

The following example shows how to create a Doris Extract node with Flink SQL:

• connector is doris

# Start flink sql-client, load the doris connector jar package
[root@tasknode001 flink-1.13.5]# ./bin/sql-client.sh -l ./opt/connectors/doris/
-- Create Doris table 'doris_extract_node' using Flink SQL
Flink SQL> CREATE TABLE doris_extract_node (
           `id` INT,
           `name` STRINTG,
           `dr` TINYINT
           ) WITH (
           'connector' = 'doris',
           'fenodes' = 'localhost:8030',
           'table.identifier' = 'test.doris_extract_node',
           'username' = 'root',
           'password' = '000000'
           );
-- query data
Flink SQL> SELECT * FROM doris_extract_node;

Usage for InLong Dashboard

TODO: It will be supported in the future.

Usage for InLong Manager Client

TODO: It will be supported in the future.

Doris Extract Node Options

Data Type Mapping

See flink-doris-connector for more details.