Overview

Overview

Load Nodes is a set of Sink Connectors based on Apache Flink® for loading data to different storage systems.

Supported Load Nodes

Supported Flink Versions

The following table shows the version mapping between InLong® Load Nodes and Flink®:

Usage for SQL API

We need several steps to setup a Flink cluster with the provided connector.

1. Setup a Flink cluster with version 1.13.5 and Java 8+ installed.
2. Download and decompress the Sort Connectors jars from the Downloads page (or build yourself).
3. Put the Sort Connectors jars under FLINK_HOME/lib/.
4. Restart the Flink cluster.

The example shows how to create a MySQL Load Node in Flink SQL Client and load data to it.

-- Creates a MySQL Extract Node
CREATE TABLE mysql_extract_node (
    id INT NOT NULL,
    name STRING,
    age INT,
    weight DECIMAL(10,3),
    PRIMARY KEY(id) NOT ENFORCED
) WITH (
      'connector' = 'mysql-cdc-inlong',
      'hostname' = 'YourHostname',
      'port' = '3306',
      'username' = 'YourUsername',
      'password' = 'YourPassword',
      'database-name' = 'YourDatabaseName',
      'table-name' = 'YourTableName'
      );
-- Creates a MySQL Load Node
CREATE TABLE mysql_load_node (
 id INT NOT NULL,
 name STRING,
 age INT,
 weight DECIMAL(10,3),
 PRIMARY KEY(id) NOT ENFORCED
) WITH (
 'connector' = 'jdbc-inlong',
 'url' = 'jdbc:mysql://YourHostname:3306/YourDatabaseName',
 'username' = 'YourUsername',
 'password' = 'YourPassword',
 'table-name' = 'YourTableName'
);
INSERT INTO mysql_load_node SELECT id, name, age, weight FROM mysql_extract_node;