Overview

Overview

Compatibility Matrix

Engine	Version	Batch Read	Batch Write	Create Table	Alter Table	Streaming Write	Streaming Read	Batch Overwrite	DELETE & UPDATE	MERGE INTO	Time Travel
Flink	1.15 - 1.20	✅	✅	✅	✅(1.17+)	✅	✅	✅	✅(1.17+)	❌	✅
Spark	3.1 - 3.5	✅	✅(3.2+)	✅	✅	✅(3.3+)	✅(3.3+)	✅(3.2+)	✅(3.2+)	✅(3.2+)	✅(3.3+)
Hive	2.1 - 3.1	✅	✅	✅	❌	❌	❌	❌	❌	❌	✅
Trino	420 - 439	✅	✅(427+)	✅(427+)	✅(427+)	❌	❌	❌	❌	❌	✅
Presto	0.236 - 0.280	✅	❌	✅	✅	❌	❌	❌	❌	❌	❌
StarRocks	3.1+	✅	❌	❌	❌	❌	❌	❌	❌	❌	✅
Doris	2.0.6+	✅	❌	❌	❌	❌	❌	❌	❌	❌	✅

Streaming Engines

Flink Streaming

Flink is the most comprehensive streaming computing engine that is widely used for data CDC ingestion and the construction of streaming pipelines.

Recommended version is Flink 1.17.2.

Spark Streaming

You can also use Spark Streaming to build a streaming pipeline. Spark’s schema evolution capability will be better implemented, but you must accept the mechanism of mini-batch.

Batch Engines

Spark Batch

Spark Batch is the most widely used batch computing engine.

Recommended version is Spark 3.4.3.

Flink Batch

Flink Batch is also available, which can make your pipeline more integrated with streaming and batch unified.

OLAP Engines

StarRocks

StarRocks is the most recommended OLAP engine with the most advanced integration.

Recommended version is StarRocks 3.2.6.

Other OLAP

You can also use Doris and Trino and Presto, or, you can just use Spark, Flink and Hive to query Paimon tables.

Download

Download Link