Use Cases A Streaming Data Lake Near Real-Time Ingestion Incremental Processing Pipelines Unified Batch and Streaming Cloud-Native Tables Schema Management ACID Transactions ...

Apache XTable (incubating) Interoperating with XTable Installation Syncing to XTable Hudi Streamer Extensions Apache XTable (incubating) Hudi (tables created from 0.14.0 onw...

Google BigQuery Sync Modes Manifest File Benefits of using the new manifest approach: View Over Files (Legacy) Configurations Partition Handling Example Google BigQuery H...

Exporter Introduction Arguments Examples Copy a Hudi dataset Export to json or parquet dataset Export to json or parquet dataset with transformation/filtering Re-partitioning ...

Design & Concepts FAQ How does Hudi ensure atomicity? Does Hudi extend the Hive table layout? What concurrency control approaches does Hudi adopt? Hudi’s commits are based on tr...

Using Kafka Connect Design Configs Current Limitations Using Kafka Connect Kafka Connect is a popularly used framework for integrating and moving streaming data between vari...

Batch Reads Spark DataSource API Daft Batch Reads Spark DataSource API The hudi-spark module offers the DataSource API to read a Hudi table into a Spark DataFrame. A time-t...

Overview What is Apache Hudi Core Concepts to Learn Getting Started Connect With The Community Join in on discussions Come to Office Hours for help Community Calls Contribut...

Docker Demo A Demo using Docker containers Prerequisites Setting up Docker Cluster Build Hudi Bringing up Demo Cluster Demo Step 1 : Publish the first batch to Kafka Step 2: ...

Troubleshooting Writing Tables org.apache.parquet.io.InvalidRecordException: Parquet/Avro schema mismatch: Avro field ‘col1’ not found java.lang.UnsupportedOperationException: org...