Cloud Storage

Talking to Cloud Storage

Immaterial of whether RDD/WriteClient APIs or Datasource is used, the following information helps configure access to cloud stores.

Cloud Storage - 图1note

Many cloud object storage systems like Amazon S3 allow you to set lifecycle policies, such as S3 Lifecycle, to manage objects. One of the policies is related to object expiration. If your organisation has configured such policies, then please ensure to exclude (or have a longer expiry period) for Hudi tables.