Cloud

The following describes parameters used to sync indexes with cloud storage. Cloud object storage, the Hugging Face Hub and custom providers are all supported.

Parameters are set via the embeddings.load and embeddings.save methods.

provider

  1. provider: string

Cloud provider. Can be one of the following:

  • Cloud object storage. Set to one of these providers.

  • Hugging Face Hub. Set to huggingface-hub.

  • Custom providers. Set to the full class path of the custom provider.

container

  1. container: string

Container/bucket/directory/repository name.

Cloud object storage configuration

In addition to the above common configuration, the cloud object storage provider has the following additional configuration parameters.

key

  1. key: string

Provider-specific access key. Can also be set via ACCESS_KEY environment variable. Ensure the configuration file is secured if added to the file.

secret

  1. secret: string

Provider-specific access secret. Can also be set via ACCESS_SECRET environment variable. Ensure the configuration file is secured if added to the file.

host

  1. host: string

Optional server host name. Set when using a local cloud storage server.

port

  1. port: int

Optional server port. Set when using a local cloud storage server.

token

  1. token: string

Optional temporary session token

region

  1. region: string

Optional parameter to specify the storage region, provider-specific.

Hugging Face Hub configuration

The huggingface-hub provider supports the following additional configuration parameters. More on these parameters can be found in the Hugging Face Hub’s documentation.

revision

  1. revision: string

Optional Git revision id which can be a branch name, a tag, or a commit hash

cache

  1. cache: string

Path to the folder where cached files are stored

token

  1. token: string|boolean

Token to be used for the download. If set to True, the token will be read from the Hugging Face config folder.