Store object storage credentials for a session

Registers credentials with DuckDB's secrets manager so a DuckLake can read and write data files on object storage (lake_path = "s3://..." and friends). Wraps CREATE SECRET.

Usage

create_storage_secret(
  type = c("s3", "gcs", "r2", "azure"),
  ...,
  provider = NULL,
  scope = NULL,
  name = NULL,
  persistent = FALSE
)

Arguments

type

Storage type: "s3" (also for S3-compatible stores), "gcs" (Google Cloud Storage), "r2" (Cloudflare R2), or "azure".

...

Named secret parameters passed through to CREATE SECRET, e.g. key_id, secret, region, session_token, endpoint, url_style, account_id (R2), or connection_string (Azure). Character values are quoted; logicals become true/false.

provider

Optional credential provider. The common one is "credential_chain", which picks up credentials the way AWS SDKs do (environment variables, profiles, instance metadata) so no key needs to be passed in code. For "s3", "gcs", and "r2" secrets this provider lives in DuckDB's aws extension, which is loaded (and installed on first use) automatically.

scope

Optional URI prefix (e.g. "s3://my-bucket") limiting which paths the secret applies to. Useful when different buckets need different credentials.

name

Optional name for the secret. Named secrets can be replaced and dropped individually; unnamed ones act as the default for their type.

persistent

If TRUE, the secret is written (unencrypted) to DuckDB's secret directory and survives the session. That directory sits under the same "home" directory as extensions (~/.duckdb when it exists, or DUCKDB_R_HOME / the duckdb.home option; see ?duckdb::duckdb_storage), and with the temporary default the secret is lost with the session anyway. The default FALSE keeps it in memory only, which is the right choice for credentials supplied from a vault or environment variable.

Value

Invisibly returns the secret's name (or NA_character_ for an unnamed secret).

Details

The httpfs extension (or the azure extension for type = "azure") is loaded automatically. With provider = "credential_chain", the aws extension is loaded too: it supplies that provider for "s3", "gcs", and "r2" secrets, and DuckDB's automatic mid-statement install of it can fail, so the package loads it up front instead. If you pre-install extensions (say, when baking a container image), include aws alongside httpfs. The azure extension provides its own credential chain.

On Windows neither the aws nor the azure extension is available for the duckdb R package, so provider = "credential_chain" and type = "azure" fail there; explicit keys for "s3", "gcs", and "r2" work.

Prefer provider = "credential_chain" over embedding long-lived keys in scripts. The secret's values are visible in the session via duckdb_secrets() (redacted) and travel with persistent storage unencrypted, so treat persistent = TRUE with the same care as a credentials file.

See also

attach_ducklake()

Other connection management: attach_ducklake(), detach_ducklake(), ducklake_extension_available(), get_ducklake_backend(), get_ducklake_connection(), get_ducklake_info(), install_ducklake(), set_ducklake_connection()

Examples

if (FALSE) { # \dontrun{
# Explicit keys, scoped to one bucket
create_storage_secret(
  "s3",
  key_id = Sys.getenv("AWS_ACCESS_KEY_ID"),
  secret = Sys.getenv("AWS_SECRET_ACCESS_KEY"),
  region = "us-east-1",
  scope = "s3://my-trial-lake"
)

# Let the AWS credential chain find credentials
create_storage_secret("s3", provider = "credential_chain")

# Then attach a lake whose data lives on S3. The catalog file stays on
# local disk; lake_path only sets where the Parquet data goes.
attach_ducklake(
  "trial_lake",
  lake_path = "s3://my-trial-lake/data",
  catalog_connection_string = "trial_lake.ducklake"
)
} # }