List the data files backing a DuckLake table

Returns the Parquet data files (and any delete files) that make up a table, optionally as of a past snapshot.

Usage

list_ducklake_files(
  table_name,
  schema_name = NULL,
  snapshot_version = NULL,
  snapshot_time = NULL,
  ducklake_name = NULL
)

Arguments

table_name

The table whose files to list, optionally qualified as "schema.table".

schema_name

Optional schema containing the table (defaults to the lake's main schema).

snapshot_version

Optional snapshot id: list the files as of that snapshot. Mutually exclusive with snapshot_time.

snapshot_time

Optional POSIXct or UTC timestamp string: list the files as of that moment. Mutually exclusive with snapshot_version.

ducklake_name

Optional name of the attached DuckLake catalog. If NULL, the current database is used.

Value

A data frame with one row per data file, including data_file, data_file_size_bytes, and the associated delete_file columns (NA when a file has no deletes).

Details

Wraps ducklake_list_files(). For per-table file counts and sizes across the whole lake, see get_table_info(); for a picture of storage layout, see plot_table_files().

See also

add_data_files(), get_table_info()

Other maintenance: backup_ducklake(), checkpoint_ducklake(), cleanup_old_files(), delete_orphaned_files(), expire_snapshots(), flush_inlined_data(), get_table_info(), merge_adjacent_files(), plot_table_files(), rewrite_data_files()

Examples

lake_dir <- tempfile("listfiles_lake_")
dir.create(lake_dir)
attach_ducklake("listfiles_lake", lake_path = lake_dir)
create_table(mtcars, "cars")
rows_insert(
  get_ducklake_table("cars"),
  data.frame(mpg = 30, cyl = 4),
  by = "mpg"
)

# Files behind a table right now
list_ducklake_files("cars")
#>                                                                                                     data_file
#> 1 /tmp/RtmpRpiQmQ/listfiles_lake_18d55023df9b/main/cars/ducklake-01a0b152-cc1f-7113-b658-c314a0fba17b.parquet
#>   data_file_size_bytes data_file_footer_size data_file_encryption_key
#> 1                 2911                  1128                     NULL
#>   delete_file delete_file_size_bytes delete_file_footer_size
#> 1        <NA>                     NA                      NA
#>   delete_file_encryption_key
#> 1                       NULL

# Files as of an earlier snapshot
first <- min(list_table_snapshots("cars")$snapshot_id)
list_ducklake_files("cars", snapshot_version = first)
#>                                                                                                     data_file
#> 1 /tmp/RtmpRpiQmQ/listfiles_lake_18d55023df9b/main/cars/ducklake-01a0b152-cc1f-7113-b658-c314a0fba17b.parquet
#>   data_file_size_bytes data_file_footer_size data_file_encryption_key
#> 1                 2911                  1128                     NULL
#>   delete_file delete_file_size_bytes delete_file_footer_size
#> 1        <NA>                     NA                      NA
#>   delete_file_encryption_key
#> 1                       NULL

detach_ducklake("listfiles_lake", shutdown = TRUE)
unlink(lake_dir, recursive = TRUE)