Rewrite data files with many deletes

Rewrites Parquet files whose rows have mostly been deleted. Deletes in DuckLake are recorded in separate delete files; heavily-deleted data files slow reads down until they are rewritten without the dead rows.

Usage

rewrite_data_files(
  ducklake_name = NULL,
  table_name = NULL,
  delete_threshold = NULL
)

Arguments

ducklake_name

Name of the attached DuckLake catalog. If NULL, the current database is used.

table_name

Optional table name. When provided, only that table's files are rewritten.

delete_threshold

Optional fraction of deleted rows (between 0 and

1. above which a file is rewritten. DuckLake's default is 0.95.

Value

A data frame with one row per output file (columns schema_name, table_name, files_processed, files_created).

Details

The rewritten originals are scheduled for deletion once no snapshot references them; run cleanup_old_files() to remove them.

See also

cleanup_old_files(), checkpoint_ducklake()

Other maintenance: backup_ducklake(), checkpoint_ducklake(), cleanup_old_files(), delete_orphaned_files(), expire_snapshots(), flush_inlined_data(), merge_adjacent_files()

Examples

if (FALSE) { # \dontrun{
# Rewrite any file that is at least half deleted
rewrite_data_files("my_lake", delete_threshold = 0.5)

# Just one table, with DuckLake's default threshold
rewrite_data_files(table_name = "events")
} # }