Export lineage as JSON

Serializes a lineage object to a small, stable JSON document: node ids with their columns and table type, plus one record per column-level edge. React Flow presentation details (positions, colors) are deliberately dropped, so the output is suitable for scripting with jq, committing to version control (a CI diff catches accidental provenance changes when a pipeline is edited), or feeding to a data catalog.

Usage

lineage_json(lineage, path = NULL, pretty = TRUE)

Arguments

lineage

The result of extract_lineage(), or any list with nodes and edges built with create_table_node() and create_column_edge().

path

Optional file to write the JSON to. When supplied, the string is returned invisibly.

pretty

If TRUE (the default), indent the output for readability. Use FALSE for a single-line document.

Value

A JSON string. With metadata (present on extract_lineage() results), nodes (objects with id, type, and columns), and edges (objects with source, source_column, target, and target_column).

See also

extract_lineage() to compute lineage automatically

Other lineage exporters: lineage_graphml()

Examples

lineage <- list(
  nodes = list(
    create_table_node("orders", c("order_id", "amount")),
    create_table_node("daily_totals", "total", table_type = "target")
  ),
  edges = list(
    create_column_edge("orders", "amount", "daily_totals", "total")
  )
)
lineage_json(lineage)
#> {
#>   "nodes": [
#>     {
#>       "id": "orders",
#>       "type": "source",
#>       "columns": ["order_id", "amount"]
#>     },
#>     {
#>       "id": "daily_totals",
#>       "type": "target",
#>       "columns": ["total"]
#>     }
#>   ],
#>   "edges": [
#>     {
#>       "source": "orders",
#>       "source_column": "amount",
#>       "target": "daily_totals",
#>       "target_column": "total"
#>     }
#>   ]
#> } 

# Write to a file instead
path <- tempfile(fileext = ".json")
lineage_json(lineage, path = path)
extract_lineage("SELECT customer_id, SUM(amount) AS total
                 FROM orders GROUP BY customer_id") |>
  lineage_json()
#> {
#>   "metadata": {
#>     "sql": "SELECT customer_id, SUM(amount) AS total\n                 FROM orders GROUP BY customer_id",
#>     "dialect": "duckdb",
#>     "engine": "sqlglot",
#>     "table_count": 2,
#>     "edge_count": 2
#>   },
#>   "nodes": [
#>     {
#>       "id": "orders",
#>       "type": "source",
#>       "columns": ["customer_id", "amount"]
#>     },
#>     {
#>       "id": "output",
#>       "type": "target",
#>       "columns": ["customer_id", "total"]
#>     }
#>   ],
#>   "edges": [
#>     {
#>       "source": "orders",
#>       "source_column": "customer_id",
#>       "target": "output",
#>       "target_column": "customer_id"
#>     },
#>     {
#>       "source": "orders",
#>       "source_column": "amount",
#>       "target": "output",
#>       "target_column": "total"
#>     }
#>   ]
#> }