Skip to contents

Validation and regulatory use

Source: vignettes/articles/validation.Rmd
validation.Rmd

This article collects the facts a risk assessor needs when qualifying ltsummary for a validated statistical computing environment: what the package depends on, how its statistics are tested, and where the evidence lives. It does not claim the package is “validated”. Validation attaches to a sponsor’s environment and process, not to a package, and the assessment is the sponsor’s to make.

The regulatory frame

FDA’s Statistical Software Clarifying Statement (May 2015) settles the threshold question: no specific software is required for statistical analyses, and none is certified. The statement creates one affirmative obligation, to document the software name, version, and build in the submission, and points to ICH E9 for the rest: the software should be reliable, and documentation of software testing procedures should be available.

What “reliable” means in practice comes from risk-based qualification of the computing environment. For R, the reference approach is the R Validation Hub’s white paper (A Risk-based Approach for Assessing R package Accuracy within a Validated Infrastructure, 2020). Base R and the recommended packages are treated as low risk on the strength of the R Foundation’s documented development process; qualify the installation rather than re-validate the code. Every contributed package intended for use needs a documented risk assessment covering its purpose, maintenance practice, community usage, and testing, and the whole dependency tree has to be pinned and reproducible.

That model is why the size of a dependency tree is a regulatory quantity. Each contributed package in the tree is one more assessment to write, one more upgrade to put through change control, and one more version to pin.

The dependency footprint

ltsummary imports lt, stats, and utils. The recursive hard-dependency closure beyond base R is two packages, and none of it is compiled:

db <- installed.packages()
tools::package_dependencies(c("ltsummary", "gtsummary"), db = db,
                            which = c("Depends", "Imports", "LinkingTo"),
                            recursive = TRUE)

Run against a library holding ltsummary 0.8.0 and gtsummary 2.6.1, the closure for ltsummary is lt and xfun, both pure R. The closure for gtsummary is 62 packages, including C and C++ toolchain dependencies (cpp11, Rcpp, stringi, V8, curl). Both trees produce the same tables; the difference is the surface a qualification has to cover and keep covering at every upgrade.

Every statistic ltsummary reports is computed in base R, almost entirely through stats. The two survival features are the exception: tbl_survfit() requires the survival package (a recommended package, inside the Hub’s trusted set), and nothing else reaches outside base R at run time.

How the statistics are tested

The testing policy is recorded in decisions/0016 and separates two kinds of claim.

Quantities (estimates, confidence limits, p-values, counts) are tested against independent oracles executed inside the test: a stats:: or survival:: call, or hand arithmetic written out in a comment. A regression table’s cells are compared to coef(), confint(), and vcov() on a refit of the same model; a p-value column is compared to the matching t.test() or fisher.test(); a Kaplan-Meier cell is compared to summary(survfit(...)). The review criterion is that every expected number can be checked without any other table package installed. Where gtsummary hands a statistic to another package, that package is installed as a second oracle in Suggests and compared directly: smd for the standardized mean difference, car for the variance inflation factors and effectsize for the effect sizes. The results tables of tbl_rift() follow the same rule: sandwich for the HC0 covariance, risks for the risk ratios and differences, cardx for the survival difference and the incidence rate, and rifttable for every cell of its own article designs.

Formatting rules (how many decimals, when “<0.001” appears) are tested against the documented rounding rules, with the boundary cases hand-derived in decimal arithmetic and annotated in the test.

Two further layers back this up. Snapshot tests pin the full numeric body of each table type as a regression net, and a live parity suite compares ltsummary’s output cell for cell against gtsummary 2.6.1 across a configuration matrix whenever gtsummary is installed, and a second one against rifttable 0.7 whenever rifttable is. The parity suite is compatibility evidence, not accuracy evidence: it shows the two packages agree on the displayed convention, while the oracle tests carry the claim that the numbers are right.

The suite runs on every push across macOS, Windows, and three Ubuntu R versions through GitHub Actions, and R CMD check must pass with no errors, warnings, or notes. Coverage is checked before each release against a 90% target, with 100% required for the formatting and summary-statistic engines.

For a package risk assessment

Inputs an assessor will want:

  • Purpose: statistical summary tables; the package computes statistics, so under the Hub model it warrants assessment by a statistician.
  • Testing: the oracle policy above; the test suite ships in the package (tests/testthat/) and runs at install time via R CMD check.
  • Maintenance: development happens in the open at github.com/tgerke/ltsummary, with design decisions recorded as numbered entries in decisions/ and behavior pinned to gtsummary 2.6.1 (decisions/0008 and decisions/0021).
  • Community usage: low. The package is new, and on this Hub criterion it scores below gtsummary, which is widely used. A sponsor weighing the two is trading assessment surface (2 packages against 62) against usage history, and should say so explicitly in the assessment.

For the submission record itself, document the package version with sessionInfo() output as usual, and pin the environment (renv or an equivalent lockfile) so the table code reruns against the same versions.

Tables in a submission document

The eCTD technical specifications call for PDF documents, and FDA’s Portable Document Format (PDF) Specifications (v4.1, September 2016, nonbinding) say what a PDF should look like. lt renders HTML, so a table reaches a PDF by one of two routes; the Quarto and R Markdown article has the code for both.

For a report or an appendix, write a Quarto document, set options(lt.lt_static = list(fragment = TRUE, css = FALSE)) before the first table, and render to PDF, either through LaTeX or with format: typst, which needs no LaTeX installation. Quarto converts each baked HTML table into a native table (a longtable in LaTeX, so a long table paginates with its header repeated). The conversion drops the inline styling: the indentation of the levels under a variable, and the bold or italics that bold_labels() and its relatives add. Column headers keep their bold. Check a rendered table for that before it goes into a submission. Without the option the table is absent from the PDF, with no warning.

For a table as a file of its own, lt::lt_export(as_lt(tbl), "table.pdf") writes a vector PDF through a headless Chromium-based browser, cropped to the table by default or on letter-size pages with crop = FALSE.

Against the PDF specification:

  • Version: 1.4 through 1.7 is acceptable. LaTeX writes 1.5, typst 1.7, the browser 1.4.
  • Text searchable, no JavaScript: all routes. The static table carries none of lt’s JavaScript.
  • Page size 8.5 by 11 inches, left margin at least 3/4 inch, other margins at least 3/8 inch: the Quarto route takes the document’s page setup. A cropped browser export is sized to the table and is meant to be placed into a document, not submitted as one; with crop = FALSE the browser prints letter-size pages, and an @page rule in a stylesheet sets the margins (the article shows one).
  • Fonts: non-standard fonts fully embedded, with Times New Roman, Arial and Courier New recommended. Every route embeds a subset of each font, as Word and LaTeX generally do. lt names no font, so a browser export uses the system font unless a stylesheet sets one; a Quarto document uses its own.
  • Sizes: 9 to 12 point, 9 to 10 for tables, 10 for footnotes, with Times New Roman 12 recommended for narrative (ICH M4 General Principles say the same). lt’s sizes are relative to the browser default of 16 pixels, so cells are 12 point and footnotes about 10; a stylesheet brings cells to 10.
  • Navigation: documents of five pages or more need a table of contents and bookmarks. Quarto’s PDF and typst formats provide both with toc: true.

Two runtime dependencies belong in the environment qualification although neither is an R package: baking the static table needs Node.js or a Chromium-based browser, and lt_export() to PDF or PNG needs the browser. Record the version next to the sessionInfo() output.

Scope limits

The statistical scope is the gtsummary interface for data frames: tbl_summary(), tbl_continuous(), tbl_cross(), the hierarchical tables, tbl_likert(), tbl_wide_summary(), tbl_custom_summary(), the regression tables for lm, glm, and coxph, tbl_survfit(), and their add-on functions. Survey designs, multiply-imputed models, and mixed models are out of scope, and the migration article lists every function that is deliberately not mirrored. Where ltsummary’s behavior departs from gtsummary on purpose (upstream bugs it fixes rather than reproduces), the departure is recorded in decisions/ and in the migration article.