From Bremen to Philadelphia: The rpact Story Continues

13th International Conference on Multiple Comparison Procedures

Daniel Sabanés Bové, Friedrich Pahlke, Gernot Wassmer

RCONIS

August 13, 2025

Looking back: Bremen 2022

  • The previous 12th International Conference on Multiple Comparison Procedures (MCP) was held in Bremen, Germany, in 2022
  • Gernot Wassmer and Friedrich Pahlke (RPACT) presented the R package rpact for confirmatory adaptive group sequential designs
    • “rpact: an R Package for Confirmatory Adaptive Group Sequential Designs”
    • fixed and group-sequential designs
    • adaptive designs (sample size reassessment, adaptive multi-arm multi-stage, population enrichment)
  • Based on version 3.2 of rpact

Today: Philadelphia

  • MCP 2025: The rpact Story Continues!
  • I have the honor to present the latest developments of rpact based on the work by Gernot Wassmer and Friedrich Pahlke
  • Based on latest release version 4.2.1 of rpact
  • You might wonder: How does Daniel relate to rpact and RPACT?

RCONIS 🚀

  • Statistical consulting and engineering services:
    Research Consulting and Innovative Solutions
  • Grow RPACT company to offer a wider range of services
  • Strengthen maintainer team for the rpact package
  • Team growth combined with scope growth
  • Joint venture between RPACT GbR (rpact.com) and inferential.biostatistics GmbH (inferential.bio)
  • Website: rconis.com

RCONIS Team

rpact

Overview 📦

  • Comprehensive validated R package implementing methodology described in Wassmer and Brannath (2016)
  • Enables the design of traditional and confirmatory adaptive group sequential designs
  • Provides interim data analysis including early efficacy stopping and futility analyses
  • Enables sample-size reassessment with different strategies
  • Enables treatment arm selection in multi-arm multi-stage (MAMS) designs
  • Enables subset selection in population enrichment designs
  • Provides a comprehensive and reliable sample size calculator

Developed by RPACT 🏢

  • RPACT company founded in 2017 by Gernot Wassmer and Friedrich Pahlke
  • Idea: open source development with help of “crowd funding”
  • Currently supported by 22 companies
  • \(>\) 80 presentations and training courses since 2018
    • including to FDA (March 2022)
  • 29 vignettes published on rpact.org/vignettes
  • 31 releases on CRAN since 2018

Trial Designs 🔬

  • Fixed sample designs:
    • continuous, binary, count, survival outcomes
  • Group sequential designs:
    • efficacy interim analyses, futility stopping, alpha-spending functions
  • Adaptive designs:
    • Inverse normal and Fisher’s combination test, conditional error rate principle
    • Provides adjusted confidence intervals and bias corrected estimates
    • Multi-arm multi-stage (MAMS) and enrichment designs, sample size reassessment

Easy to understand R commands:

getDesignGroupSequential()
getDesignInverseNormal()
getDesignFisher()
getDesignConditionalDunnett()

Sample Size and Power Calculation 💻

Sample size and power can be calulcated for testing:

  • means (continuous endpoint)
  • proportions (binary endpoint)
  • hazards (survival endpoint)
    • Note: flexible recruitment and survival time options
  • rates (count endpoint)

Easy to understand R commands:

getSampleSize[Means / Rates / Survival / Counts]()
getPower[Means / Rates / Survival / Counts]()

Example: Sample Size Calculation 🧮

# Define the design.
getDesignGroupSequential(
    typeOfDesign = "asOF",
    futilityBounds = c(0, 0)
) |>
    # Perform sample size calculation.
    getSampleSizeMeans(
        alternative = 2,
        stDev = 5
    ) |>
    # Obtain summary.
    summary()

Sample size calculation for a continuous endpoint

Sequential analysis with a maximum of 3 looks (group sequential design), one-sided overall significance level 2.5%, power 80%. The results were calculated for a two-sample t-test, H0: mu(1) - mu(2) = 0, H1: effect = 2, standard deviation = 5.

Stage 1 2 3
Planned information rate 33.3% 66.7% 100%
Cumulative alpha spent 0.0001 0.0060 0.0250
Stage levels (one-sided) 0.0001 0.0060 0.0231
Efficacy boundary (z-value scale) 3.710 2.511 1.993
Futility boundary (z-value scale) 0 0
Efficacy boundary (t) 4.690 2.152 1.384
Futility boundary (t) 0 0
Cumulative power 0.0204 0.4371 0.8000
Number of subjects 69.9 139.9 209.8
Expected number of subjects under H1 170.9
Overall exit probability (under H0) 0.5001 0.1309
Overall exit probability (under H1) 0.0684 0.4202
Exit probability for efficacy (under H0) 0.0001 0.0059
Exit probability for efficacy (under H1) 0.0204 0.4167
Exit probability for futility (under H0) 0.5000 0.1250
Exit probability for futility (under H1) 0.0480 0.0035

Legend:

  • (t): treatment effect scale

Adaptive Analysis 📈

  • Perform interim and final analyses during the trial using
    • group sequential method or
    • p-value combination test (inverse normal or Fisher’s method)
  • Calculate adjusted point estimates and confidence intervals
  • Perform sample size reassessment based on the observed data, based on calculation of conditional power

Adaptive Analysis (cont’d) 📈

  • Some highlights:
    • Automatic boundary recalculations during the trial for analysis with alpha spending approach, including under- and over-running
    • Adaptive analysis tools for multi-arm trials
    • Adaptive analysis tools for enrichment designs

Easy to understand R commands:

getStageResults()
getRepeatedConfidenceIntervals()
getAnalysisResults()

Simulation Tool 🧪

Obtain operating characteristics of different designs:

  • Assessment of adaptive sample size recalculation strategies
  • Assessment of treatment selection strategies in multi-arm trials
  • Assessment of population selection strategies in enrichment designs

Easy to understand R commands:

getSimulation[MultiArm / Enrichment][Means / Rates / Survival / Counts]()

Example:

getSimulationMeans()
getSimulationMultiArmMeans()
getSimulationEnrichmentMeans()

Package Validation Concept ✔️

Why is rpact a reliable R package?

  • Formal validation inspired by “GAMP 5” principles
  • As few dependencies as possible:
    • Imports: Rcpp, R6
    • Suggests: testthat, ggplot2, rappdirs
  • High test coverage:
  • testPackage(): installation qualification on a client computer or company server (with professional support, more later)

Professional Support Option 🤝

  • Service Level Agreement (SLA) with RPACT
  • Be part of the “RPACT User Group” and be invited to yearly customer meetings
  • Get technical software support for written support requests
  • Get company specific training
  • Get company specific software validation documentation for each rpact release on CRAN
  • Get access to the members area at connect.rpact.com
  • Make an rpact installation qualification on each company computer with your personal testPackage() token and secret
  • Determine the direction of rpact future development activities
  • Optional support for health authority interactions
  • Optional consultation to support specific clinical trials

Updates since Bremen

Count Data

  • getSampleSizeCounts() and getPowerCounts()
    • Perform sample size calculations and the assessment of test characteristics for clinical trials with negative binomial distributed count data.
    • Fixed sample size and group sequential designs (Mütze et al. 2019)
    • Perform blinded sample size reassessments according
      to Friede and Schmidli (2010)
    • Output boundary values also on the treatment effect scale
  • New function getSimulationCounts()
    • Perform power simulations for clinical trials with negative binomial distributed count data
    • Simulated power, stopping probabilities, conditional power, and expected sample size for testing mean rates for negative binomial distributed event numbers in the two treatment groups testing situation

Count Data: Options

  • You can provide either:
    • both event rates \(\lambda_1\) and \(\lambda_2\), or
    • \(\lambda_2\) and rate ratio \(\theta\), or
    • \(\lambda\) and \(\theta\) (blinded sample size reassessment!)
  • \(\lambda_1\) and \(\theta\) can be vectors
  • Different ways of calculation:
    • fixed exposure time,
    • accrual and study time,
    • or accrual and fixed number of patients
  • Staggered patient entry
  • Group sequential or fixed sample size setting

Count Data: Simple Example

getSampleSizeCounts(
    alpha = 0.05,
    beta = 0.1,
    sided = 2,
    lambda2 = 0.4,
    theta = 0.75,
    overdispersion = 0.5,
    fixedExposureTime = 1
) |> summary()

Count Data: Simple Example

Sample size calculation for a count data endpoint

Fixed sample analysis, two-sided significance level 5%, power 90%. The results were calculated for a two-sample Wald-test for count data, H0: lambda(1) / lambda(2) = 1, H1: effect = 0.75, lambda(2) = 0.4, overdispersion = 0.5, fixed exposure time = 1.

Stage Fixed
Stage level (two-sided) 0.0500
Efficacy boundary (z-value scale) 1.960
Lower efficacy boundary (t) 0.844
Upper efficacy boundary (t) 1.171
Lambda(1) 0.300
Number of subjects 1736.0
Maximum information 127.0

Legend:

  • (t): treatment effect scale

Count Data: Power Calculation

getPowerCounts(
    directionUpper = FALSE,
    lambda2 = 1.4,
    theta = c(0.75, 1),
    overdispersion = 0.7,
    maxNumberOfSubjects = 300,
    followUpTime = 12,
    accrualTime = 12
) |> fetch(overallReject)
$overallReject
[1] 0.8213803 0.0250000

Count Data: Power Simulation

tictoc::tic()
getSimulationCounts(
    directionUpper = FALSE,
    lambda2 = 1.4,
    theta = c(0.75, 1),
    overdispersion = 0.7,
    maxNumberOfSubjects = 300,
    plannedCalendarTime = 24,
    accrualTime = 12,
    fixedExposureTime = 12,
    # followUpTime = 12,
    maxNumberOfIterations = 1000,
    seed = 123
) |> fetch(overallReject)
$overallReject
[1] 0.835 0.028
tictoc::toc()
1.343 sec elapsed

Conditional Performance Score

  • Performance scores combine different performance criteria in one value (e.g. Liu, Zhu, and Cui (2008), Wu and Cui (2012))

  • Evaluation perspectives: global and conditional

    • global: looking at the properties of the adaptive design before the study starts
    • conditional: looking at the properties of the adaptive design when the interim result suggests a trial continuation

Conditional Performance Score (cont’d)

  • Idea of the conditional performance score:
    • Evaluation of second stage sample size update
    • Conditional power and sample size are random variables and therefore best described by location and variation measures
    • Score by Herrmann et al. (2020) consists of four components (location and variation of conditional power and sample size)
    • Score and its components take values between 0 and 1 (the higher the values, the better the performance)

Conditional Performance Score: Example

Suppose we have planned the following two-stage trial for a continuous endpoint:

Parameter Value
Design O’Brien-Fleming
\(I_1\) (information fraction) 0.5
\(\alpha\) 0.025
\(n\) 200
\(\sigma\) 1
\(l_1\) (futility bound) 0
  • We consider different \(H_1\) settings with effect size \(\delta \in \{0.2,0.3,0.4,0.5\}\)
  • At interim, we want to allow for sample size re-calculation based on the observed conditional power, but only up to \(n_{\text{max}} = 1.5 \cdot n = 300\) in total
  • We want to evaluate the performance of this design

Calculate Scores

# Initialize group sequential design with O'Brien-Fleming-boundaries
design <- getDesignGroupSequential(
    kMax = 2,
    informationRates = c(0.5, 1),
    alpha = 0.025,
    beta = 0.2,
    typeOfDesign = "OF",
    futilityBounds = 0,
    bindingFutility = TRUE
)

# Perform simulation
n <- 200
n1 <- n * design$informationRates[1]
alternative <- c(0.2, 0.3, 0.4, 0.5)
maxNumberOfSubjects <- 1.5 * n

design |>
    getSimulationMeans(
        normalApproximation = TRUE,
        thetaH0 = 0,
        alternative = alternative,
        plannedSubjects = c(n1, n),
        minNumberOfSubjectsPerStage = c(NA, 1),
        maxNumberOfSubjectsPerStage = c(NA, maxNumberOfSubjects),
        conditionalPower = 0.8,
        directionUpper = TRUE,
        maxNumberOfIterations = 1e05,
        seed = 123
    ) |>
    # Calculate performance score
    getPerformanceScore()

Calculate Scores

Performance

  • Location sample sizes: 0.2567, 0.7985, 0.8171, 0.7024
  • Variation sample sizes: 0.2989, 0.2334, 0.2271, 0.2819
  • Sub-score sample sizes: 0.2778, 0.5159, 0.5221, 0.4921
  • Location conditional power: 0.6938, 0.7887, 0.8698, 0.9274
  • Variation conditional power: 0.3610, 0.4194, 0.5189, 0.6285
  • Sub-score conditional power: 0.5274, 0.6041, 0.6944, 0.7779
  • Performance scores: 0.4026, 0.5600, 0.6082, 0.6350

Manual Results Tabulation

\(\delta = 0.2\) \(\delta = 0.3\) \(\delta = 0.4\) \(\delta = 0.5\)
Global Measures
Power 0.4522 0.7527 0.9118 0.9699
\(E[N]\) 279.3 257.2 215.2 169.8
Conditional Measures
\(e_N(\delta)\) 0.257 0.798 0.817 0.702
\(v_N(\delta)\) 0.299 0.233 0.227 0.282
\(e_{CP}(\delta)\) 0.694 0.789 0.87 0.927
\(v_{CP}(\delta)\) 0.361 0.419 0.519 0.628
\(S(\delta)\) 0.403 0.56 0.608 0.635

Simulation Improvements

  • Multi-Arm and Enrichment Design simulations:
    • Complete implementation of simulation tools for multi-arm (getSimulationMultiArmMeans/Rates/Survival()) and enrichment designs (getSimulationEnrichmentMeans/Rates/Survival())
    • Flexible treatment/population selection functions and dose-response modeling capabilities
  • Performance Optimization:
    • Significant speed improvements through C++ implementation of core simulation algorithms, reducing computation time for complex scenarios while maintaining numerical accuracy

Simulation Improvements (cont’d)

  • Enhanced Simulation Features:
    • Flexible allocationRatioPlanned as vectors for multi-stage designs
    • Custom calcSubjectsFunction and calcEventsFunction for user-defined calculations, e.g., promizing zone approach
    • Exact simulation versions for rate testing
  • Advanced Simulation Tools:
    • Improved simulation output with comprehensive statistics
    • Enhanced validation through expanded test coverage

RPACT Cloud

RPACT Cloud ☁️

  • Professional graphical user interface
  • Web based usage without local installation on nearly any device ☁️
  • Provides an easy entry to learn and demonstrate the usage of rpact
  • Starting point for your R Markdown or Quarto reports
  • Free demo version online available at cloud.rpact.com

Start Page

Design

Reporting

Export

Design Comparison

Professional Support Option 🤝

  • Server license available to run RPACT Cloud on company servers
  • One-click Installation Qualification on corporate servers via built-in UI features
  • Licensed already by three major pharmaceutical companies for internal deployment on their own infrastructures
  • Dedicated, company-specific GitHub repository with full documentation for installation and qualification (including e.g., Posit Connect setup)
  • Custom extensions and adaptations possible, including private R programs tailored to customer needs

What’s Next?

New Package Features

Just in the near future you will see in rpact:

  • Patient level survival data simulations also for multi-arm multi-stage and patient enrichment designs
  • Decision making for early phase trials with double confidence interval criteria
  • Least square means interface for continuous endpoints
  • Additional futility boundary scales

Second Edition of the Book

2nd edition coming in 2025 with R examples
  • Provides up-to-date overview of group sequential and confirmatory adaptive designs in clinical trials
  • Describes available software including R packages and has rpact code examples
  • Supplemented with a discussion of practical applications

Pre-order the book at Springer 🔗

Thank you!


Slides and contact 📫

These slides are at
rpact-com.github.io/slides-mcp-2025


Let’s connect:


Contact us:

References

Friede, Tim, and Heinz Schmidli. 2010. “Blinded Sample Size Reestimation with Count Data: Methods and Applications in Multiple Sclerosis.” Statistics in Medicine 29 (10): 1145–56. https://doi.org/https://doi.org/10.1002/sim.3861.
Herrmann, Carolin, Maximilian Pilz, Meinhard Kieser, and Geraldine Rauch. 2020. “A New Conditional Performance Score for the Evaluation of Adaptive Group Sequential Designs with Sample Size Recalculation.” Statistics in Medicine 39 (15): 2067–2100. https://doi.org/https://doi.org/10.1002/sim.8534.
Liu, G. Frank, G. Ray Zhu, and Lu Cui. 2008. “Evaluating the Adaptive Performance of Flexible Sample Size Designs with Treatment Difference in an Interval.” Statistics in Medicine 27 (4): 584–96. https://doi.org/https://doi.org/10.1002/sim.2998.
Mütze, Tobias, Ekkehard Glimm, Heinz Schmidli, and Tim Friede. 2019. “Group Sequential Designs for Negative Binomial Outcomes.” Statistical Methods in Medical Research 28 (8): 2326–47. https://doi.org/10.1177/0962280218773115.
Wassmer, Gernot, and Werner Brannath. 2016. Group Sequential and Confirmatory Adaptive Designs in Clinical Trials. Springer. https://link.springer.com/book/10.1007/978-3-319-32562-0.
Wassmer, Gernot, and Friedrich Pahlke. 2025. rpact: Confirmatory Adaptive Clinical Trial Design and Analysis. https://CRAN.R-project.org/package=rpact.
Wu, Xiaoru, and Lu Cui. 2012. “Group Sequential and Discretized Sample Size Re-Estimation Designs: A Comparison of Flexibility.” Statistics in Medicine 31 (24): 2844–57. https://doi.org/https://doi.org/10.1002/sim.5395.