jax_privacy.auditing.BootstrapParams

class jax_privacy.auditing.BootstrapParams(num_samples=1000, quantiles=(0.025, 0.975), bias_correction=True, acceleration=False, seed=None)[source]

Bases: object

Parameters for bootstrapping.

Several methods in this library that compute a privacy metric (e.g., AUROC, epsilon) optionally perform bootstrapping to estimate quantiles of the metric. The held-in and held-out canary scores are resampled with replacement, and metric is estimated from the resampled data. Repeating this sampling procedure many times produces an empirical distribution of the metric, from which we can estimate quantiles. This class configures the number of resamples, the quantiles to report, and the seed for the random number generator.

Optionally performs bias correction and acceleration for the bootstrap [B. Efron, “Bootstrap Confidence Intervals”, Statist. Sci. 2(3), 189-228 (1987)]. Be aware that acceleration requires jackknife resampling, which computes the function n times for n total canaries.

Variables:

  • num_samples – The number of times to resample the canary scores.

  • quantiles – An array-like of quantiles to report. E.g., for a 95% confidence interval, use (0.025, 0.975).

  • bias_correction – Whether to use bias correction.

  • acceleration – Whether to use acceleration.

  • seed – The seed for the random number generator.

Methods

__init__

confidence_interval

Creates a BootstrapParams object for a confidence interval.

Attributes

acceleration

bias_correction

num_samples

quantiles

seed

num_samples: int = 1000
quantiles: Union[_Buffer, _SupportsArray[dtype[Any]], _NestedSequence[_SupportsArray[dtype[Any]]], complex, bytes, str, _NestedSequence[complex | bytes | str]] = (0.025, 0.975)
bias_correction: bool = True
acceleration: bool = False
seed: int | None = None
classmethod confidence_interval(num_samples=1000, confidence=0.95, seed=None)[source]

Creates a BootstrapParams object for a confidence interval.

Parameters:

  • num_samples (int) – The number of times to resample the canary scores.

  • confidence (float) – The desired confidence level.

  • seed (int | None) – The seed for the random number generator.

Return type:

BootstrapParams

Returns:

A BootstrapParams object for computing a confidence interval.