jax_privacy.training.DPTrainer#

class jax_privacy.training.DPTrainer(*, config, performance_flags=<factory>, loss_fn, optimizer, compilation_strategy=<factory>, padding_example=None, noise_seed=None)[source]#

Bases: object

Stateless trainer encapsulating the static configuration of a DP loop.

DPTrainer separates configuration (plan, loss, optimizer) from per-run state (data, initial params, RNG seed). This makes the train_step method available as a standalone callable that can be compiled or used independently of the training loop.

Sharding: For multi-device training inside a jax.set_mesh() context, provide params sharded on the active mesh and configure batch_axis_name through performance_flags. When batch_axis_name is set and a mesh is active, fit() shards the batch dimension of each batch and padding mask along PartitionSpec(batch_axis_name). In multi-controller setups (jax.process_count() > 1), every process must pass the full dataset and an identical rng_or_seed to fit().

Variables:
  • config – An ExecutionPlanConfig (e.g. BandMFConfig) specifying the DP mechanism.

  • performance_flags – Performance-only flags (numerical precision, sharding, memory/compute trade-offs) that do not affect the privacy guarantee.

  • loss_fn – The per-example loss function. See LossFn.

  • optimizer – An AugmentedGradientTransformation or a plain optax.GradientTransformation.

  • compilation_strategy – Selects which train_step programs are compiled for training. PadToMultiple (default) pads batches to a multiple to bound recompilations; AutotuneMicrobatch picks the largest microbatch size that fits device memory and compiles once. See CompilationStrategy.

  • padding_example – Optional single-example PyTree specification (arrays or jax.ShapeDtypeStruct) defining the structure, shapes, and dtypes of zero-padded examples. If omitted on a grain.MapDataset, it is inferred from dataset[0], which requires a non-empty dataset with fixed element shapes.

  • noise_seed – Optional seed for the random number generator used for noise addition. If None, a fresh random seed is drawn; pass an explicit seed for multi-host coordination or reproducible tests (and keep it secret in production deployments).

Methods

__init__

fit

Runs an end-to-end differentially private training loop.

init

Initialize a TrainingState at step 0.

train_step

Executes a single DP training step.

Attributes

noise_seed

padding_example

plan

DPExecutionPlan built from config and performance_flags.

config

performance_flags

loss_fn

optimizer

compilation_strategy

config: ExecutionPlanConfig#
performance_flags: PerformanceFlags#
loss_fn: LossFn#
optimizer: AugmentedGradientTransformation | GradientTransformation#
compilation_strategy: PadToMultiple | AutotuneMicrobatch#
padding_example: Any = None#
noise_seed: int | None = None#
property plan: DPExecutionPlan#

DPExecutionPlan built from config and performance_flags.

init(params)[source]#

Initialize a TrainingState at step 0.

Return type:

TrainingState

train_step(state, batch, is_padding_example, prng_key)[source]#

Executes a single DP training step.

This method is a pure function of its inputs and is safe to wrap with jax.jit, jax.jit(...).lower(), or any other JAX transformation.

Parameters:
  • state (TrainingState) – Current TrainingState.

  • batch (Union[Array, ndarray, bool, number, bool, int, float, complex, Iterable[optax.ArrayTree], Mapping[Any, optax.ArrayTree]]) – A PyTree of arrays representing the current mini-batch.

  • is_padding_example (Array) – A boolean array indicating which examples in batch are padding (and should be ignored).

  • prng_key (Array) – Base PRNG key; a step-specific key is derived via jax.random.fold_in(prng_key, state.step).

Return type:

tuple[TrainingState, Aux]

Returns:

A tuple (new_state, aux) where new_state is the updated TrainingState and aux is the per-example auxiliary output.

fit(dataset, state, *, callback=None, rng_or_seed=None, precompile=True)[source]#

Runs an end-to-end differentially private training loop.

Parameters:
  • dataset (Union[Array, ndarray, bool, number, bool, int, float, complex, Iterable[optax.ArrayTree], Mapping[Any, optax.ArrayTree], Any]) – The training dataset, either as a PyTree of arrays where the first axis of each leaf is the batch / example dimension, or as a grain.MapDataset (or grain.RandomAccessDataSource) where each element is a PyTree of arrays for a single example.

  • state (TrainingState) – Initial or resumable TrainingState (e.g., from init()). Training starts or continues from state.step, and the batch selection iterator safely fast-forwards to maintain deterministic DP properties. Note that state buffers are donated during training (train_step() uses donate_argnames=["state"]), so copy any arrays you need to retain after fit returns.

  • callback (Callable[[int, TrainingState, Aux], None] | None) – Called after each step as callback(step, state, aux). step is a Python int.

  • rng_or_seed (Generator | int | None) – Optional random seed or numpy.random.Generator, used for sampling batches (impacting privacy) and initializing the loss PRNG key (potentially impacting utility). Does not influence the noise addition transform, which is configured via the DPExecutionPlan.

  • precompile (bool) – A boolean indicating whether to asynchronously precompile train_step for the batch sizes encountered, instead of just-in-time compiling on the fly, which can idle accelerators during training. Strategies that resolve before training (e.g. AutotuneMicrobatch) run even when this is False.

Return type:

TrainingState

Returns:

Final TrainingState.