jax_privacy.training.DPTrainer#
- class jax_privacy.training.DPTrainer(*, config, performance_flags=<factory>, loss_fn, optimizer, compilation_strategy=<factory>, padding_example=None, noise_seed=None)[source]#
Bases:
objectStateless trainer encapsulating the static configuration of a DP loop.
DPTrainerseparates configuration (plan, loss, optimizer) from per-run state (data, initial params, RNG seed). This makes thetrain_stepmethod available as a standalone callable that can be compiled or used independently of the training loop.Sharding: For multi-device training inside a
jax.set_mesh()context, provideparamssharded on the active mesh and configurebatch_axis_namethroughperformance_flags. Whenbatch_axis_nameis set and a mesh is active,fit()shards the batch dimension of each batch and padding mask alongPartitionSpec(batch_axis_name). In multi-controller setups (jax.process_count() > 1), every process must pass the fulldatasetand an identicalrng_or_seedtofit().- Variables:
config – An
ExecutionPlanConfig(e.g.BandMFConfig) specifying the DP mechanism.performance_flags – Performance-only flags (numerical precision, sharding, memory/compute trade-offs) that do not affect the privacy guarantee.
loss_fn – The per-example loss function. See
LossFn.optimizer – An
AugmentedGradientTransformationor a plainoptax.GradientTransformation.compilation_strategy – Selects which
train_stepprograms are compiled for training.PadToMultiple(default) pads batches to a multiple to bound recompilations;AutotuneMicrobatchpicks the largest microbatch size that fits device memory and compiles once. SeeCompilationStrategy.padding_example – Optional single-example PyTree specification (arrays or
jax.ShapeDtypeStruct) defining the structure, shapes, and dtypes of zero-padded examples. If omitted on agrain.MapDataset, it is inferred fromdataset[0], which requires a non-empty dataset with fixed element shapes.noise_seed – Optional seed for the random number generator used for noise addition. If
None, a fresh random seed is drawn; pass an explicit seed for multi-host coordination or reproducible tests (and keep it secret in production deployments).
Methods
__init__Runs an end-to-end differentially private training loop.
Initialize a
TrainingStateat step 0.Executes a single DP training step.
Attributes
DPExecutionPlanbuilt fromconfigandperformance_flags.-
config:
ExecutionPlanConfig#
-
performance_flags:
PerformanceFlags#
-
optimizer:
AugmentedGradientTransformation|GradientTransformation#
-
compilation_strategy:
PadToMultiple|AutotuneMicrobatch#
-
padding_example:
Any= None#
-
noise_seed:
int|None= None#
- property plan: DPExecutionPlan#
DPExecutionPlanbuilt fromconfigandperformance_flags.
- train_step(state, batch, is_padding_example, prng_key)[source]#
Executes a single DP training step.
This method is a pure function of its inputs and is safe to wrap with
jax.jit,jax.jit(...).lower(), or any other JAX transformation.- Parameters:
state (
TrainingState) – CurrentTrainingState.batch (
Union[Array,ndarray,bool,number,bool,int,float,complex,Iterable[optax.ArrayTree],Mapping[Any, optax.ArrayTree]]) – A PyTree of arrays representing the current mini-batch.is_padding_example (
Array) – A boolean array indicating which examples inbatchare padding (and should be ignored).prng_key (
Array) – Base PRNG key; a step-specific key is derived viajax.random.fold_in(prng_key, state.step).
- Return type:
tuple[TrainingState,Aux]- Returns:
A tuple
(new_state, aux)wherenew_stateis the updatedTrainingStateandauxis the per-example auxiliary output.
- fit(dataset, state, *, callback=None, rng_or_seed=None, precompile=True)[source]#
Runs an end-to-end differentially private training loop.
- Parameters:
dataset (
Union[Array,ndarray,bool,number,bool,int,float,complex,Iterable[optax.ArrayTree],Mapping[Any, optax.ArrayTree],Any]) – The training dataset, either as a PyTree of arrays where the first axis of each leaf is the batch / example dimension, or as agrain.MapDataset(orgrain.RandomAccessDataSource) where each element is a PyTree of arrays for a single example.state (
TrainingState) – Initial or resumableTrainingState(e.g., frominit()). Training starts or continues fromstate.step, and the batch selection iterator safely fast-forwards to maintain deterministic DP properties. Note thatstatebuffers are donated during training (train_step()usesdonate_argnames=["state"]), so copy any arrays you need to retain afterfitreturns.callback (
Callable[[int,TrainingState,Aux],None] |None) – Called after each step ascallback(step, state, aux).stepis a Python int.rng_or_seed (
Generator|int|None) – Optional random seed ornumpy.random.Generator, used for sampling batches (impacting privacy) and initializing the loss PRNG key (potentially impacting utility). Does not influence the noise addition transform, which is configured via theDPExecutionPlan.precompile (
bool) – A boolean indicating whether to asynchronously precompiletrain_stepfor the batch sizes encountered, instead of just-in-time compiling on the fly, which can idle accelerators during training. Strategies that resolve before training (e.g.AutotuneMicrobatch) run even when this isFalse.
- Return type:
- Returns:
Final
TrainingState.