Renderingen-US

Experimental through version 1. Version 0 was alpha. This is the first language release: the interface is relatively stable, not frozen. Nothing is promised stable until version 2.

Gradus — autograd and ML

Gradus is Faber's library for automatic differentiation, losses, optimizers, neural-network primitives, and training mechanics. The name is the Latin for step, pace, degree — the root of gradient. It is imported as gradus:* and declared as a Cista dependency, the same way as Triga.

Gradus does not implement autograd. Reverse-mode differentiation is a compiler transform inside Radix: a function annotated @ radix backward gets a generated companion function that computes its gradients. Gradus wraps that mechanism in a library surface so model code calls functions instead of writing compiler annotations by hand.

The current package version is 0.1.0. Read the status section before planning work on it — the shipped surface is deliberately narrow.

JAX-shaped, not PyTorch-shaped#

Models are pure functions of the form (params, x) → y. Parameters are ordinary values you carry and pass; there is no module class hierarchy, no implicit parameter registry, and no runtime tape. The backward pass is generated code, not a replayed graph.

The practical consequence is that everything is explicit. A training step takes the current parameters, the trainable gradients, and a learning rate, and returns the updated parameters — nothing is mutated behind your back and nothing is discovered at runtime.

Gradus is also self-contained: it imports nothing from Norma or any sibling library, so a Gradus user imports only from gradus:*. The overlap with Norma's math is deliberate isolation, not duplication waiting to be cleaned up.

Modules#

Each gradus:<stem> import resolves to src/<stem>.fab. Like Norma and Triga, the layout is flat leaves with one concern per import path, and there is no type re-export — import the leaf that owns what you use.

ImportOwns
gradus:tensorTensor construction, shape/dtype facts, value storage, element access. Plain values, not autograd-aware
gradus:dtypeThe DType tag and casting contract
gradus:shapeShape representation and rules: broadcast, reshape, expand
gradus:mathPure operation families over tensors: elementwise, reduce, matmul, cast, concat/slice
gradus:parameterParameter identity and traversal
gradus:serializeVersioned bytes wire format
gradus:gradientThe @ radix backward wrapper: forward call plus its companion gradient call
gradus:lossLoss functions
gradus:optimizeOptimizers and learning-rate schedules
gradus:nnDifferentiable primitives: linear, activation, norm, embedding, dropout
gradus:attentionFacade; the leaves are gradus:attention/rope and gradus:attention/gqa
gradus:mlpTwo-layer MLP forward and training-path companion
gradus:transformerTransformer blocks, positional encoding, heads
gradus:trainTraining loops, checkpointing
gradus:metricsDefined metrics over tensor values
gradus:dataBatching, shuffling, tokenization helpers
gradus:storageOpaque encoded numeric storage
gradus:tokenizerFacade; the leaves are tokenizer/identity, tokenizer/unicode, tokenizer/bpe
gradus:samplingToken sampling policies
gradus:decode, gradus:generation, gradus:cacheDecode loop, generation, and KV-cache contracts (generation and cache are facades over leaves)
gradus:model/*Model admission and architecture leaves: GGUF, Safetensors, capsules, dense and MoE graphs
gradus:gradusPackage map facade only — holds no genera
import from "gradus:tensor" tensor
import from "gradus:math" math
import from "gradus:gradient" gradient
import from "gradus:loss" loss
import from "gradus:optimize" optimize

The gradus:tensor / gradus:gradient split is the one to internalise: tensor values are plain data, and differentiability is a property of the function you annotate, not of the values flowing through it.

Shapes#

Tensor functions are generic over their sizes. loss.mse takes tensor<f32, [M, N]> operands with size M, size N parameters, and nn.linear is generic over M, K and N the same way, so a model whose shapes are not already covered does not need a matching overload added to the library.

Alongside the typed-tensor forms, several operations have a _carrier variant over tensor.NumericBlock, the decoded F32 host-list carrier that the CPU and reference tiers execute on. Those variants report failure through a typed error (LossError, NnError) instead of a shape type.

Status#

Gradus is pre-1.0 and its API may change. What is proven is narrower than what the module list suggests: most of it is structural, with source contracts that check and are pinned by test tests, and a narrow set of executed proofs.

LayerState
Reverse-mode AD over AIR tensor opsShipped in Radix; the admitted VJP family and its gaps live in the Radix autograd crate, not in Gradus
Tensor, dtype, shape, parameter, serialize contractsShipped as versioned schemas
Pure operation families (gradus:math)Shipped — elementwise, reduce, matmul, cast, concat/slice
Loss, optimizer, NN primitivesShipped (structural), generic over tensor sizes
Attention and transformer blocksShipped (structural)
Training loop, metrics, checkpointingShipped (structural): SGD state and step, Checkpoint, accuracy / Metric
Model admissionShipped: GGUF and Safetensors admission into typed capsules
GPU trainingNot claimed; device execution and emission are Radix scope

The gradient seam — forward loss, generated companion backward across an import boundary, and a per-element comparison against finite differences — compiles and executes end to end, and the companion gradient matches finite differences to roughly 1e-11. That is the load-bearing correctness result behind everything above.

CPU training is correct and slow. Fast training is a device-execution question, and that path is owned elsewhere: see device execution for the `faber run --device` surface and the device kernel support summary for what the accepted proof actually covers.

Who it is for#

Gradus is for people defining differentiable models in Faber who would rather call library functions than hand-write compiler annotations, and for anyone validating the Radix autograd pipeline through a clean surface. It is not for production deployment, not for GPU-scale training, and not a place to port PyTorch code shape-for-shape.

The stated forcing function is a minimal GPT trained on Shakespeare — chosen because it runs on CPU for correctness while making the GPU gradient path's absence expensive enough to close.

Source#

github.com/faberlang/gradus — MIT, like every public Faber library. See Open source.