PatchTSMixerForPrediction

PatchTSMixerForPrediction at 2 layers and hidden size 48. One published checkpoint has this shape.

Implementing it

How to read this sheet: method.

No forward was recorded for this shape, so the block is read from the published configuration and from modeling_patchtsmixer.py in transformers v4.57.1, not transcribed from a run.

Generated from granite-timeseries-patchtsmixer.

# The backbone and a head — the smallest forecaster here. No attention
# anywhere (`self_attn: false`) and no positional encoding
# (`use_positional_encoding: false`).

x = last 512 samples
z, loc, scale = std_scaler(x, observed_mask)  # `scaling: true` selects std;
                                          #   scale = sqrt(var + 1e-5)
p = patch(z, length=16, stride=16)        # 32 patches
h = p @ W_emb.T                           # -> 48

for i in 0 .. 1:            # `mode: common_channel` — no channel mix
    # Each mix: LayerNorm -> MLP(x3, exact GELU) -> gate -> +residual
    # gate(y) = y * softmax(W y + b)          (`gated_attn: true`)
    h = mix(h, across=patches)        # MLP over the patch axis, 32 wide
    h = mix(h, across=features)       # MLP over d_model, 48 wide

y = flatten(h)                             # [32 x 48]
forecast = (y @ W_head.T) * scale + loc   # [96 samples]

Sources, transformers v4.57.1: the scaler choice and PatchTSMixerStdScaler; the gate; the MLP; the patch mix and feature mix; the prediction head; the rescale.

The scaler reads observed values only. Its mean and variance are taken over the points observed_mask marks; when no mask is passed the model treats every point as observed, so padding a short series without a mask folds the pad into the statistics.

Each mix ends in a softmax gate, y * softmax(Wy + b), switched on by gated_attn: true. It carries its own weights in the checkpoint.

Geometry

How this was checked

Status and evidence grade in the assessment (see what the grades mean):

• granite-timeseries-patchtsmixer: runs, class — the architecture class is implemented; nothing checkpoint-specific was measured

The implementation meant here and below is an unpublished independent inference implementation by the author.

What stands behind the block above, beyond the published configuration it is read from:

• a checkpoint loader, which is where published tensor names are read
• an independent reference implementation of this architecture in Python, driving the published modelling code — an executable statement of what the model should compute, written against the publication rather than against any one implementation of it

Where an independent implementation and the published configuration disagree, the configuration is what this page reports and the disagreement is what it says.

Checkpoints with this architecture