PatchTSTForPrediction

PatchTSTForPrediction at 3 layers and hidden size 128. One published checkpoint has this shape.

Implementing it

One forward, written out, per checkpoint whose arithmetic actually differs. A shape groups checkpoints by class, depth, width and layer stack — none of which decides an activation, a rope base or a routing rule — so where the members of this shape disagree there is a block each, and each names the checkpoint it was generated from.

No forward was recorded for this shape, so there is no pass above to compare it against — this block is what the published configuration says the arithmetic is, not a transcript of a run. Where the configuration does not say, the line says that instead of guessing.

The names are the ones the published checkpoint uses where a name is published, and canonical otherwise. A checkpoint loader may store them differently — fusing a gate/up pair into one matrix, or splitting one published projection in two — and those are that loader's names, not the model's. Everything spelled here is what the download contains.

Generated from granite-timeseries-patchtst.

# A transformer over patches, and the only forecaster here with a CLS
# token. Nothing is a text token and there is no vocabulary.

z = StdScaler(last 512 samples, over observed values only)
p = patch(z, length=12, stride=12)   # 42 patches
h = p @ W_emb.T                           # -> 128

# Positions are a stored table, not computed sincos, and row 0 belongs to
# the CLS token. Adding row 0 to patch 0 is the obvious off-by-one and it
# shifts every position by one patch.
h = h + position_enc[1:]                  # position_enc is [43, 128]
h = prepend(cls_token + position_enc[:1], h)

for i in 0 .. 2:            # pre-norm
    h = h + attn(bn(h))               # 16 heads, scale head_dim^-0.5, no mask
    h = h + ff(bn(h))                 # gelu, the exact erf form

# `bn` is BatchNorm and in eval that is not a normalisation.
#   bn(x) = (x - running_mean)/sqrt(running_var + eps) * w + b
# A fixed affine per feature, the same constants for every patch, with no
# reduction over the sequence. Treating it as LayerNorm reduces over
# d_model and produces something plausible and wrong.

y = h[0] @ W_head.T                   # the CLS row only, not a flatten
forecast = y * scale + loc            # [96 samples]

The norm is BatchNorm, not LayerNorm, and in eval mode it is a fixed affine per feature read from running_mean/running_var that the checkpoint ships. There is no reduction over the sequence at all.

The forecast comes from the CLS row, not from flattening the patch grid. The other families here flatten; this one prepends a token and reads it back.

The position table is stored and row 0 is the CLS token's. Patches take rows 1.., so adding row 0 to patch 0 shifts every position by one.

The reversible normalization is load-bearing: the statistics come from the input window and are re-applied to the output, so a series is forecast in its own units without the model ever seeing them.

Geometry

How this was checked

implemented, evidence grade class, per the assessment: the architecture class is implemented; nothing specific to this checkpoint was measured. The implementation meant here and below is an unpublished independent inference implementation by the author.

What stands behind the block above, beyond the published configuration it is read from:

• a checkpoint loader, which is where published tensor names are read
• an independent reference implementation of this architecture in Python, driving the published modelling code — an executable statement of what the model should compute, written against the publication rather than against any one implementation of it

Where an independent implementation and the published configuration disagree, the configuration is what this page reports and the disagreement is what it says.

Checkpoints with this architecture