PatchTSTForPrediction

PatchTSTForPrediction at 3 layers and hidden size 128. One published checkpoint has this shape.

Implementing it

How to read this sheet: method.

No forward was recorded for this shape, so the block is read from the published configuration and from modeling_patchtst.py in transformers v4.57.1, not transcribed from a run.

Generated from granite-timeseries-patchtst.

# A transformer over patches, and the only forecaster here with a CLS
# token. Nothing is a text token and there is no vocabulary.

z, loc, scale = std_scaler(last 512 samples)  # observed values only;
                                          #   scale = sqrt(var + 1e-5)
p = patch(z[8:], length=12, stride=12)    # 42 patches over the last 504
h = p @ W_emb.T                           # -> 128

# `position_enc` ships in the checkpoint as a [43, 128] table (sincos,
# centred and scaled, is only how it was initialised). Row 0 belongs to
# the CLS token; patches take rows 1..42.
h = h + position_enc[1:]
h = prepend(cls_token + position_enc[:1], h)

for i in 0 .. 2:            # pre-norm
    h = h + attn(bn(h))               # 16 heads, scale head_dim^-0.5, no mask
    h = h + ff(bn(h))                 # 128 -> 512 -> 128, exact erf GELU

# `bn` is BatchNorm1d over d_model, eps 1e-5; in eval it is a fixed
# affine per feature from the shipped running statistics:
#   bn(x) = (x - running_mean)/sqrt(running_var + eps) * w + b

y = h[0] @ W_head.T                   # the CLS row only, not a flatten
forecast = y * scale + loc            # [96 samples]

Sources, transformers v4.57.1: PatchTSTStdScaler; PatchTSTPatchify (42 patches of 12 cover 504 points, so the first 8 of the 512 are dropped); PatchTSTPositionalEncoding; PatchTSTBatchNorm; the pre-norm encoder layer; the CLS pooling in the head; the rescale.

The norm is BatchNorm, not LayerNorm (norm_type: "batchnorm"), and in eval mode it is a fixed affine per feature read from the running_mean/running_var the checkpoint ships. There is no reduction over the sequence; treating it as LayerNorm reduces over d_model and gives something plausible and wrong.

The forecast comes from the CLS row (use_cls_token: true, pooling_type: null), not from flattening the patch grid.

Row 0 of the position table is the CLS token's. Patches take rows 1.., so adding row 0 to patch 0 shifts every position by one.

Geometry

How this was checked

Status and evidence grade in the assessment (see what the grades mean):

• granite-timeseries-patchtst: runs, class — the architecture class is implemented; nothing checkpoint-specific was measured

The implementation meant here and below is an unpublished independent inference implementation by the author.

What stands behind the block above, beyond the published configuration it is read from:

• a checkpoint loader, which is where published tensor names are read
• an independent reference implementation of this architecture in Python, driving the published modelling code — an executable statement of what the model should compute, written against the publication rather than against any one implementation of it

Where an independent implementation and the published configuration disagree, the configuration is what this page reports and the disagreement is what it says.

Checkpoints with this architecture