TinyTimeMixerForPrediction

TinyTimeMixerForPrediction at 2 layers and hidden size 192. 2 published checkpoints share this shape.

Implementing it

How to read this sheet: method.

No forward was recorded for this shape, so the block is read from the published configuration and from TinyTimeMixerForPrediction in granite-tsfm v0.3.9, not transcribed from a run.

Generated from granite-timeseries-ttm-r1. ttm-r2 computes the same function: its configuration differs only in dropout rates, initialiser choices, fcm_* keys that are inactive with enable_forecast_channel_mixing: false, and d_model_scale/decoder_d_model_scale/stride_ratio, which v0.3.9 does not read — none of which enters an inference forward.

# A point forecaster with no attention and no convolution.

x = last 512 samples                      # a shorter series is left-padded
                                          #   with zeros (see below)
z, loc, scale = std_scaler(x, observed_mask)   # sqrt(var + 1e-5)
p = patch(z, length=64, stride=64)        # 8 patches
h = p @ W_emb.T                           # [8, 192]

# Adaptive patching — three levels, and it is a reshape, not a projection.
#   the [8, 192] canvas reinterpreted as [32, 48], then [16, 96], then
#   [8, 192]; each level runs its own two mixer layers and reshapes back.
for factor in (4, 2, 1):
    h = reshape(h, [8 * factor, 192 // factor])
    for i in 0 .. 1:
        h = mix(h, across=patches)        # MLP over the patch axis
        h = mix(h, across=features)       # MLP over the feature axis
    h = reshape(h, [8, 192])

h = h @ W_adapter.T                       # 192 -> 128, `decoder.adapter`
for i in 0 .. 1:                          # decoder, no adaptive patching
    h = mix(h, across=patches)
    h = mix(h, across=features)

# Every mix: LayerNorm -> MLP(x2, exact GELU) -> gate -> +residual,
#   gate(y) = y * softmax(W y + b)        (`gated_attn: true`)

forecast = (flatten(h) @ W_head.T) * scale + loc   # [96 samples]

Sources, granite-tsfm v0.3.9: the context trim and left-pad; the scaler choice and TinyTimeMixerStdScaler; the adaptive-patching block, built coarsest level first; the MLP; the gate, whose gate_mode defaults to softmax when the configuration does not set it, as here; the adapter.

Adaptive patching is a reshape. The same canvas is reinterpreted at three resolutions and each level runs its own mixer layers; nothing is projected between them. A port that adds a projection has more weights than the checkpoint ships.

Each mix ends in a softmax gate, as in PatchTSMixer; only the MLP's expansion (2 here, 3 there) differs between the two layers.

The scaler reads observed values only — if it is told which they are. A series shorter than 512 is left-padded with zeros, and the pad is marked unobserved only when the caller passes past_observed_mask; without one, the model builds an all-ones mask after padding and the zeros enter the statistics, dragging the forecast toward zero.

Geometry

How this was checked

Status and evidence grade in the assessment (see what the grades mean):

• each of the 2 checkpoints of this shape: runs, class — the architecture class is implemented; nothing checkpoint-specific was measured

The implementation meant here and below is an unpublished independent inference implementation by the author.

What stands behind the block above, beyond the published configuration it is read from:

• a checkpoint loader, which is where published tensor names are read
• an independent reference implementation of this architecture in Python, driving the published modelling code — an executable statement of what the model should compute, written against the publication rather than against any one implementation of it

Where an independent implementation and the published configuration disagree, the configuration is what this page reports and the disagreement is what it says.

Checkpoints with this architecture