TinyTimeMixerForPrediction

TinyTimeMixerForPrediction at 2 layers and hidden size 192. 2 published checkpoints share this shape.

Implementing it

One forward, written out, per checkpoint whose arithmetic actually differs. A shape groups checkpoints by class, depth, width and layer stack — none of which decides an activation, a rope base or a routing rule — so where the members of this shape disagree there is a block each, and each names the checkpoint it was generated from.

No forward was recorded for this shape, so there is no pass above to compare it against — this block is what the published configuration says the arithmetic is, not a transcript of a run. Where the configuration does not say, the line says that instead of guessing.

The names are the ones the published checkpoint uses where a name is published, and canonical otherwise. A checkpoint loader may store them differently — fusing a gate/up pair into one matrix, or splitting one published projection in two — and those are that loader's names, not the model's. Everything spelled here is what the download contains.

Generated from granite-timeseries-ttm-r1, which covers all 2 checkpoints of this shape — they agree on every field below.

# A point forecaster with no attention and no convolution.

x = last 512 samples, front-padded if short
z = StdScaler(x, over observed values only)
p = patch(z, length=64)                # 8 patches
h = p @ W_emb.T                           # [8, 192]

# Adaptive patching — three levels, and it is a reshape, not a projection.
#   the [8, 192] canvas reinterpreted as [32, 48] / [16, 96] / [8, 192]
#   Each level runs its own two mixer layers over its own view.
for level in adaptive_levels:             # 3 of them
    h_l = reshape(h, level)
        # A mixer layer is two mixes, each:
        #   LayerNorm -> MLP(x2, exact GELU) -> softmax gate -> +residual
        # The gate is `y * softmax(Wy + b)` and it is the library default
        # `gate_mode`. Dropping it leaves a model that forecasts.
        h = mix(h, across=patches)        # MLP over the patch axis
        h = mix(h, across=features)       # MLP over the feature axis

h = h @ W_adapter.T                       # 192 -> 128
#   the decoder runs at 128, the encoder at 192 — two widths,
#   one adapter between them, and each mixer's MLP expands by 2x.
#   `patch stride 64` over 1 input channel(s): at stride == patch length the
#   patches do not overlap, which is this family's whole tiling.
for i in 0 .. decoder_layers-1:
        # A mixer layer is two mixes, each:
        #   LayerNorm -> MLP(x2, exact GELU) -> softmax gate -> +residual
        # The gate is `y * softmax(Wy + b)` and it is the library default
        # `gate_mode`. Dropping it leaves a model that forecasts.
        h = mix(h, across=patches)        # MLP over the patch axis
        h = mix(h, across=features)       # MLP over the feature axis

forecast = (flatten(h) @ W_head.T) * scale + loc   # [96 samples]

Adaptive patching is a reshape. The same canvas is reinterpreted at three resolutions and each level runs its own mixer layers; nothing is projected between them. A port that adds a projection has more weights than the checkpoint ships.

Each mix ends in a softmax gate, as in PatchTSMixer — the layer is the same function, and only the MLP's expansion differs between the two families.

The scaler reads observed values only, and the shipped forward front-pads a short series with zeros. Counting the pad as data drags the forecast toward zero.

The reversible normalization is load-bearing: the statistics come from the input window and are re-applied to the output, so a series is forecast in its own units without the model ever seeing them.

Geometry

How this was checked

implemented, evidence grade class, per the assessment, for each checkpoint of this shape: the architecture class is implemented; nothing specific to this checkpoint was measured. The implementation meant here and below is an unpublished independent inference implementation by the author.

What stands behind the block above, beyond the published configuration it is read from:

• a checkpoint loader, which is where published tensor names are read
• an independent reference implementation of this architecture in Python, driving the published modelling code — an executable statement of what the model should compute, written against the publication rather than against any one implementation of it

Where an independent implementation and the published configuration disagree, the configuration is what this page reports and the disagreement is what it says.

Checkpoints with this architecture