tinytimemixer

tinytimemixer at 10 layers and hidden size 128. One published checkpoint has this shape.

Implementing it

One forward, written out, per checkpoint whose arithmetic actually differs. A shape groups checkpoints by class, depth, width and layer stack — none of which decides an activation, a rope base or a routing rule — so where the members of this shape disagree there is a block each, and each names the checkpoint it was generated from.

No forward was recorded for this shape, so there is no pass above to compare it against — this block is what the published configuration says the arithmetic is, not a transcript of a run. Where the configuration does not say, the line says that instead of guessing.

The names are the ones the published checkpoint uses where a name is published, and canonical otherwise. A checkpoint loader may store them differently — fusing a gate/up pair into one matrix, or splitting one published projection in two — and those are that loader's names, not the model's. Everything spelled here is what the download contains.

Generated from granite-timeseries-ttm-r3.

# Two full mixer stacks, not one. The checkpoint is a decomposition: a
# robust LOWESS-like smoother splits the scaled context into a trend and
# a residual, one stack forecasts each, and the two are added back.
# 468 tensors — 217 trend, 251 residual. Not a clean halving, because the
# residual stack is wider, has twice the decoder depth, and carries an
# adapter the trend stack does not need.

scaled = StdScaler(last 512 samples)
trend  = robustLowess(scaled)             # Gaussian + Tukey bisquare,
                                          # 2 passes, reflect padding
resid  = scaled[-90:] - trend[-90:]

# The trend stack is fed the signal, not the trend. `trend_signal` exists
# only to build the residual; the trend forecaster reads the same scaled
# series the smoother read. The reference has the other line commented
# out directly above it.
yt, qt = stack_trend(scaled)              # patch 64, d 8,  17 tokens
yr, qr = stack_resid(resid)               # patch 9,  d 128, 20 tokens

# Each stack patches at four resolutions. `MultiScaleFromPatchedSequence`
# average-pools by 1, 2, 4, 8 and projects each with its own Linear, then
# concatenates coarsest first — the reference builds the list
# fine-to-coarse and reverses it.
# Two learnable register tokens are prepended in front of all of them, so
# token 0 is not a patch of anything. (TSPulse appends its ten.)
# The LayerNorm after patching is over the whole flattened token grid,
# width n_patch * d_model, not per token.

point = (yt + yr) * scale + loc
band  = combineQuantiles(qt, qr) * scale + loc
#       ^ the two stacks' quantile ladders are combined by width in
#         quadrature, not added: they are independent forecasts.

This is two models. One mixer stack over the trend and one over the residual, with different patch lengths, different widths and different decoder depths. A sheet that showed one stack described neither.

The trend stack reads the signal, not the trend. The smoothed series exists only to build the residual. The reference has the plausible version commented out one line above, which is how easy it is to wire.

The register tokens are prepended here and appended in TSPulse. Two models in one directory, opposite conventions, and both load either way.

Each stack patches at four resolutions, concatenated coarsest first — the reference builds the list fine-to-coarse and reverses it, so the obvious order is the wrong one.

Geometry

How this was checked

implemented, evidence grade class, per the assessment: the architecture class is implemented; nothing specific to this checkpoint was measured. The implementation meant here and below is an unpublished independent inference implementation by the author.

What stands behind the block above, beyond the published configuration it is read from:

• a checkpoint loader, which is where published tensor names are read
• an independent reference implementation of this architecture in Python, driving the published modelling code — an executable statement of what the model should compute, written against the publication rather than against any one implementation of it

Where an independent implementation and the published configuration disagree, the configuration is what this page reports and the disagreement is what it says.

Checkpoints with this architecture