PatchTSTFMForPrediction

PatchTSTFMForPrediction at 20 layers and hidden size 1024. One published checkpoint has this shape.

Implementing it

One forward, written out, per checkpoint whose arithmetic actually differs. A shape groups checkpoints by class, depth, width and layer stack — none of which decides an activation, a rope base or a routing rule — so where the members of this shape disagree there is a block each, and each names the checkpoint it was generated from.

No forward was recorded for this shape, so there is no pass above to compare it against — this block is what the published configuration says the arithmetic is, not a transcript of a run. Where the configuration does not say, the line says that instead of guessing.

The names are the ones the published checkpoint uses where a name is published, and canonical otherwise. A checkpoint loader may store them differently — fusing a gate/up pair into one matrix, or splitting one published projection in two — and those are that loader's names, not the model's. Everything spelled here is what the download contains.

Generated from granite-timeseries-patchtst-fm-r1.

# There is no separate horizon. The last `prediction_length` points of
# the input are masked and the model fills them, so the mask is an input
# rather than a convention: it drives the scaler's statistics, it zeroes
# the masked values, and its complement is half of what the model reads.

x = last 8192 samples
mask = [0]*(8064) + [1]*(128)       # 128 masked, not 64
#      ^ the masked block is 128 and the forecast is 64. The wrapper asks
#        for max(pred_len, d_patch * max(pretrain_mask_cont, 2)) =
#        max(64, 16 * 8) = 128 positions and returns the first 64.
#        Masking 64 instead changes both the scaler's statistics —
#        they are taken over the unmasked points — and the question the
#        model is asked, and forecasts either way.

# The scaler squashes, and the std is clamped rather than epsilon'd.
mean, std = stats(x, where=~mask, divisor=N)   # N, not torch's N-1
std = std if std > 1e-5 else 1                # a flat series gets 1
z = asinh((x - mean) / std)                   # `use_sinh`; the inverse
                                              # is sinh(y)*std + mean and
                                              # a linear one is wrong by a
                                              # factor that grows with x
z = z * ~mask                                 # masked points read zero

p = patch(concat(z, ~mask), length=16)   # 512 patches
#   ^ the observed-value channel is concatenated, so each patch carries
#     both the value and whether it was there.
h = residual_block(p)                         # see below

for i in 0 .. 19:            # pre-norm, width 1024
    h = h + attn(norm(h))
    h = h + mlp(norm(h))                  # tanh-approximate GELU
#   `norm_first: true` — pre-norm blocks: the norm feeds the sublayer
#   and the residual carries the unnormalised stream, which is the
#   opposite of the BERT lineage on these pages.

# `residual_block` activates with sigmoid, on both projections:
#     layer2(sigmoid(layer1(x))) + residual(x)
# Two activations in this model and neither is the erf GELU the rest of
# this directory uses.

# The quantile head is cumulative, so the 99 levels cannot cross.
rows = residual_block(h)                      # [., 100] per patch position
#      ^ 100 rows, 99 levels. Row 0 is a base and is not a quantile;
#        each of rows 1..99 adds to the running total. Treating the 100
#        rows as 100 levels shifts every level by one and gives the
#        median at index 50 instead of 49.
cum = rows[0]
for i in 1 .. 99:
    cum += softplus(rows[i]) / 99
    q[i-1] = sinh(cum) * std + mean           # denormalise each level
                                              #   here, before anything
                                              #   is combined

# The point forecast is the mean, and the mean is an integral — the area
# under the quantile function, trapezoid over 0.01..0.99 plus the tails.
forecast = integrate(q)                       # trapezoid + q[0]*step +
                                              #   q[98]*step
#          ^ integrate(sinh(...)), not sinh(integrate(...)). sinh is not
#            linear, so moving it outside the integral is a different
#            number — and a plausible one, which is why the order is
#            written out rather than left to the reader's habit.

# The patches are folded back onto the time axis, not concatenated.
# This revision publishes no `patch_stride`, so the stride is the patch
# length, the coverage is one, and the window below cancels exactly.
# The same code is right for both revisions, which is also why a port
# tested only here cannot tell it got the fold wrong.
w = hamming(16); w /= mean(w)                        # 0.54 - 0.46*cos(2*pi*i/(P-1)),
                                              #   normalised to mean 1
out[t] = sum(value * w) / sum(w)              # accumulate both, then
                                              #   divide — not a plain sum

The forecast window is an input, not an output shape. The model fills a masked tail of its own context, so a port that appends a horizon has built a different interface and will scale by the wrong statistics.

The scaler is not affine. asinh on the way in and sinh on the way out; a linear inverse is wrong by a factor that grows with the value. The std is clamped at 1e-5 rather than having an epsilon added, so a flat series gets scale 1.

The masked block is 128 positions and the forecast is 64. window = max(pred_len, d_patch * max(pretrain_mask_cont, 2)) is 128, the model fills all of it, and the caller keeps the first 64. Masking only the 64 it wants back changes the scaler's statistics (they are computed over the unmasked points) and changes what the model is asked to fill, and it forecasts either way.

The head emits 100 rows and 99 levels. Row 0 is a base, not a quantile; each of rows 1..99 adds softplus(...)/99 to a running total. The median is level 0.50, which is index 49 of the 99 and index 50 of the 100 rows — reading the rows as levels puts it one off, which is a smooth, plausible, slightly wrong forecast.

sinh is applied to every level before the integration, not after it. The shipped order is integrate(sinh(cum) * std + mean); sinh(integrate(cum)) * std + mean has the same shape and, because sinh is not linear, is a different number. The gap grows with the value, so a series near zero agrees and a large one does not.

Overlapping patches are folded with a normalised Hamming window, not concatenated. Where the stride is shorter than the patch, every output point is covered more than once; the reference combines the copies by accumulating value * w and w and dividing, with w a Hamming window scaled to mean 1. At stride equal to the patch length the coverage is one and the window cancels exactly, which is why one code path serves both revisions — and why a port that only ever tested the non-overlapping revision cannot tell it got this wrong.

The masked statistics use the N divisor where the unmasked path uses torch's default N-1. This model always has a mask, so N is the right one — and at 8,192 points the two disagree by 0.4%, which is above the parity floor.

Geometry

Other shapes of PatchTSTFMForPrediction

Identical across all of them: width 1,024, heads 16.

Only the columns that differ are shown; a cell with several values means the checkpoints of that shape disagree.

How this was checked

implemented, evidence grade class, per the assessment: the architecture class is implemented; nothing specific to this checkpoint was measured. The implementation meant here and below is an unpublished independent inference implementation by the author.

What stands behind the block above, beyond the published configuration it is read from:

• a checkpoint loader, which is where published tensor names are read
• an independent reference implementation of this architecture in Python, driving the published modelling code — an executable statement of what the model should compute, written against the publication rather than against any one implementation of it

Where an independent implementation and the published configuration disagree, the configuration is what this page reports and the disagreement is what it says.

Checkpoints with this architecture