TSPulseForReconstruction

TSPulseForReconstruction at 8 layers and hidden size 24. One published checkpoint has this shape.

Implementing it

One forward, written out, per checkpoint whose arithmetic actually differs. A shape groups checkpoints by class, depth, width and layer stack — none of which decides an activation, a rope base or a routing rule — so where the members of this shape disagree there is a block each, and each names the checkpoint it was generated from.

No forward was recorded for this shape, so there is no pass above to compare it against — this block is what the published configuration says the arithmetic is, not a transcript of a run. Where the configuration does not say, the line says that instead of guessing.

The names are the ones the published checkpoint uses where a name is published, and canonical otherwise. A checkpoint loader may store them differently — fusing a gate/up pair into one matrix, or splitting one published projection in two — and those are that loader's names, not the model's. Everything spelled here is what the download contains.

Generated from granite-timeseries-tspulse-r1.

# Two views of one series in a single grid. The spectrum is an input
# here, not a post-processing step.

# 1. RevIN — and its affine is not the identity (`revin_affine: True`).
std = sqrt(max(variance, 1e-3))           # the variance is clamped, not
                                          #   epsilon-added: the smallest
                                          #   scale this can produce is
                                          #   0.0316, and above the clamp
                                          #   the constant does nothing.
                                          #   `sqrt(var + 1e-3)` perturbs
                                          #   every window instead — 0.1000
                                          #   becomes 0.1049 at var = 1e-2.
scaled = (series - mean) / std
scaled = scaled * revin_weight + revin_bias   # learned; the inverse must
                                              # divide this out before
                                              # undoing mean/std

# 2. The FFT is real and its halves are kept separately.
bins = rfft(scaled)                       # 512 samples -> 257 bins
bins = bins[:-1]                          # drop the last bin, not the DC one
re, im_ = Re(bins), Im(bins)
re  /= (max(|re|)  + 1e-3)                # each half by its own max-abs,
im_ /= (max(|im_|) + 1e-3)                #   and the guard is added here
                                          #   where RevIN's above clamps.
                                          #   Same constant, two different
                                          #   operations, and on a flat or
                                          #   near-constant signal the max
                                          #   is ~0 and a bare divide is
                                          #   inf or a huge number that
                                          #   still patches and still
                                          #   reconstructs.
spec = concat(re, im_)                    # end to end, not a magnitude

# 3. Two patchers, trained separately. Time and spectrum do not share an
#    embedding — `time_patcher` and `fft_patcher` are distinct matrices,
#    and reusing one for both is a model that runs.
time = patch(scaled, 8) @ time_patcher.T   # 64 x 24
spec = patch(spec,   8) @ fft_patcher.T    # 64 x 24
grid = concat(time, spec, registers)      # 138 x 24; 10 registers
#      ^ appended, the opposite of TTM-R3 which prepends its two. The
#        heads read [-10:] and the reconstruction reads [:64].
grid = layernorm(grid, norm)              # over the whole token grid

h = mixer_layers(grid)                    # 8 layers
h = decoder_layers(h)

# 4. The forecast head reads the registers only.
y = mixer3(h[-10:]) ; y = layernorm(y)
forecast = y @ W_fc.T                     # -> 16 samples
recon    = h[:64] @ W_rec.T          # -> 512 samples

forecast = invert_revin(forecast)

Time and spectrum have separate trained patchers. They are the same shape and reusing one for both produces a model that forecasts, which is why this line is spelled out.

It does not consume magnitudes. The real and imaginary halves are normalised separately by their own maximum absolute value and laid end to end. A port that patches |rfft(x)| has thrown the phase away and halved the sequence.

The dropped bin is the last one, not DC (fft_remove_component: last). Dropping DC instead is the plausible choice and shifts every bin by one.

Two numerical guards, one constant, two different operations. RevIN clamps the variance — sqrt(max(var, 1e-3)) — and the spectrum normalisation adds to the maximum — re / (max|re| + 1e-3). Neither is an epsilon for the other's job, and both matter exactly where a series is flat or nearly constant: without them the scale is unbounded and the model still patches, still mixes and still returns a forecast.

The published forward is stochastic: it masks 70% of its input at random and does not gate that on training mode, so two eval-mode calls disagree. That masking is how the model scores anomalies, not how it forecasts; the unmasked arm is the same weights reading the whole context.

Geometry

How this was checked

implemented, evidence grade class, per the assessment: the architecture class is implemented; nothing specific to this checkpoint was measured. The implementation meant here and below is an unpublished independent inference implementation by the author.

What stands behind the block above, beyond the published configuration it is read from:

• a checkpoint loader, which is where published tensor names are read
• an independent reference implementation of this architecture in Python, driving the published modelling code — an executable statement of what the model should compute, written against the publication rather than against any one implementation of it

Where an independent implementation and the published configuration disagree, the configuration is what this page reports and the disagreement is what it says.

Checkpoints with this architecture