GraniteSpeech5ForCTC

GraniteSpeech5ForCTC at 16 layers and hidden size 1024. 2 published checkpoints share this shape.

Implementing it

One forward, written out, per checkpoint whose arithmetic actually differs. A shape groups checkpoints by class, depth, width and layer stack — none of which decides an activation, a rope base or a routing rule — so where the members of this shape disagree there is a block each, and each names the checkpoint it was generated from.

No forward was recorded for this shape, so there is no pass above to compare it against — this block is what the published configuration says the arithmetic is, not a transcript of a run. Where the configuration does not say, the line says that instead of guessing.

The names are the ones the published checkpoint uses where a name is published, and canonical otherwise. A checkpoint loader may store them differently — fusing a gate/up pair into one matrix, or splitting one published projection in two — and those are that loader's names, not the model's. Everything spelled here is what the download contains.

Generated from granite-speech-5.0-470m-turboctc, which covers all 2 checkpoints of this shape — they agree on every field below.

# The input width is not in the configuration. It publishes
# `num_mel_bins: 80` and the encoder reads 320 — 80 x 2 x 2, and
# the two factors are not the same reason:
#     x2   each frame carries its own delta
#     x2   frames are then stacked in pairs
# Writing `4 * num_mel_bins` without knowing which two is how a mel-only
# 4-stack gets fed in at exactly the right width and transcribes noise.

feat = stack_pairs(with_deltas(mel_80(audio)))   # [T, 320]
x    = input_linear(feat)                 # -> 1024

for i in 0 .. 15:                          # conformer, five modules
    x = x + 0.5 * ff1(prenorm(x))
    x = x + attn_blocked(prenorm(x), window=128)
    #     ^ block-diagonal over non-overlapping windows, Shaw relative
    #       positions added to the scores. No rope in this tower.
    stride = 2 if i in [0, 1] else 1       # `subsample_layers`
    x = conv_module(x, stride)            # the stride is in here, and
                                          #   the residual is mean-pooled
                                          #   over each pair, not sliced
    #  ^ the frame count halves here, so the two modules below already
    #    run at the shorter length. Subsampling after post_norm instead
    #    runs ff2 and the norm over twice the rows and then throws half
    #    of that work away — a different function, same output shape.
    x = x + 0.5 * ff2(prenorm(x))         # already at the shorter length
    x = post_norm(x)

# T' = T / 4: stride 2 at each of 2 layers [0, 1]. One CTC row
# per surviving frame, so a port that subsamples nowhere emits 4x too
# many rows and still collapses to something readable.

#   The conformer tower, as published. Every line here is a tensor shape a
#   port allocates, and none of it is stated anywhere else on this page.
#     layers            16   `num_hidden_layers`
#     width           1024   `hidden_size`
#     heads              8   `num_attention_heads`
#     kv heads           8   `num_key_value_heads`
#     head dim         128   `head_dim`
#     ffn width       4096   `intermediate_size`
#     conv kernel        7   `conv_kernel_size`
#     activation      silu   `hidden_act`
#     vocab          16384   `vocab_size`
#     max positions    512   `max_position_embeddings`
#     attn bias       True   `attention_bias`

logits = out(x)                           # [T', 16384]
#        ^ `encoder.out` and the CTC head are the same tensor — the
#          checkpoint aliases them, so there is one matrix to load and
#          two names for it.

# CTC, not autoregression: one distribution per frame, then collapse.
ids = argmax(logits, axis=-1)
ids = drop_consecutive_duplicates(ids)    # in that order —
ids = drop(ids, blank_id)                 # collapse first, then drop
#     Dropping blanks first merges two genuine repeats separated by a
#     blank into one, which is what the blank is there to prevent.

The input is 320 wide and the configuration says 80. The factor of four is a delta expansion and a pair-stack, in that order, and neither appears as a key. A port that reads num_mel_bins and builds a mel-only stack of four gets the right width and the wrong content.

There is no decoder and no text config. This class transcribes by committing a token per frame, which is why its output_dim is a character-scale vocabulary and not a language model's.

The blank is dropped after the repeat collapse. The other order merges two real repeats that a blank was separating — the one job the blank symbol has.

Time is subsampled inside the convolution module, at layers [0, 1]. That is stride 2 in the depthwise conv with the residual mean-pooled over each pair, and the frame count is 4x shorter by the end of the second one. It is not a slice after the block: the second feed-forward and the post-norm of those layers already run at the shorter length, so x = x[::2] after post_norm does twice the work and then discards half of a different number. Both produce [T/4, vocab].

A buffer sized from the input length is wrong for every layer after the first subsample, and any per-frame alignment has to be rescaled by 4 before it means anything.

Geometry

How this was checked

implemented, evidence grade shape-pack, per the assessment, for each checkpoint of this shape: converted checkpoints of this shape run; this checkpoint's own weights are not among them. The implementation meant here and below is an unpublished independent inference implementation by the author.

What stands behind the block above, beyond the published configuration it is read from:

• a conformer tower and projector implemented from the published configuration
• a checkpoint loader, which is where published tensor names are read
• an independent reference implementation of this architecture in Python, driving the published modelling code — an executable statement of what the model should compute, written against the publication rather than against any one implementation of it

Where an independent implementation and the published configuration disagree, the configuration is what this page reports and the disagreement is what it says.

Checkpoints with this architecture