Architecture classes

The class name a model declares in its configuration is the most reliable label it carries: two checkpoints that share it run through the same code path, whatever their names suggest. It does not make them the same function — the configuration still decides the layer mix, the activation, the rope base, the multipliers and more, and findings §1 shows a class whose members do not even share a layer type. A base model, an instruction-tuned model and a safety classifier of one generation are routinely one class.

The class name links to its sheets — one per distinct shape, with the layer map, the recorded order of operations, the geometry and the weight stack. A class usually covers several shapes, and the sheet index shows exactly what separates them.

class	shapes	models	family	largest checkpoint
`GraniteForCausalLM`	5	36	Language — dense	`granite-4.2-30b`
`LlamaForCausalLM`	3	19	Language — dense	`granite-8b-code-instruct-4k`
`GraniteMoeHybridForCausalLM`	7	19	Language — hybrid (12) and Language — dense (7 that list every layer as attention)	`granite-4.0-h-small-base`
`GraniteMoeForCausalLM`	2	9	Language — sparse (mixture of experts)	`granite-3.0-3b-a800m-instruct`
`GPTBigCodeForCausalLM`	2	7	Code	`granite-34b-code-instruct-8k`
`MLPSpeculatorPreTrainedModel`	3	6	Speculative decoding heads	`granite-8b-code-instruct-accelerator` — largest of the 2 with a published parameter count; 4 of 6 publish none
`GraniteSpeechForConditionalGeneration`	2	5	Speech	`granite-speech-3.3-8b`
`ModernBertModel`	2	4	Text embedding and reranking	`granite-embedding-311m-multilingual-r2`
`LlavaNextForConditionalGeneration`	1	4	Vision–language	`granite-vision-3.3-2b-chart2csv-preview`
`GraniteSwitchForCausalLM`	3	3	Language — adapter switching	`granite-switch-4.1-30b-preview`
`Granite4VisionForConditionalGeneration`	2	2	Vision–language	`granite-4.0-3b-vision`
`XLMRobertaModel`	2	2	Text embedding and reranking	`granite-embedding-278m-multilingual`
`RobertaModel`	2	2	Text embedding and reranking	`granite-embedding-125m-english`
`RobertaForSequenceClassification`	2	2	Text embedding and reranking	`granite-guardian-hap-125m`
`GraniteSpeech5ForCTC`	1	2	Speech	`granite-speech-5.0-470m-turboctc-nc`
`PatchTSTFMForPrediction`	2	2	Time series	`granite-timeseries-patchtst-fm-r2`
`TinyTimeMixerForPrediction`	1	2	Time series	`granite-timeseries-ttm-r2`
`Idefics3ForConditionalGeneration`	1	1	Vision–language	`granite-docling-258M`
`RobertaForMaskedLM`	1	1	Text embedding and reranking	`granite-embedding-30m-sparse`
`ModernBertForSequenceClassification`	1	1	Text embedding and reranking	`granite-embedding-reranker-english-r2`
`GraniteSpeechNarForASR`	1	1	Speech	`granite-speech-4.1-2b-nar`
`GraniteSpeechPlusForConditionalGeneration`	1	1	Speech	`granite-speech-4.1-2b-plus`
`GraniteSWAForCausalLM`	1	1	Language — sliding-window attention	`granite-swash-2b`
`GraniteMoeSWAForCausalLM`	1	1	Language — sliding-window attention	`granite-swash-3b-a600m`
`FlowStateModel`	1	1	Time series	`granite-timeseries-flowstate-r1`
`PatchTSMixerForPrediction`	1	1	Time series	`granite-timeseries-patchtsmixer`
`PatchTSTForPrediction`	1	1	Time series	`granite-timeseries-patchtst`
`TSPulseForReconstruction`	1	1	Time series	`granite-timeseries-tspulse-r1`
`tinytimemixer`	1	1	Time series	`granite-timeseries-ttm-r3`
`GraniteVisionEmb`	1	1	Vision retrieval	`granite-vision-3.3-2b-embedding`

What actually differs between them

Dense. Every layer is identical: attention, then a feed-forward block — gated in the Granite and Llama-class models, an ungated GELU block in the older GPTBigCode code models. The whole parameter count is active for every token.

Sparse (mixture of experts). The feed-forward block becomes many smaller expert blocks and a router that picks a few per token. Storage scales with the expert count; arithmetic per token scales with how many are picked. The published parameter total therefore overstates the work done per token by a large factor.

Hybrid. Most layers replace attention with a state-space mixer, which carries a fixed-size recurrent state instead of a growing key/value cache. A minority of layers keep full attention. The practical consequence is that memory stops growing linearly with context in most of the stack.

Sliding-window attention. Ordinary attention, but most layers see only a fixed span of recent positions; a minority see everything. The windowed layers keep the same projections, and they need a key/value cache only as long as the window rather than as long as the context. The family is a per-layer span schedule rather than a new mixer.

Adapter switching. A base model that selects among low-rank adapters at a token position mid-sequence, rather than having them merged in advance. This is the one Granite language family whose behaviour cannot be expressed by folding adapter weights into the base ahead of time.

Code, two generations. The older code models are a multi-query-attention architecture with learned absolute positions, layer normalisation and an ungated feed-forward block. The later ones are a Llama-class configuration: rotary positions, RMS normalisation, a gated feed-forward block. They share a name and almost nothing else.

Vision–language. An image tower produces patch embeddings that are projected into the language model's embedding space and placed at reserved positions in the prompt. The families differ chiefly in how an image is tiled and how the tiles are laid out.

Speech. Three distinct shapes: a conformer encoder feeding a language model through a projector; an encoder that emits tokens directly by connectionist temporal classification, with no autoregressive decoder at all; and a non-autoregressive decoder that fills a fixed number of slots in one pass rather than left to right.