Safe By Design AI

Method

How this reference is built, in enough detail to follow the method or to disagree with it.

1. Enumerate, then exclude

The vendor's public model catalogue is listed in full, and anything whose name marks it as a converted copy of a model already present is dropped — gguf, fp8, mxfp4, nvfp4, int4, int8, awq, gptq, onnx, openvino, mlx. They are the same model in a different container, and counting them inflates every total. That leaves 172.

2. Read each model's own configuration

A published config.json is fetched directly for 139 of the 172. The other 33 publish none, and that absence is recorded rather than filled: no geometry is inferred from a placeholder, and their rows say config_published: false instead of carrying a digest of an empty object. Nothing is inferred from a model card, a README or a name where a configuration has an answer — a card is prose someone wrote, and the configuration is what the code loads.

Geometry is not in a fixed place or a fixed spelling. Depth is num_hidden_layers, n_layer, num_layers or encoder_num_layers; width is hidden_size, n_embd, d_model or embedding_feature_dim; the context bound is max_position_embeddings or n_positions. It may sit at the top level, under text_config for a model with a language half, or under encoder_config for one without. All of them are read. Reading only the common spelling in the common place silently loses whole lineages — see findings.md.

3. Key on the declared class, not on names

Two checkpoints that declare the same architecture class go through the same code path, whatever their names suggest. The configuration still decides the layer mix, the activation, the rope base and so on — findings §1 shows how far one class can stretch — but names add nothing the class and configuration do not already say. 172 models reduce to a much smaller set of classes on that basis, and findings §6 shows how far it goes: a base and its instruction-tuned model, or two consecutive releases, can be byte-identical in configuration. Safety classifiers and adapter libraries that repeat their base's geometry are candidates for the same merge, not checked groups.

So nothing here picks "the" model for an architecture. Where a set of checkpoints is indistinguishable, the set is what gets shown.

4. A shape is class + depth + width + stack

The class alone is not a shape — most classes cover several. Neither is class-plus-size: granite-4.0-1b is 40 layers of attention and granite-4.0-h-micro is 36 state-space layers plus 4 attention, and they share a class, a depth and a width. Keyed without the stack they would merge, and the merged sheet would show one of the two layer maps and attribute it to both.

The stack signature — what the layer_types table actually contains — is therefore part of a shape's identity. That gives 55 shapes.

5. Group by what the model is, in a fixed precedence

Three rules, applied in this order, because the order is what makes them correct:

  1. an adapter is an adapter, even though an adapter library ships its base model's configuration and would otherwise be filed as a dense language model;
  2. the layer table beats the class name — a model declaring the hybrid class with no state-space layer is grouped by what its layers say, not by the label;
  3. the class, then the name — and names are consulted only where the class genuinely does not distinguish the models, as it does not between a general language model and a code model that share a Llama-class configuration.

6. The pass structure is recorded, not derived

The order of operations on a sheet is captured by running one forward pass in the author's unpublished implementation (see support.md) and recording the sequence of operations it performs — not by reading the configuration a second time and writing down what it implies. A structure re-derived from the configuration would be the same claim expressed twice, free to disagree with the thing it describes, and it would be right about models nobody had ever run.

The operation names in a recorded pass are that implementation's own labels, not names from transformers or from the checkpoint: delta_mix, for example, is its name for the token-mixing step of a state-space layer. Positions in the pass are numbered by their place in the structure, not by layer index: one position fires once per layer. Run-length form marks a repeated subtree with xN and deliberately does not record N, so two models differing only in depth have the same pass — depth is geometry, not structure.

31 of the 55 shapes carry a recorded pass. The rest are described from configuration and weight structure alone, and say so on their face. A pass nobody ran is not a pass anybody can write down.

A recording is matched to a shape by the same key the sheets are grouped by, stack included, and both come from one function. Otherwise granite-4.0-h-micro — 36 state-space layers and 4 attention — would be shown granite-4.0-1b's all-attention recording, because the two share a class, a depth and a width.

7. The implementation sketch is derived, and it is not the recorded pass

Each sheet also carries a forward pass written out as code. It exists because the sections around it answer a different question: the layer map and the geometry say what shape a model is, and the recorded pass says what ran and in what order, but neither says what to multiply by what.

Every number in it is the checkpoint's own, lifted out of its config.json into the catalogue and substituted at generation time, so the block is regenerated rather than maintained. Where a configuration does not carry a value the line says so instead of printing a default — a generated pass that quietly filled in an epsilon nobody published would be a made-up fact in a code block, which is the most believable place to put one.

It is deliberately not the recorded pass, and the two are shown separately. One is captured from a real forward pass and is evidence; the other is what the published configuration implies and is a reading. Where they disagree the sheet says so and names both sides — a model that declares no positional encoding and whose recorded pass enters a rotary step is not a contradiction, but a reader is owed the sentence that reconciles them.

8. Every page is checked against what generates it

Every page here is generated. Knowing the pages exist proves nothing about whether they agree with their sources, so every build regenerates all of them into a temporary directory and diffs the result against the published pages. An edit to the catalogue, the assessment, a published configuration or a generator therefore cannot leave a page stale. The check never rewrites a page — one that fixes what it finds is one nobody reads the output of — and, like every other consistency check here, it makes the build fail rather than warn.

It checks the extraction as well as the rendering. Regenerating pages from the catalogue proves the renderers are fresh but says nothing about the impl blocks they read, so the check first re-extracts the catalogue from the stored configurations and compares that. Without this step an extractor bug stays invisible while every page faithfully renders stale data — as with rope_parameters, transformers' newer spelling of rope_scaling, which two models use for their rope base and nowhere else.

What a reader can reproduce. The check runs offline from stored copies of all 172 configuration files (the 139 published ones and 33 empty placeholders), because findings §6 compares whole configurations, not just the geometry lifted out of them. Those copies, the generators and the implementation are not public, so the checks in sections 8 to 11 are reproducible by the author only. What you can check yourself is the input: download a model's config.json, compute its sha256 and compare it with config_sha256 for that model in evidence/claims.json. A match means these pages describe the file you hold.

9. Check the reference against itself

Generated pages are easy to believe, so the numbers are not written down — they are recomputed from the data every time these pages are produced, and every internal link and heading anchor is resolved before they are published. An anchor is computed with the same rule the renderers use rather than guessed, because a dash in a heading is punctuation and vanishes from the slug: a contents list written by hand silently does not work.

10. Check that every published fact reaches a page

Section 9 checks the reference against itself and section 8 against its generators. Neither can see a fact that is extracted correctly, rendered by nothing, and therefore missing from both sides of every diff — and tests that assert "this page says X" have the same blind spot from the other side: a fact nobody thought to assert is a fact nobody notices is missing.

So a third check reads no prose. For every published fact the catalogue holds about a checkpoint it perturbs that one field, re-renders the page that checkpoint lands on, and asks whether a single byte moved. If nothing moved, the reference is carrying a fact it does not show. There are 4,025 probes; the input is the stored data and the oracle is byte equality, so the check cannot forget in sympathy with a renderer.

365 of those facts are waived and 0 are unaccounted for. A waiver does not mean "unimportant"; it means "this page is correct without it", written down by name with the reason, in four tables that are four different arguments: inert at inference (a dropout rate, an initializer range), carried by the catalogue rather than the page (a comparison column the sheet's own table prints), said in another spelling on the same page (rope_theta where the page prints the resolved base), or specific to one class. Zero means zero unaccounted-for facts, not that every extracted field is printed on every page.

It proves influence, not correctness. A renderer that printed its lines backwards would change no field's effect on any page and pass this check completely. It cannot see an equation, and it cannot see the order of operations; separate tests cover order, and neither replaces the other.

11. What an evidence grade is computed from

support.md grades each row by the strongest evidence that exists for it in the unpublished implementation. Nothing in that column is maintained by hand: a written-down evidence field would be one more place to disagree with the packs, the recordings and the comparisons it claims to summarize.

The distinction the grades are built around: a reference implementation existing is not the same as a comparison having been run. One says somebody could check an implementation against the publication; a recording kept beside it says somebody did, and that the result was retained. They are different grades, and a grade whose name ends in "parity" must name a recording made against that checkpoint or shape. A recorded forward pass is the implementation running its own copy, which is weaker and different.

An earlier rule inferred comparisons from recording names mentioned in reference scripts; those names turned out to be arguments in usage examples, and the rule credited thirteen Llama-class rows with comparisons nobody had run. It was removed, and those rows carry the weaker grade their evidence supports.

A row that does not claim to run gets no implementation grade at all. A deliberate refusal is graded by what refuses it, and that depends on the feature: aLoRA adapters are refused by the step that loads adapters, the switch models by the step that reads the configuration, for different reasons.

12. Published without the implementation

These pages are published without the implementation they describe. Internally, a sheet can cite the source file that decides a behaviour, the converted checkpoint a forward pass was recorded from, and the tensor names a loader writes — useful to the author, and meaningless or misleading to anyone else.

So the public text is generated in a public form rather than scrubbed afterwards. A pass over finished prose would have to rewrite several hundred paths and names into sentences that still read, and would silently miss the next one; a flag that whole sections are written against cannot. The failure mode becomes a missing paragraph, which is visible, instead of a leaked path, which is not.

The public form leaves out every source and storage path, the implementation's name, its internal build names, and the tensor names a loader invents — a fused gate/up pair, a split projection — because a reader matching those against the published download would look for tensors it does not contain. It keeps every claim those things supported, stated as what was done.

What is published is the catalogue, the assessment with the names of converted checkpoint files removed, and a claims manifest, claims.json — evidence grades and configuration fingerprints, not a verification record. It has one row per model: the grade, what the grade means, and, where the model publishes a configuration, the sha256 of the exact file these pages read. It carries no result, metric, tolerance, run date or recording digest, because none of those are recorded in a form this export can read, and a field filled by inference is worse than a field left out.

The digest makes a claim traceable, not reproducible. It lets a reader establish that the config.json they have is the one analysed here — and it identifies a configuration, which is neither a checkpoint identity nor a repository revision (every row also carries its pinned revision; see below). 139 of the 172 rows carry a digest. The other 33 publish no configuration; their stored file is an empty object, whose digest would be the same on all 33, so they carry config_published: false and no digest instead.

The finished bundle is then checked twice: every relative link and heading anchor is resolved with the bundle as the filesystem root, which is the only place that check means anything, and the whole bundle is scanned for anything that still describes the private layout. Both must come back empty.

13. Keep catalogue facts separate from implementation evidence

The public catalogue says what IBM publishes; it cannot say what any implementation of it does. support.md is generated separately from the support assessment, and the build requires that assessment to cover exactly the same model ids as the public catalogue.

For ordinary model rows, a matched pack is a configuration match, not checkpoint identity. Base, instruct and guardian releases can share a normalized configuration even though their weights and roles differ. For adapter rows, the pack list may instead name compatible bases, so classification follows the execution status first. The page reports both without turning either into a claim that the exact published checkpoint was converted or tested.

What this method cannot tell you

A configuration is a declaration, not a measurement. Where a field is absent the tables show a dash rather than a default; where a field is present but vestigial it is reported as published. findings.md collects the places where that distinction matters.

Parameter counts are totals. For a sparse model this includes every expert, most of them inactive for any given token. Comparing a sparse total against a dense total compares storage, not work.

Nothing here is a benchmark. No output quality was measured, and no claim is made about how any model performs. This is a description of what was built and how it is put together.

Provenance. These pages are generated from two kinds of public source — the catalogue, and the 139 published config.json files that exist. The other 33 repositories supply none, and nothing is inferred to stand in for them. Where a page shows a recorded pass, it comes additionally from a run of the author's unpublished implementation, which is not a public source and is described as such wherever it is used.

2026-09-18 is the latest upstream lastModified timestamp in this snapshot, and that is all it is. The hub distinguishes createdAt from lastModified, and this is the second: the repository carrying it was created earlier and touched on that date. It is not a publication date, not the date this reference was built, and not the date anything was retrieved. It dates the snapshot because it is the newest fact in it, and it does not move when these pages are regenerated — a date that changes when nothing else did makes every rebuild look like new information.

Each model is pinned to an upstream commit, and createdAt is not recorded. Two identifiers travel together in evidence/claims.json. The revision (upstream_revision) says which commit these pages describe; the digest (config_sha256, for the 139 configurations that exist) says what the file at that commit contained. On 2026-09-22 every repository was queried on the Hugging Face Hub and each published config.json was re-downloaded at its pinned commit and hashed: all 139 matched, and every pin was the repository's head on that date. The 33 repositories without a configuration are pinned to their head commit with nothing to compare. A pin does not hash the weights; it fixes the commit, so a reader can fetch exactly what was described.