Safe By Design AI

Evidence

The four data files this reference cites, travelling with it.

Provenance

This is the one place the reference's provenance is set out in full; every other page links here.

Every one of the 172 rows is pinned to an upstream commit. In claims.json each row carries upstream_revision (the commit these pages describe), that commit's upstream_last_modified date, and the date the pin was checked (revision_checked). The revision column in models.md links each model's repository as it stood at that commit.

139 of the 172 publish a configuration, and each of those rows carries the sha256 of the exact config.json these pages read (config_sha256, with its size in config_bytes). The other 33 publish none. Their stored file is an empty object, and a digest of that would be the same value on all 33 — identifying nothing, while sitting in a field that looks like it identifies something — so those rows carry config_published: false and no digest.

The pins were checked, not inferred. On 2026-09-22 each of the 172 repositories was queried on the Hugging Face Hub, and for the 139 that publish a configuration the config.json at the pinned commit was downloaded and hashed. All 139 matched config_sha256 byte for byte (config_matches_revision: true), and in every case the pinned commit was the repository's head on that date. The 33 without a configuration are pinned to their head commit on the same date, with nothing to compare.

The tensor inventory was read from the published headers, not from the weights. On 2026-09-23 every weight file at each model's pinned upstream_revision was listed on the Hugging Face Hub, and each safetensors file's header — the JSON block at the start of the file that names every tensor with its dtype and shape — was fetched with HTTP range requests. No weights were downloaded. 155 of the 172 models publish safetensors, and their rows in tensors.json are complete. The other 17 publish only .bin, .pt or .ckpt files, whose tensor lists cannot be read without downloading the whole file, or publish no weights at all; their rows list the files and no tensors. Where a repository holds more than one shard set, only the set its model.safetensors.index.json references is inventoried; the other is listed under unreferenced_files (one model, granite-speech-3.3-2b). The inventory records names, dtypes and shapes only. It holds no tensor values and no weight digests, so it can show that two checkpoints have the same tensors but not that those tensors hold the same numbers.

What these identify, and what they do not. A pin says which commit these pages describe; it does not hash the weights at that commit. A configuration digest identifies a configuration. It is not a checkpoint identity and not a repository revision: two models can publish byte-identical configurations and different weights, and findings §6 is about exactly that. Nor does a digest validate the analysis: one that still matches says the file is the one that was read, not that it was read correctly.

To check a model yourself, download its config.json from Hugging Face, compute its sha256 and compare it with that model's config_sha256. A match means these pages describe the file you hold; a mismatch means the upstream file has changed since this snapshot, and these pages describe the pinned commit rather than the current one.

What claims.json is, and is not

It gives traceability, not independent verification. A row says what grade a model's support claim carries and what that grade rests on, and its digest lets you establish that the config.json you have is the one analysed here.

It carries no comparison result, no metric, no tolerance, no run date and no recording digest — because none of those are recorded in a form this export can read, and a field filled by inference is worse than a field left out. Those are exactly what would make it a verification record, and they are named here so the gap is visible rather than implied.

The implementation these pages were checked against is not here, and neither is its inventory. Where a page says a forward pass was recorded or a comparison was run, that is a statement about work done in an implementation you cannot inspect. The reference is to IBM's models; that implementation produced the execution evidence summarised here.