Models and pinned downloads

Every model and accelerator component semlith downloads, with the commit, digest and size it is pinned to. Read it to check what runs on your machine or to verify a download yourself.

semlith computes vectors with three ONNX models: granite for text, a cross-encoder for optional rescoring, and CLIP for images. They are fetched from Hugging Face on first use, cached under ~/.cache/semlith/models (or SEMLITH_MODEL_CACHE), and never fetched again.

Each model is pinned to a commit, and each file is verified against a SHA-256 recorded in the source. A model that changed upstream would change every answer without changing anything you can see. A file whose digest does not match is refused by name with both digests printed, and nothing is loaded. If that happens twice on a clean cache, the upstream bytes have changed and semlith needs a new release, not another download.

Text: granite-embedding-small-english-r2Copy Anchor Link

The default model: 384 dimensions, int8, Apache-2.0. semlith fetches these files itself and verifies each as it is read.

• Repository: onnx-community/granite-embedding-small-english-r2-ONNX
• Commit: 1dc7835ba0cb9c76a3618d0bf0c427c97671b3c8
• Recorded in src/embed.rs as GRANITE_REVISION and GRANITE_FILES.

The fp16 and fp32 exportsCopy Anchor Link

Two more exports of the same model, at the same commit and with the same tokenizer files. Recorded in src/embed.rs as Variant::files; the fp16 pair is also in src/gpu.rs as FP16_FILES, with the sizes the download is checked against.

• fp16 is what the GPU lanes run. The int8 graph does not load on WebGPU. The fp16 pair is downloaded with the WebGPU plugin, the first time a run starts on a machine with a hardware GPU and the GPU lane on, or when CUDA is turned on. Both files are stored in the plugin’s directory in the model cache, not in the Hugging Face snapshot. fp16 and int8 vectors of the same text agree at cosine 0.987 (2026-09-23 corpus). A store records how many chunks each variant embedded in its variants meta row.
• fp32 is what the known-answer fixture was made with. tests/fixtures/gpu/ holds 32 chunks and their CPU fp32 vectors; semlith doctor --gpu, and every GPU lane before its first real batch, compare against them. The product never downloads fp32. A harness fetches it with SEMLITH_EMBED_VARIANT=fp32, which is not part of the documented environment.

Accelerator componentsCopy Anchor Link

Not models, but pinned and verified the same way. Each download is streamed through SHA-256 and deleted if its digest does not match. Each set is kept under accel/ in the model cache, in a directory named by its pinned version. Turning a lane off leaves the directory in place; semlith accel remove <gpu | cuda> deletes it. Under --airgap each download is refused unless its directory was seeded beforehand.

The WebGPU pluginCopy Anchor Link

Microsoft’s WebGPU plugin execution provider, onnxruntime-ep-webgpu 0.4.0 (MIT), taken from the wheel Microsoft publishes on PyPI. semlith extracts the plugin library, plus the two shader compilers beside it on Windows, into accel/webgpu-0.4.0/. Recorded in src/gpu.rs as WEBGPU_VERSION and wheel().

It is fetched only after a hardware adapter is found. A machine whose only adapter is a software renderer (Mesa lavapipe or llvmpipe, SwiftShader, Microsoft WARP or the Basic Render Driver) counts as having no GPU, and nothing is downloaded.

The CUDA packCopy Anchor Link

x86_64 Linux only. Pack 1.24.4-cu12.8, recorded in src/cuda.rs as PACK_VERSION and PARTS, with PACK_BYTES asserted against the sum of the sizes in tests/cuda.rs. It is downloaded only after CUDA is turned on (semlith accel on cuda or the switch on the Machine limits card), and the size is stated before the download starts: 1 891 522 804 bytes (1.89 GB) when the system supplies none of the libraries. A library the system already has, at the pinned version or a newer one in the same major version, is used from the system and its wheel is not fetched. Parts are listed in load order.

The NVIDIA versions are the CUDA 12.8 set that PyTorch 2.8’s cu128 wheels pin; ONNX Runtime’s onnxruntime-gpu[cuda,cudnn] extras resolve to the same set. The card needs driver 525.60.13 or newer, the minimum for CUDA 12.x on Linux x86_64; a card on an older driver is reported with that minimum and not used. On Windows, an NVIDIA card is used through WebGPU on D3D12.

Licences. ONNX Runtime is Microsoft’s MIT-licensed release, downloaded from its GitHub release. NVIDIA’s libraries never pass through semlith: your machine downloads each wheel from files.pythonhosted.org, where NVIDIA publishes it — the same bytes from the same place pip install nvidia-cudnn-cu12 would fetch — and semlith never hosts, mirrors or ships a copy. Your use of the libraries is governed by the licence each wheel carries (“NVIDIA Proprietary Software”), the terms a pip install of the same wheel accepts. When a wheel includes that licence text, semlith keeps it beside the libraries in licences/. Because semlith does not redistribute the libraries, the redistribution terms in the CUDA Toolkit EULA and the cuDNN Software License Agreement do not apply to it.

The Core ML packCopy Anchor Link

Apple silicon only, for the Neural Engine lane and the Core ML GPU lane. semlith builds it: .github/workflows/packs.yml runs packs/coreml/convert.py on a macOS runner and publishes the zip to this repository’s pack-coreml-v2 release, and src/packs.rs pins it by URL, SHA-256 and size. granite at ibm-granite/granite-embedding-small-english-r2@2ab6fa8e is converted twice:

• for the Neural Engine: batch 4; sequence buckets 128, 192, 256, 320, 384, 512;
• for the GPU, in the standard layout: batch 8; buckets 128, 192, 256, 320, 400 (no chunk is longer than 400 tokens). These run with fp16 accumulation, which leaves the fixture’s cosine against fp32 at 0.999999.

Each layout is one multifunction model with a function per bucket, so the weights are stored once per layout.

On the M1 Air, every Neural Engine function places 99.85 % of its operations on the Neural Engine, and its vectors agree with the fp32 reference at cosine 0.99998 or better on the 512-chunk fixture in tests/fixtures/coreml.

The first load on a Mac compiles the models for that machine. The lane is ready once the longest bucket has compiled (39 s on the M1) and compiles the other five behind it, about three minutes in all. macOS keeps the result for the program that compiled it, which is a copy of semlith kept beside the models, so a later start is ready in about 2 s and an upgrade does not compile again. macOS empties that cache when the disk runs low: with 13 GB free on the M1 it had done so within ten minutes, and the next start compiled again.

The llama.cpp packCopy Anchor Link

Experimental. ggml-org’s own llama-server build b11146 for the platform, and granite as GGUF f16, which semlith builds with llama.cpp’s converter at the same build (packs/llama/convert.sh) and publishes to pack-llama-v1.

TensorRT for RTX and OpenVINOCopy Anchor Link

Experimental, x86_64 Windows and Linux. Each is the vendor’s own ONNX Runtime plugin execution provider, downloaded from PyPI where the vendor publishes it; semlith never hosts a copy.

Intel’s plugin offers Intel hardware only: on a machine with an AMD CPU and no Intel GPU or NPU it offers no device, and the lane says so.

Rescoring: jina-reranker-v1-turbo-enCopy Anchor Link

A cross-encoder: 37 M parameters, int8, Apache-2.0. It reads the query and a candidate together and reorders the head of the fused list. It embeds nothing and adds no candidates, so a store’s vectors do not depend on it.

It is off unless you set SEMLITH_RERANK=on. A search over one store takes 8.2 ms, and 132.2 ms with rescoring over twelve candidates — sixteen times slower — for a small gain at the top of the list. Turn it on when an answer matters more than a tenth of a second.

semlith setup fetches it beside the embedding model, so turning it on never pauses a query for a download. semlith stats and semlith doctor both say which ranking a search used.

• Repository: jinaai/jina-reranker-v1-turbo-en
• Commit: b8c14f4e723d9e0aab4732a7b7b93741eeeb77c2
• Recorded in src/rerank.rs as RERANK_REVISION and RERANK_FILES.

Images: CLIP ViT-B/32Copy Anchor Link

Two repositories, a vision encoder and a text encoder, fixed as a pair. A query about a picture goes through CLIP’s own text encoder, never through the store’s model, because a granite vector and a CLIP vector have different lengths and describe different things.

These are fastembed’s built-in models, so fastembed resolves and caches them through its own client and semlith cannot hand it a revision. Instead, semlith checks every snapshot in the cache, not only the pinned one, before either encoder is used and again after a fetch, and refuses to load bytes that are not the ones this release was built against.

• Vision: Qdrant/clip-ViT-B-32-vision at e0c24ed0fa57fa3e4f97f30de74c51d944036ace
• Text: Qdrant/clip-ViT-B-32-text at 48ca1db27cb4063eb311ec2aa7f087a808112876
• Recorded in src/image.rs as VISION_REVISION, VISION_FILES, TEXT_REVISION and TEXT_FILES.

The cache directoryCopy Anchor Link

semlith refuses to load weights from a model cache owned by another account or writable by other users, and names the chown or chmod that fixes it. A directory somebody else can write to is a model somebody else chooses.

Checking a digest yourselfCopy Anchor Link

Hugging Face publishes the SHA-256 of every LFS file in its tree API, and the small files can be hashed directly:

curl -sSL "https://huggingface.co/onnx-community/granite-embedding-small-english-r2-ONNX/resolve/1dc7835ba0cb9c76a3618d0bf0c427c97671b3c8/tokenizer.json" \
  | shasum -a 256

curl -sS "https://huggingface.co/api/models/Qdrant/clip-ViT-B-32-vision/tree/e0c24ed0fa57fa3e4f97f30de74c51d944036ace?recursive=true" \
  | jq -r '.[] | select(.lfs) | "\(.path) \(.lfs.oid)"'

Updating a pinCopy Anchor Link

Changing a model is a release, not a patch to a table. The commit and every digest move together, docs/models.md and the constants are updated in the same change, and CHANGELOG.md says which model moved and why. A store’s vectors are comparable only with vectors from the model that built it.