# Image autotagging: quality, latency and vocabulary coverage

Experiment date: 26 October 2026. Five configured pipelines, 1,024 jointly successful images, incomplete silver general/character source labels.

Question: what changes when you increase CAFormer's input resolution or use PixAI for the same tagging task?

## Observed results

| Pipeline | General mAP | General F1 | Warm median ms | Warm p95 ms | Dedicated process GiB |
|---|---:|---:|---:|---:|---:|
| CAFormer native 384 | 0.5358 | 0.5998 | 32.5 | 41.7 | 0.975 |
| CAFormer direct 384 | 0.5313 | 0.5954 | 29.5 | 38.5 | 0.975 |
| CAFormer direct 512 | 0.5383 | 0.5967 | 43.6 | 52.2 | 1.014 |
| CAFormer direct 640 | 0.5237 | 0.5801 | 60.2 | 70.2 | 1.089 |
| PixAI 1.0 Mexes | 0.5824 | 0.5997 | 223.6 | 247.9 | 6.270 |

Quality uses common general vocabulary and identical successful images. Each model uses its own fixed profile threshold. Timings use 1,019 warm rows. Dedicated memory is sampled process high water over loading and inference.

PixAI has the highest observed general mAP and broader native character coverage, at higher measured latency and process memory. General F1 is numerically close at the configured thresholds. Direct 512 gives a small general-mAP increase over the native CAFormer control; direct 640 does not improve the reported quality metrics.

## Follow-up: character coverage with Danbooru aliases

On 19 November 2026, the same five saved runs were compared on CPU with 13,791
active character aliases from the [Danbooru API](https://danbooru.donmai.us/tag_aliases.json?search%5Bstatus%5D=active&search%5Bcategory%5D=4&limit=5). Matching aliases
in output names and annotations raised the shared character vocabulary from
3,190 to 3,232 names. Both models could represent 703 of 1,687 source tag
occurrences (41.67%), up from 696 (41.26%) with exact names. The seven recovered
occurrences came from seven images. This measures vocabulary coverage before
considering scores. The [tagging guide](https://github.com/TimeOfAnarchy/vision-bench/blob/v0.4.0/docs/taggers.md#compare-with-aliases-or-implications)
describes the alias-aware comparison.

## Conditions and limits

CAFormer variants share unchanged weights and AMP FP16. Native 384 retains the original two-stage preprocessing control; direct variants use one bicubic resize. PixAI uses ONNX FP16 at 1008px. RTX 3080 Ti, Ryzen 9 9950X3D, Windows 11; PyTorch 2.11.0+cu128 versus ONNX Runtime CUDA 1.23.2. One pass, background GPU applications not stopped, no significance intervals or threshold tuning. Silver annotations do not establish verified precision or image-level accuracy.

The [reproduction kit](reproduce/v1/index.html) freezes exact sample IDs, image hashes/source URLs, annotations, five model profiles and sanitized full predictions. All 1,024 image files were independently downloaded and matched their frozen hashes. Replay the saved metrics without a GPU, or recover the exact inputs and rerun inference. Model access and upstream image availability still apply. [Immutable HF snapshot](https://huggingface.co/spaces/TimeOfAnarchy/vision-bench/tree/d69d975a3d0a07ceae2ceb4bb16b236bb6dc9a92/experiments/autotagging/reproduce/v1).

[Readable case study](index.html) · [Detailed report](data/index.html) · [Aggregate JSON](data/results.json) · [Figure](quality-latency.svg) · [Source](https://github.com/TimeOfAnarchy/vision-bench)
