VisionID

The model was never the hard part

Pretrained face detection and embedding models are good and freely available. Cosine similarity is four lines. If this project were only the recognition, it would be an afternoon.

What takes the time is that a camera system runs somewhere, on hardware nobody chose for it, watched by people with different levels of authority, producing data about faces — and every one of those sentences turns into a design decision.

One build, four execution providers

The first thing that breaks a computer vision deployment is an install that assumed a GPU.

Inference goes through ONNX Runtime, and the provider is resolved at load time from a priority list in config.py:

PROVIDER_PRIORITY  →  CUDA → TensorRT / DirectML → CPU

Whatever is available wins; if nothing is, it falls back to CPU and still starts. That means a Jetson on a wall, a Windows desktop with DirectML, and a laptop with no accelerator at all run the same artefact. There is no GPU build and CPU build to keep in step, and no install-time flag that turns out to have been wrong when the machine is replaced.

The cost of this is honesty about performance: CPU works and is slow, and the throughput knobs below exist because of it.

Recognition, and the threshold that is a policy

Detected faces become embeddings, and an embedding is matched against the enrolled set by cosine similarity. A match is anything above RECOGNITION_THRESHOLD, which defaults to 0.35.

That number is not a tuning detail, it is a policy, and it is exposed as configuration for that reason. Raise it and the system stops recognising people wearing glasses they were not enrolled in; lower it and it starts confidently calling strangers by a colleague’s name. Which of those two errors is acceptable is a property of the site, not of the model, so the site sets it.

The matching was deliberately tuned for tolerance to slight side profiles rather than for frontal accuracy alone. People walk past cameras; they do not pose for them.

The knobs are the performance story

There is no single frames-per-second number worth quoting, because the whole design is a set of levers you pull until the hardware in front of you keeps up:

• PROCESS_EVERY_N_FRAMES — skip frames entirely
• DETECTION_SCALE — downscale before detection, the cheapest large win
• DETECTION_INPUT_SIZE — smaller detector input
• MAX_FACES_PER_FRAME — cap the work a crowded frame can cause, default 8
• TARGET_FPS, JPEG_QUALITY — bound the capture and streaming cost

MAX_FACES_PER_FRAME is the interesting one. Without it, cost per frame is unbounded in the number of people in shot, so the system degrades exactly when it matters — a busy doorway — and stays fast when nothing is happening. A cap converts that into a predictable ceiling and a known failure mode.

Who is allowed to do what

A recognition system that anyone on the network can enrol faces into is a recognition system you cannot trust the output of. So there are three roles:

• viewer — streams and status
• operator — viewer, plus switching source and enrol / promote / delete
• admin — operator, plus the audit trail

Auth is off by default and switches on by setting AUTH_SECRET, at which point tokens are signed and expiring (TOKEN_TTL_SECONDS, an hour by default) and users come from a local store with hashed passwords. On top of that, OPERATOR_PIN puts a second barrier in front of the destructive actions specifically, with its own session timeout — because “logged in as operator” and “about to delete an identity” deserve different amounts of friction.

The unknowns need a policy too

Every face that does not match is a row you could keep, and keeping all of them is how an empty corridor fills a database overnight.

Two settings bound it. UNKNOWN_INSERT_COOLDOWN_SECONDS stops the same unrecognised person being written once per frame — one visit is one record, not three hundred. UNKNOWN_RETENTION_HOURS expires them, at 72 hours by default.

This is the part of the system that is least about computer vision and most about not building a surveillance archive by accident. A retention window is a decision to forget, made once, in configuration, rather than never.

What is deliberately left out

No tracking across frames. Each frame is matched independently. Identity persistence across occlusion is a genuinely harder problem and the wrong one to solve here: for a door or a corridor, “was this person recognised at 14:02” is the question, and a track id would be precision the system has not earned.

No cloud anything. Embeddings, enrolment and the unknown store are all local. For this class of system that is not a limitation to apologise for — it is most of the point.