Understanding Face Match Scoring

When we compare a live selfie to a driver's license photo, the result isn't a simple yes or no. Here's how our scoring system works, what the categories mean, and why comparing a selfie to an ID photo is harder than it sounds.

The Challenge

Comparing two selfies taken seconds apart with the same camera is relatively straightforward. Comparing a live selfie to a printed driver's license photo is a different problem entirely.

That license photo was taken at a DMV, possibly years ago, under fluorescent lighting, at a fixed distance, with a specific camera. Since then, it's been printed on a small card, carried in a wallet, and subjected to wear, fading, and scratching. The person may have aged, changed hairstyles, or grown facial hair.

Now we're asking our system to compare that single, degraded image to a live selfie captured on a phone, in whatever lighting happens to be available, at whatever angle the user holds the camera. The two images differ in resolution, color temperature, contrast, perspective, and age of the subject.

We built our scoring system with these real-world conditions in mind.

How Face Comparison Works

All face comparison happens directly in your end user's browser — no facial images are sent to our servers for matching. Here's what happens under the hood:

Step 1: Face Embedding

When the system detects a face — whether from the license photo or the live selfie — it converts it into a compact mathematical representation called a face embedding. This is a list of 128 numbers that describe the geometry and structure of the face: the distance between the eyes, the shape of the jawline, the proportions of the nose, and dozens of other spatial relationships.

These embeddings are not images. They can't be reversed back into a photo. They're abstract numerical fingerprints that capture what makes a face unique.

Step 2: Measuring Similarity

To determine whether two faces belong to the same person, we compare their embeddings using two complementary measurements:

Euclidean Distance

This measures the straight-line distance between two embeddings in 128-dimensional space. Think of it like measuring how far apart two points are on a map — except the map has 128 dimensions instead of two. A smaller distance means the faces are more similar.

Euclidean distance is the industry-standard metric for face recognition models trained with contrastive or triplet loss functions. Research benchmarks (such as those from the Labeled Faces in the Wild dataset) typically find that same-person pairs produce distances well under 0.6, while different-person pairs score above that range. We use this as our primary signal.

Cosine Similarity

Where Euclidean distance measures the magnitude of difference, cosine similarity measures the direction of the two embeddings — whether they point the same way in that 128-dimensional space, regardless of how "far apart" they are in absolute terms.

This makes cosine similarity more resilient to variations in lighting and exposure, which can shift the overall magnitude of an embedding without changing its directional character. Two photos of the same person — one in bright sunlight, one under dim indoor lighting — may have inflated Euclidean distance but still show strong cosine similarity.

We use cosine similarity as a secondary signal to strengthen our confidence when the primary metric is borderline.

Step 3: Multiple Samples

During the selfie capture, we don't rely on a single frame. The liveness challenge (blink detection, head movement) produces several face captures from slightly different angles and moments. Each of these is compared individually against the single license photo embedding, giving us multiple data points rather than one all-or-nothing comparison.

The license photo, however, gives us just one frame to work with. This asymmetry is an inherent limitation — and one reason why we weight multiple signals rather than relying on a single measurement.

Step 4: Composite Scoring

Rather than making a pass/fail decision on a single number, we combine our signals into a weighted composite score:

The Euclidean confidence (converted from distance to a percentage scale) is the largest contributor
The best cosine similarity across all selfie frames provides a secondary signal
The average of the top cosine similarities across multiple frames provides stability and reduces the impact of any single outlier frame

This composite score is then mapped to one of four match categories.

Match Score Categories

Instead of showing raw percentages — which can be misleading without context — we report face comparison results as one of four categories. Each represents our system's overall assessment after weighing all available signals.

Definite Match

All signals strongly agree. The Euclidean distance is well within the same-person range, cosine similarity is high, and multiple selfie frames are consistent. This is the level of agreement we'd expect from two recent, well-lit photos of the same person.

High confidence — the face on the ID and the live selfie are very likely the same person.

Likely Match

The primary metric passes and secondary signals support it, but the match isn't as emphatic as a definite match. This commonly occurs when the ID photo is older, the lighting conditions differ significantly, or the license has some physical wear. The signals still point clearly toward the same person.

Good confidence — the comparison supports that these are the same person, with some expected variation.

Possible Match

The signals are mixed. The primary metric may be borderline while the secondary metric remains supportive, or vice versa. This can happen with particularly degraded license photos, significant aging, or challenging lighting conditions during the selfie capture.

Moderate confidence — the results suggest similarity but cannot confirm a match with full certainty. Consider additional verification steps if this result is critical to your process.

No Match

Neither signal produces a strong indication that the faces belong to the same person. The Euclidean distance is outside the expected same-person range and cosine similarity does not compensate.

Low confidence — the comparison does not support that these are the same person. This could indicate a genuine mismatch, or it could result from extremely poor image quality.

What Affects Match Quality

Face comparison between a selfie and an ID photo is one of the more difficult problems in applied facial recognition. Several factors can influence the result:

License photo quality A worn, scratched, or faded license produces a weaker embedding. We apply image normalization (contrast and brightness correction) to compensate, but there are limits to what preprocessing can recover from a physically degraded source.

Age difference If the license photo was taken many years ago, natural aging — changes in weight, facial hair, skin texture — will increase the distance between embeddings. The underlying bone structure and proportions remain relatively stable, which is why matches are still possible, but confidence will be lower.

Lighting conditions DMV photos are taken under controlled, flat lighting. Selfies are taken in whatever environment the user happens to be in. Strong shadows, backlighting, or very dim conditions can shift the embedding away from where it would be under neutral lighting.

Camera angle and distance We use face alignment to normalize for head tilt and rotation, but perspective distortion (holding the phone very close vs. at arm's length) can still affect results. The license photo is taken at a fixed, standardized distance.

Glasses, masks, and accessories Accessories that partially obscure the face reduce the amount of information available to the embedding model. If the license photo shows the person without glasses but the selfie includes them (or vice versa), match confidence may decrease.

Industry Context

Face recognition accuracy is typically measured by two rates: the false match rate (incorrectly saying two different people are the same) and the false non-match rate (incorrectly saying the same person is different). These two rates are in tension — lowering one raises the other.

Industry benchmarks from controlled datasets like Labeled Faces in the Wild (LFW) report same-person Euclidean distances typically under 0.4–0.5 and different-person distances above 0.7–0.8. However, these benchmarks use curated, high-quality photographs taken under consistent conditions — a very different environment from comparing a phone selfie to a worn driver's license.

In the real-world scenario of ID-to-selfie comparison, the expected distance for a genuine match shifts higher due to the factors described above. Our scoring system is calibrated for this specific use case, not for the idealized conditions of academic benchmarks.

A note on confidence: No face comparison system is infallible. A "Definite Match" is not a guarantee of identity, and a "No Match" is not conclusive proof that two faces are different people. These categories represent our system's best assessment given the available data. We recommend using face match scoring as one factor in your verification decision — alongside liveness detection, license validity checks, and barcode cross-validation — rather than as the sole determinant.

Why Categories Instead of Percentages

Early in development, we displayed raw confidence percentages to clients. We moved away from this for a practical reason: a number like "52%" invites comparison to a coin flip, which is misleading. A 52% Euclidean confidence with a 93% cosine similarity is actually a reasonably strong indication of a match — but "52%" doesn't communicate that.

Categories let us present an honest, holistic assessment that accounts for all of our signals without asking you to interpret the math yourself. The underlying numerical data is still recorded in your verification logs for audit purposes.

Questions about a specific verification result?

Your First Verification →