1,680 Faces, One Question: Can e-pondok's Face Recognition Be Trusted at the Scale of Thousands of Students?

A while ago one of our users raised a fair concern: during a normal check-in, the match score on screen hovered around 50–60%. The system wasn’t failing; faces were recognized. But the number felt fragile, and the bigger question behind it was simple: if a pesantren has not 50 students but 500 or 1,000, can the system still tell one face from another? Or will it start recording the wrong attendance?

We didn’t want to answer with “trust us”. So we tested it.

The test: 1,680 identities, thousands of photos

We took a public collection of photos labeled by person: 13,233 photos of 5,749 different people, with varied lighting, off-angle poses, and changing expressions. From it we built a gallery the size of a large pesantren: 1,680 synthetic “students” holding 4,255 enrolled face samples, then ran 1,085 check-in trials. Each face was recognized against the entire gallery at once, the way a real check-in happens.

The scenario mirrors reality down to the decision: a face walks up, the system compares it against every enrolled student, then it has to choose. Record automatically, or hand the decision to an operator?

What we found before the fix

The honest result was uncomfortable to read. In a crowd of 1,680 identities:

• The correct face ranked first in only 60.5% of trials. Nearly 4 out of 10 check-ins left the system unsure, or pointing at the wrong person.
• More seriously: 1 in 7 automatic records was the wrong student. At a school with hundreds of students, that means a case of digital attendance fraud in almost every queue.

The 50–60% score our user complained about was a symptom.

What we changed

Three things, with no new hardware and no added cost:

1. The system now reads the geometry of the face, then straightens it before comparing. Two photos of the same student with a slightly tilted head now read as the same face.
2. Each student is enrolled with 3 face samples in a single click. Matching runs against all of them, so one unflattering photo no longer breaks a check-in.
3. Matching decisions got stricter: when two students score nearly the same (think twins, or siblings), the system refuses to guess and defers to the operator. Better one button press by staff than one wrong record.

Results after the fix: same test, rerun from scratch

• The correct face ranked first in 97.5% of trials against 1,680 candidates.
• Wrong automatic records dropped to 0.31%, from roughly 1 in 7 to roughly 1 in 320.
• Speed went up rather than down: a settled check-in cycle now completes in under 1.2 seconds, and all processing still happens on the device, not queued on a server.
• Every measurement was repeated in a real browser, including a check that recognition results are identical across runs.

We also measured how much competition the system can beat: we compared the highest score a stranger’s face reached against the genuine face’s score. Before the fix the two overlapped. After, they separate cleanly. That separation is the most important result of this exercise; the accuracy figure is just its summary.

A note for schools that already enrolled students

The system now reads faces more carefully, so faces enrolled before this update should be re-enrolled once. Old data poses no wrong-record risk: the system will simply choose not to recognize, and ask for a fresh enrollment.

Closing

Trust in an attendance system is built from unglamorous things: repeated tests across 1,680 identities, published numbers, weaknesses acknowledged and then fixed. That is what we did this month. We will rerun this battle test every time the recognition engine is updated, and the numbers stay available if you ask.