With modern privacy laws getting tighter and tighter all over the world, especially in Europe, many facial recognition system cannot be used in an indiscriminate way, just looking for a face that is recognized. Currently biometric information from the general public cannot be transmitted over networks if they are not the object of an active investigation. This is why ‘facial recognition by similarity’ is becoming ever more popular. In this article you can find out what it is and how it works, compared to “traditional” face recognition.
The difference between both facial recognition systems
Facial recognition and facial recognition by similarity are related concepts, but focus on different aspects of how faces are identified and compared. The “standard” facial recognition is a broad term that refers to the technology and methods used to identify or verify a person’s identity based on their facial features. It typically involves capturing an image of a face, analyzing it using AI algorithms, and comparing it against a database of known faces. The process includes getting the facial features as precisely as posible, and creating a kind of ID fingerprint of each specific person. These IDs are compared against a database of known faces (that has been previously trained on those faces and the names assigned by hand).
Facial recognition by similarity is a more specific approach within facial recognition that emphasizes comparing the features of a given face to find similar faces, rather than strictly matching to a known identity. It focuses on using algorithms to measure how closely two faces resemble each other, and returns a “similarity score”. This means you do not get a concrete name, but how similar the current face is with one you are comparing it to. As a result, you can get clusters of similar faces or of lookalikes. The advantage of this system is that no personal information is processed, except for the original face you are comparing all others to.
General face detection technology in video
- Usually, when processing a video looking for faces, the video stream is Split into individual frames. Each of these frames is then analyzed, looking for faces, as if it were a still photograph. In general, face detection uses Deep Learning models to find facial features in an image.
- If a face is detected (an oval on top of a human body, usually) the system extracts its key features. These are normally the position and size of the eyes, the tip of the nose, jaw outline, lip position, ears, etc.
- All of this information is converted into what is called a “vector” in the Deep learning lingo. This is a numerical representation of that particular face.
- All of these vectors are matched with an existing database of face vectors. If a match is found, the person in both images must be the same. Only exact or almost exact matches are allowed.
Faces are converted into vectors (numerical values) to compare them with others.
Facial recognition by similarity
As we have seen, “traditional” face recognition tries to get exact matches, so if the person in the videos we are analyzing matches one of the known faces, it must be the same, and an identification can be made.
When working with similarity, however, you may start with a face of which you don’t know the name, but only that the person has commited a crime or is being searched for for any reason. This system does the same analytics as the usual facial recognition, but also shows similar faces. This is, faces that look like the reference one, but are not identical. Due to this fuzzy comparison, usually you end up with a cluster or group of similar faces. Even without knowing the name, you can see if any of the found people matches the original. With that information you can continue your investigation, until you get a name.
In this way, no personally identifiable information is transmitted and the GDPR or similar legal frames are ok with biometric data transmission. You only know that A is similar to B, but you don’t know who A or B actually are.
Uses
Apart from the obvious, in Law Enforcement, where you have a screnshot from a security camera where a crime was being commited and you want to find that (as of now) unknown face in the city surveillance cameras, there are a number of other puposes you can similarity face recognition to:
- Social Media apps can find similar faces in image posts from other people and suggest them or group them.
- Look-alike finding, for example for locating actor look-alikes for cinema.
- Ancestry or genealogy, locating people with similar facial features as the original one.
In summary
While both facial recognition and facial recognition by similarity leverage similar underlying technologies, they serve different purposes and come with distinct advantages and challenges. Understanding these nuances is essential for deciding when to use each one. In fact exact face matching con mostly only be done by law enforcement with a judicial warrant.