Written by Sisong Wang, AI-Engineering Lead

The face recognition system is a piece of technology that can be utilised to match a human face from a digital image against a database of faces, typically deployed to authenticate users through ID verification services. The face recognition system is steadily making its way into people's daily lives. It is widely used to unlock smartphone screens, tag faces in social media posts, ensure public security, prevent fraud, and improve customer experience. Below are just a few examples of what the face recognition system can do.

How does the face recognition system work?

The face recognition system uses computer algorithms to pick out specific, distinctive details about a person's face. These details, such as distance between the eyes or shape of the chin, are then converted into a mathematical representation and compared to data from other faces collected from a face recognition database. The data about a particular face is often called a "face template" and is distinct from a photograph because it is only designed with certain details that can be used to distinguish one face from another.

Major steps of the face recognition system include face detection, feature extraction, and database-based feature matching.

• Step 1: Face detection

  Filtering procedures are used to distinguish facial features and filter them with accurate classifiers in the input image. It is notable that all translations, scaling, and rotational variations have to be dealt with in the face detection phase.

  
  

• Step 2: Feature extraction

  This step obtains the locations and outlines of landmarks and nodal points that make up each facial feature and then encode this information into numbers. The numerical code is called a "Face Feature" and it is unique to each person.

  
  

• Step 3: Feature matching

  The Face Feature is then compared against a database of other known face images using Euclidean distances. The pair-wise distances are then ranked and thresholded to filter out unmatched face images.

The challenges of the face recognition system: building a large-scale database

We have talked about algorithms so far. Now, let us put on our engineer hat and explore the challenges that need to be overcome.

One of the biggest difficulties is to tackle the scaling issue. It usually becomes more complicated when databases swell into millions of entries. To build a face recognition system that contains 100 million face images requires different solutions from a system with 100 face images.

A large-scale face recognition system imposes the following requirements on the design:

• Speed:

  The performance of real-time response speed (300ms to 3s), which is usually determined by database size and some recognition-irrelevant tasks. The search speed apparently depends on the user's CPU, yet memory is another factor as users tend to cache all faces in memory to avoid disk drive IO overhead. The best balance is calculated based on both the targeted speed and the estimated data set size.

• Sharding:

  Data is usually partitioned and spread over multiple nodes in order to support very large data sets and high throughput operations. The challenges for engineers here are how to reasonably distribute your data without impacting the overall system latency and how to do auto re-sharding when a node is added at runtime.

• Dynamic database:

  Depending on the use cases, the face database might be growing apace. For example, ADVANCE.AI's ADVANCE Face Search service inserts faces from every request into the database. The system should react if available RAM in any node is low by automatically adding a new node.

• Data persistence and recovery:

  Face data should not be merely cached in memory but needs persistent storage. In case a node is disconnected, the system should be able to recover it at a later stage.

• Availability:

  The system should continue operation when a subset of nodes is experiencing failures.

• Node management:

  Since data is spread across multiple nodes, we need a mechanism to identify and manage the nodes in the cluster to perform node addition, failover and data sharding.

• Rolling update:

  How to support the face recognition service's rolling update without impacting the connected client applications imposes another challenge on the engineering capability.

ADVANCE.AI's independently developed face recognition engine, which is one of the face-related infrastructural components, provides standard service to other face-based products.

ADVANCE.AI's face recognition engine offers a variety of customisation options in order to target different business contexts. For example, client applications can build and search multiple databases in one request. They could also associate custom attributes to data entries, like applying a timestamp to each entry. The client application can then limit the search to a certain time range and add filtering logic executed directly within the engine to remove irrelevant matches.

Facilitating eKYC adoption through the face recognition system

In the digital era, banks and financial institutions are required to adopt modern and reliable electronic Know Your Customer processes (eKYC) to satisfy customer's all-digital experience and meet new regulations while protecting users against online fraud. Meanwhile, the increased risk of data breaches coupled with customer demand for all-digital onboarding processes, rising mobile phone penetration, and security concerns among customers, is expected to propel the future global growth of technology-based identification.

Nowadays, there are several approaches and technologies to perform eKYC, but face recognition technology has been particularly popular. The increased usage of face recognition in the banking industry coincides with newly introduced regulation, also facilitating eKYC's promotion.