🔍 How to Build a Facial Recognition Search Engine 🔎

When you need to find bad guys fast, text search just doesn't cut it.

We worked with an organization that wanted to stop violence extremist attacks and save lives. One big challenge they faced was identifying extremists quickly. Extremists were gathering at an increasing number of events with larger numbers of attendees. To help the organization more quickly identify extremists we build a prototype of a facial recognition search engine.

VISION

We worked with an organization that wanted to save lives by stopping violent extremist attacks.

CHALLENGE

With violent extremism proliferating online and offline, the organization faced two unique challenges in identifying extremists:

1. Too Many Events

   1. There were hundreds of attendees at events (Unite the Right, January 6th, etc) that they were tracking. They would see individuals on social media and wanted a quick way to see if they had any information about the person in question.

2. Fragmented Data

   1. Each investigator had their data stored locally in their own folders created separate datasets that didn't communicate with each other.
      

SOLUTION

The solution we found was simple: create an internal search engine that leveraged facial recognition.

We broke this down into two parts:

1. Existing Data

   1. We set up a database on MySQL using AWS RDS where each extremist had a case file. The files related to that extremist were stored on AWS S3. Whenever a file was an image type, we setup a background process that used AWK Rekognition to: (a) check if the image had faces in it and (b) then count the number of faces. This way the data was primed for the second part below.

2. Face Search Tool

   1. Leveraging AWS Rekognition, we built a front-end tool that allowed users to upload a photo of a violent extremist, set the matching threshold, and search for the individual. On the backend, the tool would then search the database where the face check and face number columns were flagged (as opposed to searching the entire database).

This approach saved time and money because it broke down the facial recognition process into two parts.

WATCH THE FULL VIDEO

Here's a short video where we talk through the product vision board.

#facialrecognition #aws #solutionsarchitecture