Smile, Facial Recognition in Use

At a sporting event kiosk, you stand there watching rehearsal clips of a singer that will be performing at the half time show. What would you say if I told you that a facial-recognition camera inside the display was taking your photos and cross-referencing them with a database of the performer’s known stalkers? After reading about this happening to many attendees of the Rose Bowl, I started to look into how else facial recognition is being used. I found that this software has been growing in popularity with companies and government agencies throughout the world.

As a social engineer, I was concerned. I wanted to see how easily facial recognition technology could be circumvented and used maliciously through social engineering techniques, because it utilizes openly available information: your face. I started thinking, can we trust using facial recognition technology, such as on the new iPhones? Would the phones allow access to the device just by using a photograph of myself? Another question I wanted to answer was, could facial recognition technology be used by someone malicious to exploit others?

How did facial recognition software get its start?

Facial recognition isn’t new. It’s been developing since the 1800s. It all started with photographs being used to track down criminals and escaped prisoners. Then, through the work of pioneers studying facial expressions such as that of Silvan Tomkins, Dr. Ekman, and others, many uses of facial recognition technology started to emerge. When the 9/11 tragedy struck, biometrics, that is the measurement and analysis of unique physical or behavioral characteristics especially as a means of verifying personal identity, began to expand rapidly. This was especially true with facial recognition technology. Companies began coming out with various forms of this technology such as Microsoft’s Face API, Amazon’s Rekognition, and RealNetworks’ SAFR.

Smile, Cameras are Watching

So, where can you expect to see facial recognition technology in use?

When it comes to security, you can look at biometric security. It is being used with the FaceID on the latest iPhone and iPad Pro, and Microsoft’s Hello is being used on Windows 10 PCs. It’s also at offices and schools where they are using it to watch over the main entrance, allowing only those that are in the database to enter. One example of this is at the University Child Development School in Seattle, Washington.

We can also look at customer satisfaction. In China, people are using it to pay for coffee, visit tourist attractions, and withdraw cash from ATMs. When you are shopping, stores are using facial recognition to monitor how customers react to certain product displays.

Then there is law enforcement, where it is being used to identify suspects. By using facial recognition technology, the 2013 Boston Marathon bombers were identified. It’s also being used to prevent identity theft. To protect us from that, many states are using facial recognition technology to prevent the issuance of fraudulent drivers’ licenses. New York Officials have reported that using the technology has resulted in spotting some 21,000 cases of possible identity fraud. Customs and Border Protection used the technology to nab two imposters attempting to cross from Mexico into the U.S. using someone else’s border-crossing cards.

These are just some of the many ways facial recognition is being implemented, there are too many to cover in this newsletter alone. So, how can facial recognition technology be circumvented? In what way can it be used by someone malicious to exploit others?

The SE Angles

When looking at how facial recognition technology could be used to protect the main entrance at an office or school, one thing it still doesn’t prevent is tailgating. This allows someone without clearance to gain entrance to the building by following behind someone that does have clearance.

What about trusting the use of the technology to keep our new iPhone or iPad secure? One week after the iPhone X was released, a Vietnamese security firm demonstrated how they bypassed the FaceID with a combination of a composite mask of 3-D printed plastic, silicone, makeup, and simple paper cutouts. In another video, they demonstrated how they did it with a mask 3-D printed in stone powder and 2-D eyes printed with infrared-sensitive ink. That was in 2017, is it any better now in 2019?

In January, the Dutch non-profit Consumentenbond published its findings on a test of 110 smartphones and found that holding up a good portrait photo of the phone’s owner was all that was needed to unlock 42 of the phones tested. Microsoft’s Hello, the Windows 10 version of facial recognition, failed along with many others, according to an article by Graham Cluley. He mentioned that some researchers were able to bypass it just by using a “modified printed photo of an authorized user.” Microsoft has since patched the vulnerability found by these researchers, but can we really trust this technology to be the only source of securing our device? Until it is proven by the security community that facial recognition technology can secure our device without it being circumvented, it is best to stick with the old-fashioned password lock with a strong, unique password. We don’t want someone taking our phone, tablet, or pc to gain access to our private information, since all it took to bypass was a good photograph that can often be found easily with the use of open source intelligence research.

Another security concern with facial recognition technology is that someone with malicious intent can use a photograph and facial recognition software to quickly find personal information about us. In a report by The Telegraph, a Russian photographer took photos of complete strangers and used the facial recognition software Find Face to identify them. From that, he was able to find a trove of personal details about their lives. For social engineers, this can be a windfall for doing our job but for individuals this can be alarming. The criminal can take the information found and develop a targeted phish and send it to the victim. Unfortunately, there isn’t a way to prevent the use of facial recognition software being used to identify us, but we can limit the photos that are posted and secure our social media accounts, so our personal information is more secure.

While there are many conveniences and applications for facial recognition technology, there is still work to be done before it can be trusted as the sole mechanism in securing a device or as the only source of protection at the front entrance of an office or school. We should continue to use strong passwords on devices, some form of multifactor authentication, and security guards at the front entrance to prevent tailgaters where applicable. Since it’s getting easier to find information about us, we need to be aware of what is out there about us and know that the information could be used against us including facial recognition technology.

“Knowledge is power,” so with what we now know about the use of facial recognition technology, we need to use that knowledge to keep us secure.

Stay safe and secure.

Written By: Michael Hadnagy

Social-Engineer Newsletter Vol 09 – Issue 114

Need an Event Speaker?

Social Engineering Training

Join the Newsletter

Join the Newsletter