Can disguises fool surveillance technology?

Antivirus pioneer John McAfee, who recently fled from Belize after his neighbour was shot dead, supposedly used disguises to outwit his pursuers. Could technology have spotted what humans failed to see?

Stick on a fake moustache. Add some glasses.  Dye your hair. And perhaps pop on a hat. If you are a man – or woman - on the run in the movies then this kind of low-tech disguise is all that is needed to evade the authorities.

But, in a case of life imitating art, a similar array of tactics seems to have met with some success in the real world.

One of the more bizarre news stories of recent weeks concerns John McAfee, founder of the eponymous anti-virus software company, going on the run from the Belize police. According to his blog, McAfee disguised himself by colouring his hair and beard grey, darkening his face with shoe polish, padding his cheeks with bubble gum and stuffing his right nostril to give it - in McAfee's own words, "an awkward, lopsided, disgusting appearance".

This rather theatrical approach to disguise apparently helped McAfee observe the police going about their investigations and evade detection until he made his way to Guatemala, where he surfaced earlier this week.

But, fugitives in the future may not have it so easy. Recently, the FBI revealed plans for its Biometric Identification Tools Program, which amongst other things, aims to develop mobile facial recognition software – allowing a field agent to “access the biometric identification power of the US Government, in real time, at any point on the planet”. In essence, it takes the kind of surveillance technology that is commonplace in streets, shopping centres and sports stadiums in Europe and the US, and allows it to be used anywhere in the world.

So would this kind of app succeeded in catching out McAfee? Probably not if it is based on current technology. In reality, facial recognition technology is still surprising clumsy. Some people do not need to do anything nearly as extreme as McAfee to fool facial recognition systems. In fact, they don’t need to do anything at all. That's because certain faces are just too "normal" for facial recognition systems to work with, according to Jean-Luc Dugelay, a video surveillance expert at Eurecom, a French research institution. "Certain people have faces that just seem to be hard for computers to recognize," he said. "It's difficult to know why, and the faces that are hard to recognize vary from one recognition system to another. But if you have something that is close to the average face, then it will be harder for a computer system to recognize you."

Grinning fool

To understand why this might be, it's worth considering how most face recognition systems work. According to Anil Jain, a computer science professor at Michigan State University, they first have to work out that they are being presented with a face - a process known as face detection – and then move on to recognition and matching it with a face that is already known to the system.

Face detection usually involves detecting tell-tale "intensity signatures" of dark and light spots on an image that are typical of a human face. "Humans look for an oval for the face, with two eyes, a nose between them and a mouth beneath," Dr Jain explained. "Computers work in a different way: they don't look for physical features. Instead they may look for a horizontal pattern of dark, light dark, which corresponds to a line between the eyes."

Once a face has been detected, there are a number of techniques that can be used to recognize it. One way is to create a mathematical representation of the face - something known as a "feature vector - that is constructed from pieces of hundreds of "standard faces" in different proportions. These standard faces are known as Eigenfaces, and are themselves generated by analysing thousands of real faces using a process called principal component analysis.

The feature vector is essentially a recipe for a face that can then be used to match it against others with similar feature vectors. However, it may be that with a given set of Eigenfaces, certain human faces produce very similar feature vectors.  Hence, they are "average faces" that are hard for recognition systems to tell apart.

Systems that use Eigenfaces have another flaw which stems from the fact that they need to use the whole face as part of the recognition process. That means that it may be possible to successfully disguise your face against a recognition system simply by grinning or pulling some other face, according to Dr Jain - a strategy which would be unlikely to fool a real person.

Face to the floor

In addition, putting a scarf over the mouth and nose, or simply wearing dark glasses could fool the system. However, this is beginning to change, says Shengcai Liao, an assistant professor at the Center for Biometrics and Security Research in Beijing, China. He says new techniques are being developed that can use information from the nose or mouth alone if the eyes are occluded, or from the eyes and eyebrows if a scarf is covering the lower part of the face.  "It's not possible to recognize a fully occluded face, but we can currently recognize faces with 30% or even 50% occlusion," he said. "We have even had success performing recognition from a mouth alone - something that it would be very difficult for a human to do."

But what about other countermeasures, such as those used by McAfee, which included skin darkening, facial distortion and colouring his hair?

Consider the use of shoe polish to change the colour of the skin. This would mute the intensity signature of a face, but light hitting the contours of the face would still produce an intensity signature that would allow for face detection, according to Dr Jain.

The distortion of the cheeks and nose would also have had little success. Research into the effects of nose jobs – or rhinoplasty - on facial recognition systems carried out by Dugelay show that it has little effect on recognition rates. (Ironically ,experimental facial recognition systems that use 3D imagery and which are generally more accurate than today's  2D systems are far more easily fooled by rhinoplasty, because it alters the shape of the face in three dimensions.)

Colouring the beard and hair would also have no effect, according to Alex Kilpatrick, a facial recognition researcher at Tactical Information Systems, a Texas-based biometric systems firm. "Changing your facial hair changes your appearance to a human dramatically whilst most computer systems are only interested in the area of your face from just above your eyebrows down to your chin, so hairstyles and colouring don't matter at all. You could grow a foot-long beard and it would make no difference," he said

However, there is a sure fire way to beat current systems, he says. Humans are rather good at recognizing people from almost angle, but one of facial recognition systems' key weaknesses is that they have a hard time detecting - let alone recognizing - a face if it is not looking towards the camera, according to Dr Kilpatrick. "Absolutely the easiest thing you can do is look down at your feet," he concluded. "That won't attract much attention, but because surveillance cameras are generally mounted high up or at least at eye level, it will defeat pretty much any recognition system."

If you would like to comment on this story or anything else you have seen on Future, head over to our Facebook page or message us on Twitter.