Face ID on the iPhone X: Everything you need to know about Apple’s facial recognition

The  iPhone X uses Face ID, technology that unlocks the phone by using infrared and visible light scans to uniquely identify your face. It works in a variety of conditions and is extremely secure.

This brings up a lot of questions about Face ID, which is why we have this Face ID FAQ.

Face ID white paper and support document from Apple

Apple has released a Face ID white paper (PDF) that provides a bit of detail on how it works. There’s also a Face ID support document that explains how the technology protects your information.

What is Face ID?

Face ID a form of biometric authentication. Rather than a password (something you know) or a security dongle or authentication app (something you have), biometrics are something you are. Fingerprint recognition is also a biometric.

Instead of one or more fingerprints, as with Touch ID, Face ID relies on the unique characteristics of your face. Apple is betting that its technology can meet six separate hurdles:

• Initially scan your face accurately enough to recognize it later.
• Compare a new scan with the stored one with enough flexibility to recognize you nearly all the time.
• Scan your face in a wide variety of lighting conditions.
• Update your facial details as you age, change hairstyles, grow a mustache, change your eyebrows, get plastic surgery, and so forth to still recognize you.
• Let you wear hats, scarves, gloves, contact lenses, and sunglasses, and still be recognized.
• Not allow a similar-looking person, a photograph, a mask, or other techniques to unlock your phone.

If you’ve had trouble with Touch ID and your fingerprints over time, you might have concerns about whether scanning and recognizing a face is easier. It might be!

What devices use Face ID?

At this announcement, only the iPhone X is slated to include Face ID.

Does Face ID replace Touch ID?

For now, the hardware requirements for Face ID are only found in the upcoming iPhone X. All other new iPhone models and all existing ones will retain Touch ID. Apple might choose to retain Face ID as a premium differentiating feature, but based on the company’s previous patterns of high-end feature rollouts, it will likely be in both new iPhone models in late 2018, and then in the iPhone SE at some point thereafter. It would seemingly also be likely for future iPad Pro updates, but perhaps not until the second quarter of 2018 or even mid-year.

What will I use Face ID for?

The same things you currently do: Apple Pay, App Store and iTunes purchases, and third-party apps that currently rely on Touch ID. Apple says that third-party apps—as with Touch ID—will be able to allow a Face ID authentication, and iOS only informs the app whether or not the match was accurate.

Interestingly, Apple says developers can use Face ID without a fallback to a passcode if a developer wants to use the biometric identification as a kind of second factor that can’t be bypassed.  

Third parties will also have access to live depth maps, just as the rear two-camera systems provide in iOS 11, but not the raw data of sensors sampling your face.

How do I set up Face ID?

Face ID uses an “enrollment” process just like with Touch ID. You’ll go to Settings > Face ID & Passcode and tap Enroll Face, and then the iPhone will use the front-facing camera to display your face within a circle with green tick marks surrounding it. The enrollment software will overlay quasi-3D markings onscreen to show your eye line and facial center. You’ll be prompted to move your head in a circle, while your facial characteristics are captured.

Apple

Apple says the odds that someone else’s fingerprint will unlock Touch ID is 1 in 50,000, a pretty low number given there’s no way to test for that without trying. Apple says Face ID’s chance of another face matching is 1 in 1,000,000.

Apple’s senior vice president of worldwide marketing, Phil Schiller, did say during the iPhone X introduction that, “The statistics are lower if the person shares a close genetic relationship with you.” Apple clarified this in its white paper, noting that the accuracy is “different” for twins and siblings. If you have an evil twin, you should probably avoid Face ID.

It also said that children under 13 had a higher rate of false matches, though it didn’t provide a number, because distinct facial features “may not have fully developed.”

How does Face ID work?

Apple uses a combination of infrared emitter and sensor (which it calls TrueDepth) to paint 30,000 points of infrared light on and around your face and also capture flat or 2D infrared snapshots. For the points, the reflection is measured, which allows it to calculate depth and angle from the camera for each dot and construct a depth map. 

Apple

Live depth mapping is also used for live tracking for Animoji, the talking animals heads—and piles of poo—that match your facial expressions and lip movement, and other selfie special effects, and is provided to third-party developers. But live depth mapping doesn’t offer up raw sensor data that would let a developer re-create a Face ID map.

How do I unlock with Face ID?

The phone isn’t scanning all the time, thank goodness! Instead, you’ll need to wake the phone with one of several methods, which includes rise to wake or tapping the Sleep/Wake button or an app or Apple service requesting authentication. Then you’ll look at the phone. Apple says the infrared sensors should penetrate sunglasses, but your eyes need to be open—so kids can’t unlock the phone of a sleeping parent by sneaking into their room, unless said parent sleeps with their eyes open.

To avoid unintentional unlocking, Apple says Face ID is “attention aware.” If you’re not looking at the phone attentively—that is, you’re looking away or engaged in an activity on the phone’s lock screen—it won’t perform an unlock operation. This awareness can be turned off by a user as an accessibility option. (Thus a sleeping parent with eyes open would also have to be looking intently!)

Apple says that Face ID can be used in a wide variety of lighting conditions, including dark rooms. On one diagram of the iPhone X, Apple shows a “flood illuminator,” which conceivably provides infrared illumination in the dark to aid the TrueDepth system. No visible light is used, so the scan will be invisible in all lighting conditions.

Over time, Apple will create temporary updates to your Face ID profile for good matches that aren’t perfect to keep it up to date. But it warns these updates are only good for a “finite” number of unlocks, meaning it expects changes to your face to either revert to the mean—you changed eye shadow and eyebrow shape and change back, or grow a mustache and shave it?—or you will have to re-enroll your face. The details aren’t clear.

Interestingly, if you fail to unlock your phone with Face ID but the failure was iffy—close, but not close enough of a match—and then you immediately enter your passcode successfully, Apple says it will take a new capture and adapt the stored Face ID profile with it. It will retain this new information for a period of time as with other facial changes above, but it will also discard it if you stop matching against the changes and look like your old self again.

Apple

How do I use Face ID with Apple Pay?

It’s a little different than with Touch ID, because you can tap and then confirm with your fingerprint with Touch ID while your device is still within range of the payment reader.

With Face ID, it works like this: 

• Before you put your iPhone X near the contactless reader, you have to “confirm intent”: double-click the side button, not the Home button, which is the default with Touch ID.
• Next, you authenticate with Face ID.
• Finally, you place your phone near the payment reader.

Apple notes that you can change your payment type after authenticating with Face ID and before going ahead with the payment: you tap the different payment method and then authenticate again with Face ID.

For Apple Pay within apps and through Safari, you double-click the side button and then authenticate with Face ID within 30 seconds. Otherwise, you have to double-click and authenticate again.

How secure is Face ID?

Apple’s description of enrollment and comparison is very similar to Touch ID. The enrollment sends data through a one-way channel to the Secure Enclave, a special tamper-resistant chip bound deeply inside the iPhone and iPad architecture that can only respond with limited information, such as confirming a match was made when unlocking for Apple Pay and the like. Secure Enclave also stores some other private information.

As a result, Apple doesn’t collect this information and process it centrally, nor does it store it on the device in a manner that can be retrieved by cracking a phone, a phone backup, or intercepting information to and from it.

However, the concern remains that, with proprietary technology under the control of Apple, a government could force changes that would pass or extract facial identification information, or perform comparisons with faces that a government is looking for.

In the current hardware architecture, however, that seems unlikely. Apple has engineered its systems so that there’s no reasonable way to rework it to change the flow of facial (or, with Touch ID, fingerprint) information to a different source. It would have to create a whole new kind of phone and new firmware.

Apple notes that developers can create encryption keys protected by Face ID that are stored in an iPhone X’s Secure Enclave. These keys can be used by the developer’s app entirely within the Secure Enclave, performing operations that are as protected as Apple Pay and biometric identification. It will be interesting to see how developers make us of this additional security level. (It doesn’t seem as if this is an option for Touch ID.)

Can someone fool Face ID?

The answer should be no, based on the approach Apple has taken: number of data points, use of infrared scanning, and attention awareness. With other systems, people have used photographs, plaster models, and other approaches that Face ID would seemingly resist. A plaster model doesn’t offer the same reflection as a 100 percent identical human face, because infrared reflects off living skin differently than off an inanimate material.

Apple says in its white paper that it introduces randomness to make it even more difficult. The sequence of 2D infrared scans and depth-map dot captures are sent in a random order, and the project dot-pattern is both random and unique for each device. This makes it harder for an attacker to use predictable elements to dupe a scanner—or they might succeed in fooling an iPhone X in their possession, but fail with any other iPhone X out there.

The Face ID security white paper notes in passing that while there’s a primary neural network that performs identification, a second neural network checks against spoofing, looking out for telltales of photos and masks.

Bkav, a security company and smartphone manufacturer in Vietnam, posted a video in November that demonstrated how Face ID can be fooled with a mask made with “materials and tools are casual for anyone.”

Bkav believes its mask method for fooling Face ID is easy for anyone to perform in “9-10 hours.” Here’s what the process involves (decide for yourself if this is easy):

1. Find a way to make a record of the face for the mask. If you can’t get direct access to the actual face from the person you want to copy, Bkav says you can set up a room with “a pre-setup system of cameras located at different angles” to secretly take photos of your subject. We assume those cameras should be hidden.
2. Get your subject in the room with multiple hidden cameras and secretly take photos of that person’s face.
3. Assemble those photos in a 3D modeling program to create a 3D model for the mask.
4. Use a 3D printer to create the mask out of stone powder.

Maybe a corporate or government agency or Malcolm Walsh can pull of something like this, but your typical phone thief? Your local police department probably doesn’t have the means to do this, either (despite what network TV cop dramas would have you believe). If you’re in a position where you truly think you could be in this kind of scenario, a movie should be made about your life. And use a passcode instead of Face ID.

Can Face ID be set up so that multiple people’s faces unlock the same iPhone?

For now, an iPhone X will recognize only a single face. That could change in the future. But you can no longer give a spouse, partner, or other person access to your phone through a biometric means, like you could by enrolling one of their fingers with Touch ID. You’ll need to share a password with them.

According to a MacRumors report, an email from Apple VP Craig Federighi to a customer states that Touch ID was intended for a single user only, not for multiple users.

Will Face ID recognize people of color’s faces as well as it does white people’s faces?

We hope Apple has learned from machine-learning and body-recognition debacles at other companies that have led to people of color not having the same accuracy of automatic photo tagging, facial recognition, and other problems. (See the video of a man unable to get a hand drier seemingly to recognize his hand color.)

The kind of machine learning used widely now for voice, image, and other recognition relies on training databases. Companies or academic projects have to find often hundreds of millions to s of examples that they can mark correctly to feed into a deep-learning system to have it develop the pathways that let it recognize features more generically instead of as exact matches.

In the past, these training databases have apparently been heavily biased towards white faces and often towards men, leading to racially insensitive and upsetting results. Apple VP Phil Schiller said in the keynote that Apple used a billion images to train Face ID, but not which faces.

In its announcement and on its website, Apple features a number of people of color more in proportion with the global population than American or European ones in the Face ID and other TrueDepth sections, as well as showing heavily freckled faces and women with elaborate and enormous hair.

Apple’s senior VP Craig Federighi later  expanded on this to TechCrunch, explaining, “We’d done data gathering around the globe to make sure that we had broad geographic and ethnic data sets.” In Apple’s white paper, the company notes it created “a representative group of people accounting for gender, age, ethnicity, and other factors.”

Apple also keeps Face ID enrollment strictly on device, which means it can’t learn directly from real-world usage about how well its algorithm performs for given individuals’ faces. This is great from a privacy standpoint, but might lead to awkward results. (Federighi confirmed to TechCrunch that customers’ use of Face ID won’t in any way be folded into its training. “We do not gather customer data when you enroll in Face ID, it stays on your device, we do not send it to the cloud for training data,” he told the news outlet.)

Apple

How will Apple improve facial recognition if it keeps all Face ID matches private to each iPhone?

We assume it will keep testing and improving its neural network, as machine learning only gets better with bigger training sets. But it also mentions in its white paper something called Face ID Diagnostics. This will aid its customers who can’t get Face ID to work reliably, and who are willing to send diagnostic data back to Apple, which can then obviously improve its algorithms.

The diagnostic mode requires a lot of consent:

• Apple has to send a cryptographically signed request to authorize enabling Face ID Diagnostics.
• Once enabled, the existing Face ID is enrollment is deleted and you set up a new one.
• All unlock attempts are captured for 7 days and then it stops.
• You can review everything captured, and aren’t required to send it to Apple, and can approve specific images instead of all of them. Images you reject are deleted immediately.
• Any images you approve are encrypted before transmitting and then deleted from your iPhone.
• Face ID Diagnostics automatically ends after 90 days if you don’t review image and approve any images. It can also be disabled manually.

It’s cold (or hot), and I’m wearing something on or near my face. Will my accoutrements prevent Face ID from working? 

Apple says in its white paper that in addition to making sure it recognizes faces around the globe, Face ID can also handle “hats, scarves, glasses, contact lenses, and many sunglasses.” Infrared can penetrate most sunglasses to identify an attentive unlock.

I was in an accident and suffered a facial injury. Am I locked out of my iPhone?

No. Your passcode is always the key to unlock your phone. Face ID (and Touch ID) are a convenience as relates to all iOS purposes. You can disable and re-enroll in Face ID with the passcode as well. (We received this question from a future iPhone X buyer. And we’re sorry to hear about the accident.)

Is Face ID more annoying to use than Touch ID?

That’s entirely subjective, but if you prefer to unlock your phone without paying much attention to it, and have gotten used to a smooth sequence of finger slip and touch and press to unlock without even looking at your phone, you might find Face ID much more irritating than Touch ID. Some people dislike the Touch ID sequence, and would prefer a different method that involves less interaction, and Face ID might be the ticket.

Other phones have facial recognition and aren’t great at it. Will Face ID be better?

Apple wades into a market full of people disappointed by previous attempts to get a kind of technology to work. Sound familiar? A New York Times review of the Samsung Note 8—which has iris, facial, and fingerprint scanning—offered this scathing opinion: “Some of the biometrics, including the ability to unlock your phone by scanning your face or irises, are so poorly executed that they feel like marketing gimmicks as opposed to actual security features.”

Until all-comers can start using Face ID, we can’t know whether or not it will perform up to the standards that Apple has set for widescale introduction of features other companies offer failed or weak versions of. Touch ID’s early days were generally good, but plenty of people—this writer included—had to regularly retrain fingers. Over time, that problem seems to have disappeared, as complaints are rare these days in forums in which they were once common.

The real test is one of time: as you change make-up, glasses, and hair styles, and as our features tick away the passage of time, will Face ID keep up with us?

Face ID appeared to fail for Apple exec Craig Federighi during the demo. Apple later said that because the demo unit was handled by a number of people between when Federighi trained it and the demo, it exceeded the bad-match limit, a security feature, and locked out Face ID as it was supposed to.

Do I have to use Face ID?

No, you can use a passcode.

In fact, you can use a passcode at any time instead of Face ID, except in cases of third-party apps that are using Face ID explicitly as a biometric second factor. Apple doesn’t rely on this for unlocking iOS or other purposes yet.

How do I disable Face ID?

You have several options:

• You can use Settings in iOS 11 to disable it.
• You can power down your phone and restart it.
• You can make five failed attempts to unlock it.
• You can press the Wake/Sleep button five times in sequence. (This disables Touch ID in iOS 11 on all models.)
• You can press and hold the Wake/Sleep button and either volume button on the opposite side while the phone is sleeping. This new option triggers a power-down screen, but also disables Face ID. (This also works to disable Touch ID on an iPhone 8 and 8 Plus, but isn’t available on older phones.)
• You send a remote lock command via Find My iPhone.

When can’t I use Face ID?

In several cases, you have to use a passcode (these parameters also apply with Touch ID):

• After restarting.
• After any 48-hour period in which you haven’t used Face ID to unlock the phone.
• After 6 1/2 days, a four-hour timer starts, and if you don’t use Face ID within those four hours, you’re prompted for a passcode the next time you unlock.
• After five failed attempts to unlock with Face ID.
• After disabling using any technique above not already mentioned in this list.