Face-time with iPhoto ’09’s Faces feature

iLife ’09 hit retail shelves this past week, bringing with it perhaps one of the most anticipated additions to the suite of digital lifestyle apps—the Faces feature in iPhoto ’09.

As outlined by Apple at this month’s Macworld Expo, Faces incorporates face-detection and -recognition technology to allow iPhoto to identify faces in your photos, matching them with other faces throughout your photo library. You can read more about Faces in our preview of iPhoto ’09.

Of course, now we’ve had the chance to spend some quality time with iPhoto ’09, allowing us to answer some of the more burning questions we’ve been hearing about Faces: How does it perform? Will it recognize people—kids, say—as they age? And what about pets? Several Macworld editors have spent the week playing around with iPhoto; here are their initial impressions of the Faces feature.

How it works

When you first click on the Faces item in the iPhoto sidebar, iPhoto automatically analyzes all your photos. How long this process takes depends mainly on how many photos you have. If you’ve got several thousand photos, be prepared to wait quite a while. Jackie Dove reported that iPhoto took about eight minutes to scan 1,373 photos, for example. iPhoto estimates how long the process will take; in our testing, that estimate was pretty close to accurate.

Once you’re up and running, Faces will automatically scan any new photos you import and if it recognizes the faces, will automatically tag them properly.

After iPhoto finishes its facial analysis, you have to tell the application who is who. Select a photo and then click on the Name button at the bottom of the window. Boxes will appear around each face iPhoto has detected, along with a “name tag” below the photo—at this point, all photos will be labeled “unknown face.” Click on the “unknown face” text for a photo and enter the person’s name. In the background iPhoto will immediately try to find other photos containing the same person, and an entry for that person will appear in the Faces view.

If there are multiple people in a photo, you can identify them all at once. If iPhoto doesn’t automatically recognize a face, you can manually add a face box by clicking on the Add Missing Face button, positioning the box that appears over that face, and typing the person’s name. (You can also manually remove an automatically-generated face box—for example, for someone you don’t know or don’t plan on using with the Face feature—by clicking on the box and then clicking on the X in the upper-left corner of that box.) Once you’re done naming faces, click on the Done button to finish.

The Name view is also very easily navigated via the keyboard: you can press tab to move between faces and to confirm identities Faces isn’t sure about. When a particular face is selected, you can press return to begin typing the name of the person you’re identifying. By using the tab and return keys, along with the arrow keys, you can quickly tag a lot of faces in a lot of photos.

Faces will do its thing with just a single instance of each person, but the more examples of a person you label, the better iPhoto’s facial recognition feature will be—the more photos it will be able to find containing that person, and the more accurate its guesses will be. For example, after Dan Frakes named just a few photos of his wife, Faces found only 20 or so photos of her out of several hundred; once he manually labeled 30, faces found well over 100.

Once you’ve started naming people, the Faces view in iPhoto shows a tile—which looks like an old Polaroid snapshot—for each person. (Apple actually calls the Faces view the “corkboard,” because it looks a bit like a bulletin board with snapshots tacked to it.) Moving your mouse cursor over one of these tiles flips through all the photos it contains, similar to how you can quickly scan the photos in an iPhoto Event. Double-click a person’s tile to view all photos containing that person. (In this case, “all” means all those images you have manually labeled as well as all those iPhoto thinks contain that person.) View buttons at the bottom of the window let you toggle between viewing each photo in its entirety or zooming in on just the selected person. If you’ve got videos in your iPhoto library—for example, movies you’ve taken with your digital camera—Faces will even try to identify movies that include the person.

When you double-click a person’s tile, the ensuing view also displays, at the bottom of the list of photos, a section of images iPhoto suspects may include that person but that haven’t yet been confirmed. Click on Confirm Name and the view changes to display a zoomed-in view the selected person—or what iPhoto thinks is that person—for each photo, along with the label Click To Confirm. Click once on an image to confirm that it does, indeed, contain the selected person; click twice if it does not. When you’re done confirming and rejecting photos, click on Done.

Here’s a tip: Once you click on Done, go back and view the person’s Faces view again, as there’s a good chance that confirming and rejecting iPhoto’s previous guesses will have caused it to find more matches for that person.

The Faces training system could be a bit more straightforward, and it’s not as automatic as you’d expect. For example, we were hoping for some sort of batch-training mode, but you have to spend quite a bit of time manually confirming or rejecting each suspected match. You need to enter a collection of a person’s photos to get iPhoto started, then click Confirm Name and hope that there are some good candidate photos for you to train. (It’d be nice if the naming mode was the default for unconfirmed photos.) And, as mentioned above, once you’ve done a single training pass, you may discover more photos are available for training, so you can take another pass. Faces would benefit from an initial step-by-step training system to seed your library with a solid amount of data.

Here’s a little shortcut: Click and drag to confirm a whole group of photos; option-click and drag to dismiss a whole group of photos. You can also drag a photo above the dividing line between confirmed and unconfirmed photos to accept it. It would be nice if there were a way to just select the ones that aren’t the person in question and then confirm the rest without having to click on them all. We also wish that, when iPhoto comes up with a wrong suggestion, you could tell iPhoto who that person really is while in Confirmation mode, rather than simply marking them as “not [somebody].” But it doesn’t appear that there’s a way to do this.

When recognition fails

If you think Faces will automatically recognize everyone in your iPhoto library, you might want to recalibrate your expectations. It does learn and grow, but there’s still a lot of manual tagging involved, too. Of course, in many photos people are not looking directly at the camera—Faces needs to learn about faces from different angles before it can recognize a particular person at a particular angle. In several photos we tested in which people were looking straight at the camera, Faces detected no faces at all, requiring us to place a box over the person’s face and then name that person. (And Faces’ box control is a little weird—when you grab an edge and resize it, the entire box resizes, not just the corner you’re grabbing. So the right process is to place the face in the center of the box, then shrink to fit.) This manual-selection process also requires quite a bit of mousing around.

All our recognition stories were similar. Jackie Dove reported Faces recognizing pictures of her as far back as 10 years ago. But other photos were completely unrecognized. Jason Snell found that Faces recognized his daughter even with half of the photos in a sample library dating from her first year and the other half dating from her at age 7. By lumping both the baby and the second-grader under the name, Faces figured out that the two very different faces were part of the same person.

Kelly Turner was impressed with the range of faces iPhoto found, including in low resolution images, images with terrible color casts, underexposed images that are very dark, and slightly obscured faces. In only a few cases has it detected a face where there wasn’t one. And at least in initial training, face recognition was definitely a bit wacky. While trying to detect Kelly’s face, iPhoto repeatedly came up with an ex-boyfriend, lots of relatives (which makes sense), friends who look very different from her (including a very dark-skinned Jamaican friend) and Neve Campbell (a face it had detected in a poster on the wall behind one photo!). Kelly is actually flattered at that last one.

Dan Frakes was both impressed and disappointed by the extremes of success and failure he experienced with Faces. On the one hand, like Jason, he found that Faces correctly identified a wide range of pictures of his older daughter during her baby and toddler years—a period in which her face changed dramatically. And Faces successfully identified a good number of faces in out-of-focus pictures. On the other hand, Faces seemed to think every blonde-haired female in his photo library was the same person, even ones 30 years apart in age, and Faces occasionally “finds” faces in flowers, bottles, and other inanimate objects. On numerous occasions, Faces suggested pictures of a man as matches for a woman, and vice versa. (We’re talking men with short, brown hair for women with long, blonde hair.) And we all got a kick out of iPhoto suggesting that happy faces on toys in photos of his kids were matches for 40-year-old men.

And sorry, animal lovers—Faces didn’t work very well with pets in our tests. (Though others have reported mixed results.)

Limited access

Despite Faces’ new role as a marquee addition to iPhoto, the feature is largely sequestered in the Faces interface itself. Face information doesn’t seem to appear outside of the individual boxes overlayed on a photo. You can view keywords below all photos in the Photos view, but apparently not faces.

However, you can create Smart Albums that are targeted on a particular face or faces, by using the “Name is” rule.

Those who try to gain access to Faces data via AppleScript are also likely to be disappointed. We couldn’t figure out any way to extract face information from iPhoto via script.

More to come

Hungry for more information on Faces in particular and iPhoto ’09 in general? Check out Christopher Breen’s hands-on video with iPhoto ’09.

We’ll also have a final word on Faces—and the rest of the features and enhancements in the new version of iPhoto—in our review of iPhoto ’09 next week.

Editorial director Jason Snell and senior editors Jackie Dove, Dan Frakes, and Kelly Turner contributed to this report.