WHAT CAN YOU TELL FROM A FACE?

.

The picture posted on the web site is of a man is in his 20s, with a dozen tufts of brown hair dotting his mostly shaved head. He is smiling, standing sideways to the camera in what looks like a dorm room.

The site asks whether, at the end of dinner, this anonymous person would:

a. Pay for the check

b. Split the check with you

c. Wait for you to pay

d. Put a roach in the leftovers and demand the meal for free.

Knowing no more than what you see, you click on an answer. Most respondents say he would pay the check.

This is EasyMixing.com, a two-month-old web site created by Yuri Ostrovsky, currently a graduate student of Pawan Sinha, assistant professor of Brain and Cognitive Sciences, and Timothy Nichols, who was a student of Sinha's when both were at the University of Wisconsin at Madison.

Besides being a fun site where you can upload your photo, answer questions about strangers or maybe find a date, Sinha says the site is a large-scale experiment in face perception. And the questions, while sometimes funny, are scientifically designed to pertain to different personality traits.

What information can be gleaned from a face? Ostrovsky and Nichols plan to statistically analyze the responses to assess consistency and variance across observers. As statistics accumulate, they will use them to conduct searches for faces that share similar trait profiles.

Thus, a user may be able to say, "Show me people who are trustworthy, amusing and sincere," and the site will supply them with appropriate individuals. This, Ostrovsky says, is fairly straightforward. A more intriguing question is whether individuals with matching trait profiles will also match visually. If so, users will be able to say, "Show me people who look like this person" and the site will be able to find them.

Finding similar-looking people is an ill-defined problem in computer vision. Computers cannot easily fathom what it means for two people to look like each other.

"The hope is that this new method may be more useful for certain types of face-search problems," Ostrovsky said. "This would be a significant departure from conventional face-search schemes where visual searches are based solely on image structure."

"There are two basic questions this experiment is designed to address," said Sinha, scientific advisor to EasyMixing.com. "Can aspects of an individual's traits (other than identity, age and gender) be reliably assessed just from their facial appearance (and how valid are these assessments)? Conversely, do a collection of such assessments constrain the appearance of a face? If so, can we devise new strategies for searching large databases of faces?"

The web is an ideal medium for the experiment because it provides a very large and diverse collection of faces and a large group of respondents, Ostrovsky said. While most people have preferred to post their pictures than to answer questions, and the site has not been advertised in any way, there are a few hundred members and the numbers are increasing daily.

"It's not a controlled experiment as it would be in a lab, but it's a more realistic experience. There's no experimenter bias where the very design of the experiment coaxes the subject to answer in a particular way, or the subject answers in a way he thinks will please the experimenter," Ostrovsky said.

Accompanying a black-and-white picture of a vampiress-like woman with large upcast eyes and long straight black hair is a question about how she might act in relationships. Most respondents decided that after her last relationship, she was the "psycho girlfriend." Can you really tell from someone's face whether she is a reasonable person or if he would wait 45 minutes for a table in a restaurant? "This is an ongoing project," Ostrovsky said. "We've let it take on a life of its own."

Ostrovsky's thesis is on how people process information to recognize three-dimensional objects. While this includes faces, he says EasyMixing.com is mostly for fun. The real challenge will lie in phase two of the site, in which he will try to come up with a way to use the lists of traits - and no visual cues - to search the face database.

"People may say that a person has an 'honest face' or 'looks aggressive,'" Sinha said. "These inferences may be incorrect, but it is interesting to examine what they are and which facial attributes they correspond to. How consistent are these judgments across different observers and how valid, compared to other independent measures of these traits? These are largely unexplored questions and the answers can turn out to have far-reaching implications."


A version of this article appeared in MIT Tech Talk on December 4, 2002.