What if You Can Judge a Book by its Cover?

They say you can’t tell a book by its cover. But what if you actually can?

This is the subject explored in Matthew Hertenstein’s new book, The Tell: The Little Clues That Reveal Big Truths about Who We Are.

Hertenstein, a psychologist who runs the Touch and Emotions Lab at DePauw University, studies the science of nonverbal communication and what our faces and bodies say—and don’t say—about our personalities, attractiveness, and trajectory for success. His book is a revealing look at the mind’s powers of social observation.

A “tell,” explains Hertenstein, is what poker players call the subtle nonverbal clues opponents inadvertently display when they are bluffing. We are all attuned to these tells, whether or not we are aware of it, because our brains are constantly on the alert to understand and predict other people’s behaviors and intentions. And, although we can be wrong interpreting them, we are more right about them than we might think.

For example, we often use physical attributes to form our impressions. Tallness is associated with power; therefore, we generally choose taller men in presidential elections. Men also find women with lower waist-to-hip ratios more attractive, probably because this physique is associated with fertility.

These may be familiar to readers, as they’ve been discussed before in the media—and their cultural ramifications have been extensively debated.

But did you know that men with wide faces are assumed to be more aggressive, and in fact will face stiffer penalties if convicted of a crime? In fact, wider-faced men do commit more crimes, possibly because of a connection between wide faces and higher levels of testosterone. This suggests that, in some cases, our brains may be better equipped than our conscious minds of making judgments based on tells like face structure.

There are several more examples in the book. We can tell whether or not someone will, on average, do better on intelligence tests based on three non-verbal clues: eye gaze, vocal qualities, and attractiveness (although only for those considered to be “in the lower half of the attractiveness distribution”).

Expert couples therapist John Gottman can tell whether or not couples will stay together based on 3-minute conversations, while Hertenstein’s own research has predicted couples staying together based on smiles in high-school yearbook photos. Research has found that certain tells can help us determine whether or not someone is lying (though not the ones we usually associate with lying, like nervousness or looking away), and whether or not someone will act pro-socially in experimental economics games.

Of course, the science isn’t foolproof, and therein lies the rub. The consequences of trusting our tells are dire when we’re talking about accusing people of crimes or meting out punishment based on face shape, or when we elect presidents based on their height. The right waist-to-hip ratio may suggest fertility, but it doesn’t tell a man anything about the woman’s ability to be a good mother and partner. On the other hand, understanding the tells of babies who are at risk of developing autism could be useful for parents seeking early intervention.

In the end, Hertenstein suggests we should trust our instincts more, not less—that, in fact, our brains seem to be equipped to detect certain things about other people better than we could with conscious effort.

But it’s hard for the reader to know what to do with this information, except marvel at science and the mysteries of our minds. Hertenstein even encourages this by having “party-worthy findings” listed at the end of his chapters, consisting of provocative or unexpected findings in the research. That’s fun, but it’s a poor guide to knowing when we should cognitively override kneejerk, even prejudiced, reactions.

To be fair, Hertenstein warns his readers early on not to expect a self-help book. But much of what he does prescribe in the book would take special training—such as being attuned to the nuances of facial expressions to catch liars. Should we trust our unconscious minds to make these social decisions? Or should we be questioning our assumptions? The research seems to suggest both possibilities, depending on the “tell,” which doesn’t exactly leave the reader with a clear course of action.

Still, I liked this book. It certainly is entertaining—in the Malcolm Gladwell-ian tradition. Perhaps with a few more decades of research we will be closer to understanding our non-verbal communication and know how to harness it better. In the meantime, it’s interesting to see how much we can read about each other, even from the smallest of cues.