Humans make judgements about the perceived gender of other humans so quickly that we are usually not aware of having done it. What factors influence these snap decisions?
Ganel et al (2002) suggest that identity and gender are inseparably linked in cognitive facial recognition. This is in contrast to other models that suggest identity and sex are processed in separate pathways; Ganel et al only found these routes separable when gender decisions were based on hairstyle.
Kovacs et al (2004) found that exposing male raters to gender-specific steroid scents influenced their gender judgment.
Hoss et al (2005) found that attractiveness increased speed and accuracy of both male and female classification of faces by adults. Masculinity facilitated classification of males by both adults and children, but facial femininity did not affect identification of females.
A lack of clarity in this decision may be distinctly unsettling.
Krendl et al (2006) found (via fMRI) that amygdala activation was stronger in evaluating transsexual and "unattractive" faces than in evaluating pierced, overweight, or control faces.
Baudouin and Gallay (2006) found that raters responded to composite faces of males or females as more "distinctive" than composite faces that were not specifically gendered.
When I was first transitioning, I tried to stop classifying people by gender. This proved to be a lot more difficult than I expected. All I succeeded in doing was being wrong more often, which was enlightening in and of itself. Still, it's upsetting when you realize that the "lesbian" you've been checking out really is a teenaged boy. The only success I've managed on this front is in being less unsettled when I turn out to be wrong. Since I'm very bad at admitting when I'm wrong in other areas of my life, I consider this a major victory.