Recommendation letters
Schmader et al (2007) analyzed 886 letters of recommendation for male (235) and female (42) applicants to tenure-track positions in Chemistry and Biochemistry. Very few gender differences reached even a p < .05 level of significance (possibly partially due to the very small female sample). In fact, more differences were found between the tone and word types of the letters between departments than between genders. Biochemistry recommendation letters, for example, contained more negative feeling words and fewer positive feeling words. However "standout adjectives" (e.g. outstanding, unique, exceptional) were more often found in letters written for male applicants (although it is unclear in the paper whether this is based on subgroup analysis, see "Underpowered", 8/24/07 for more info). No analysis was done on the gender of the letter writer.
A similar study was done by Trix and Psenka (2003) on recommendation letters for applicants to the medical faculty (71% male). The recipients of these letters (those reviewing the applications) were 96% male, and the writers were 85% male. Trix and Psenka found letters for female applicants were shorter (avg 26 words shorter), and tended to include more "doubt raisers" -- hesitant language, qualifiers, and faint praise.
I have to admit a certain amount of skepticism about most types of language analysis. Schmader et al used Pennebaker's 2001 "Linguistic Inquiry and Word Count" (LIWC) software, which was also used in gender predictions of blog text, discussed in December. Trix and Psenka used a more qualitative method involving non-blind reviewer ratings of letters (and here I admit extreme skepticism about qualitative analysis). It's extremely easy to find sexism when you're looking for it. It is not clear if there is any difference in the year in which the letters were written. Trix and Psenka's letters were written in the 1990's, while there is no date information on Schmader's sample.
Business writing does not constitute natural language use for most people. It tends to say more about the letter writer's training than about the letter subject. I was extremely disappointed by the lack of writer-analysis in Schmader's study, and a study I'd be very interested to see is an analysis of the sources of recommendation letters for male and female applicants: is there a difference in which, and how many, people they ask for letters of recommendation?
A similar study was done by Trix and Psenka (2003) on recommendation letters for applicants to the medical faculty (71% male). The recipients of these letters (those reviewing the applications) were 96% male, and the writers were 85% male. Trix and Psenka found letters for female applicants were shorter (avg 26 words shorter), and tended to include more "doubt raisers" -- hesitant language, qualifiers, and faint praise.
I have to admit a certain amount of skepticism about most types of language analysis. Schmader et al used Pennebaker's 2001 "Linguistic Inquiry and Word Count" (LIWC) software, which was also used in gender predictions of blog text, discussed in December. Trix and Psenka used a more qualitative method involving non-blind reviewer ratings of letters (and here I admit extreme skepticism about qualitative analysis). It's extremely easy to find sexism when you're looking for it. It is not clear if there is any difference in the year in which the letters were written. Trix and Psenka's letters were written in the 1990's, while there is no date information on Schmader's sample.
Business writing does not constitute natural language use for most people. It tends to say more about the letter writer's training than about the letter subject. I was extremely disappointed by the lack of writer-analysis in Schmader's study, and a study I'd be very interested to see is an analysis of the sources of recommendation letters for male and female applicants: is there a difference in which, and how many, people they ask for letters of recommendation?