How to measure race
More by luck than judgement, I have become Mr. Survey at Sustainable Seattle, which I'm actually rather pleased about. We have this happiness survey that we're trying to get everybody to take (fairly successfully - we had more than 6,000 responses the last time I checked), and I am the main person doing both tech support for the online survey application itself and statistical analysis on the data, plus I get some say (as a small part of a much bigger group) on questions of survey design. In reality I haven't had time to actually do much data analysis yet, but that's about to change because we've decided to reduce my training load significantly.
The survey itself has some flaws which will limit what we can validly say from this data, and which I plan on dealing with in two ways: being completely honest about limitations when I make claims from it, and trying to get them all addressed in version 2, which we're already starting to talk about developing. Meanwhile there are plenty of valid comparisons that can be made within this data, and I'm keen to keep promoting even an imperfect survey because it's also the best tool I know of for making people think about what's really important.
I don't get to just dive in and rewrite the survey, which is for the best really - while it will take much longer to do this collaboratively, we'll end up with something much better in the end. But I do have today and tomorrow set aside to update the online survey app, and one of the things I'll be doing is adding a race question to the demographics page. I had initially been the champion of leaving this question out, for reasons that I now think were hopelessly naïve. The main reason I wanted it left out is that race is a social construct that only means as much as we decide to make it mean. Idealistically, I would love to stop the world from talking about race altogether. The trouble is that while race is nonsense, racism is real in both an individual prejudice sense and as a series of institutional barriers to success, and not talking about this doesn't make it go away. Instead it makes the institutional side harder to address, and sends a misleading signal that we don't care about the issue, so the survey will now include a race question.
That, though, opens its own can of worms. The secondary reason I don't like asking race questions is that it's impossible to ask the question right. We have two options: make everyone write their own in (a nightmare for data analysis!), or make people choose from a frustrating list. As an example of why these lists are so frustrating, consider the "Caucasian" category that is often what I am expected to check. It means different things in different contexts:
And then there's the question of resolution. Which groups we lump together and which we split is very much a product of which questions we're trying to answer. Given that our purpose is to get at how big an effect racism has on peoples' happiness, we could almost get away with asking "are you a member of a race or ethnic group that is discriminated against in your country?", but we already ask whether people feel discriminated against in their own lives, and we want more objective information to go with that. Yesterday I found a fascinating reference for the diverse ways race questions are asked around the world: everything from Canada's breakdown of First Nations into subgroups to Brazil's shades-of-brown palette and Bulgaria's "Bulgarian or Turk or Gypsy or Other".
For the time being, we're running with the US Census categories because at least in this country those are sort of understood, and we can credit/blame the Census Bureau for the particular selection. But we really want this survey to make sense internationally. We have a Spanish translation almost ready to go, Arabic and Chinese in the works, and groups interested in using the instrument in Norfolk Island and Spain. The other internationalisation issues are relatively easy to work out-the update I'm doing this week will make it handle non-US location coding properly, and there are a few questions that need slight rewording-but I still really don't know how to ask the race question right. Is there an appropriate universal way?
The survey itself has some flaws which will limit what we can validly say from this data, and which I plan on dealing with in two ways: being completely honest about limitations when I make claims from it, and trying to get them all addressed in version 2, which we're already starting to talk about developing. Meanwhile there are plenty of valid comparisons that can be made within this data, and I'm keen to keep promoting even an imperfect survey because it's also the best tool I know of for making people think about what's really important.
I don't get to just dive in and rewrite the survey, which is for the best really - while it will take much longer to do this collaboratively, we'll end up with something much better in the end. But I do have today and tomorrow set aside to update the online survey app, and one of the things I'll be doing is adding a race question to the demographics page. I had initially been the champion of leaving this question out, for reasons that I now think were hopelessly naïve. The main reason I wanted it left out is that race is a social construct that only means as much as we decide to make it mean. Idealistically, I would love to stop the world from talking about race altogether. The trouble is that while race is nonsense, racism is real in both an individual prejudice sense and as a series of institutional barriers to success, and not talking about this doesn't make it go away. Instead it makes the institutional side harder to address, and sends a misleading signal that we don't care about the issue, so the survey will now include a race question.
That, though, opens its own can of worms. The secondary reason I don't like asking race questions is that it's impossible to ask the question right. We have two options: make everyone write their own in (a nightmare for data analysis!), or make people choose from a frustrating list. As an example of why these lists are so frustrating, consider the "Caucasian" category that is often what I am expected to check. It means different things in different contexts:
- Its origin is synonymous with "Indo-European", in that it describes an inferred migration of people out from the Caucasus, into most of Europe and some of Central Asia and the Indian subcontinent, bringing a language family with them.
- In the US, it's generally used as a shorthand for "white".
- In Singapore it seems to be shorthand for "of European descent".
- In Russia it means "from the Caucasus region".
And then there's the question of resolution. Which groups we lump together and which we split is very much a product of which questions we're trying to answer. Given that our purpose is to get at how big an effect racism has on peoples' happiness, we could almost get away with asking "are you a member of a race or ethnic group that is discriminated against in your country?", but we already ask whether people feel discriminated against in their own lives, and we want more objective information to go with that. Yesterday I found a fascinating reference for the diverse ways race questions are asked around the world: everything from Canada's breakdown of First Nations into subgroups to Brazil's shades-of-brown palette and Bulgaria's "Bulgarian or Turk or Gypsy or Other".
For the time being, we're running with the US Census categories because at least in this country those are sort of understood, and we can credit/blame the Census Bureau for the particular selection. But we really want this survey to make sense internationally. We have a Spanish translation almost ready to go, Arabic and Chinese in the works, and groups interested in using the instrument in Norfolk Island and Spain. The other internationalisation issues are relatively easy to work out-the update I'm doing this week will make it handle non-US location coding properly, and there are a few questions that need slight rewording-but I still really don't know how to ask the race question right. Is there an appropriate universal way?