April 21, 2014
Martin H. Krieger
1. What to look for: There is always the idea that if you can’t measure it, you are not doing it right. In much of public policy, you try for various measures, but you know as well that they are proxies, and they may not be reliable or even accurate. So what do you do? What you want to do is to do fieldwork, in effect it is sophisticated theory-informed deep journalism. You interview people, you may use some survey instruments (but often this is only helpful after you have found out what’s up). You find out what is happening. In other words, whether it be in health, education, housing, you have to find out what is happening as a consequence of your policies, and then you might be able to figure out how to measure it. If you don’t know what to look for, you are out of business. Some of the time you are doing exploratory research, and you will see surprises. And it’s not a matter of quantitative stuff, per se. It’s phenomena you had not expected.
Reliability. People worry about objectivity or about bias. But looking at the world is surely more accurate than at some data set whose connection with the world is dubious. The problem is not about measureable, it is to figure out what is going on and then you might want to measure it. Preconceived ideas about what is going on are typically artifacts of fantasies. There may be earlier studies, done by sociologists, anthropologists, economists, historians that will inform what you look for.
A wide range of indicators and methods.
a. Robert Sampson (Chicago, now Harvard) just did a book on neighborhood effects in cities, and surely there were dicey neighborhoods. But they drove up and down the streets and photographed the facades of buildings. Sampson is a sociologist, not a public policy person, but his work is central to public policy. (more…)