Much of what we do here at CMF involves research on microfinance that aims to have strong internal validity. Proponents of the kind of evaluations we are involved in point to the fact that in our studies we are able to eliminate confounding variables by randomizing the groups who do and do not receive an intervention. We also aim to collect data through questionnaires that are rigorous and unbiased. I am proud that we try to do this, but I can’t emphasize enough that collecting data in this manner is REALLY hard.
This came to my mind after reading this article on the unreliability of exit polling from the website www.fivethirtyeight.com. It is an excellent examination of the variety of ways that data can be collected poorly. Reason number 4 is particularly pertinent to our research.
If you are interested in the
3 comments:
Good point. Unfortunately, these problems are even more acute in the case of normal pre-election polls which are typically conducted via random digit dialing. In the US these days, the proportion of people who actually take the time to answer questions from a pollster over the phone is miniscule (I know I don't) so extrapolating from this extremely biased sample to the general voting population is extremely difficult.
Second, the polling companies are not incentivized to simply gauge public opinion as accurately as possible. There is a huge payoff to a polling agency if their prediction goes against that of most other polling agencies and their prediction turns out to be correct which can subtly influence how the pollsters call elections.
http://www.nytimes.com/2008/11/10/business/media/10silver.html?em
a times article on fivethirtyeight...had no idea he started by playing around with baseball stats.
Yeah, I came to know about him from following his baseball site. It is crazy he has become so famous.
Post a Comment