Mar 11, 2016 07:40 PM
Statistics is like basketball, or knitting
http://andrewgelman.com/2016/03/11/stati...-knitting/
EXCERPT: I had a recent exchange with a news reporter regarding one of those silly psychology studies. I took a look at the article in question—this time it wasn’t published in Psychological Science or PPNAS so it didn’t get saturation publicity—and indeed it was bad, laughably bad. They didn’t just have the garden of forking paths, they very clearly did a series of analyses, then they finally reached something statistically significant and then they stopped and made some graphs and presented their conclusions.
OK, fine. There’s a lot of incompetent research out there. It’s easier to do bad research than to do good research, so if the bad research keeps getting published and publicized, we can expect to see more of it.
But what about these specific errors, which we keep seeing over and over again. I can’t imagine these researchers are making these mistakes on purpose!
The only reasonable inference to conclude here is that applied statistics is hard. Doing a statistical analysis is like playing basketball, or knitting a sweater. You can get better with practice.
How should we think about all this? To start with...
Behind The Numbers: Jobs Figures, and a Grain of Salt
http://blogs.wsj.com/numbers/behind-the-...od=WSJBlog
EXCERPT: As numbers go, the monthly nonfarm payroll report offers some of the most parsed and open-to-interpretation there are. The report includes all manner of numbers and statistics, from the official unemployment rate to the number of jobs added (or lost) to hourly wages and workweek hours. They’re immensely important as a gauge on the health of the economy, and yet they don’t exist in a vacuum. The people are real and livelihoods are at stake.....
The day after the shock: pollsters, forecasters scratch their heads
http://junkcharts.typepad.com/numbersrul...heads.html
EXCERPT: Pollsters, forecasters, and the likes were embarrassed by the Bernie Sanders upset in Michigan Tuesday night. Nate Silver called it among the greatest polling error in primary history. Now, they are struggling to explain the big miss. Recall the polls conducted close to the contest showed a Clinton lead of about 20 percent points. The actual outcome was a gap of 1.5 points, with a million votes cast. This type of miss is hard to explain because any plausible explanation must deal with these facts....
http://andrewgelman.com/2016/03/11/stati...-knitting/
EXCERPT: I had a recent exchange with a news reporter regarding one of those silly psychology studies. I took a look at the article in question—this time it wasn’t published in Psychological Science or PPNAS so it didn’t get saturation publicity—and indeed it was bad, laughably bad. They didn’t just have the garden of forking paths, they very clearly did a series of analyses, then they finally reached something statistically significant and then they stopped and made some graphs and presented their conclusions.
OK, fine. There’s a lot of incompetent research out there. It’s easier to do bad research than to do good research, so if the bad research keeps getting published and publicized, we can expect to see more of it.
But what about these specific errors, which we keep seeing over and over again. I can’t imagine these researchers are making these mistakes on purpose!
The only reasonable inference to conclude here is that applied statistics is hard. Doing a statistical analysis is like playing basketball, or knitting a sweater. You can get better with practice.
How should we think about all this? To start with...
Behind The Numbers: Jobs Figures, and a Grain of Salt
http://blogs.wsj.com/numbers/behind-the-...od=WSJBlog
EXCERPT: As numbers go, the monthly nonfarm payroll report offers some of the most parsed and open-to-interpretation there are. The report includes all manner of numbers and statistics, from the official unemployment rate to the number of jobs added (or lost) to hourly wages and workweek hours. They’re immensely important as a gauge on the health of the economy, and yet they don’t exist in a vacuum. The people are real and livelihoods are at stake.....
The day after the shock: pollsters, forecasters scratch their heads
http://junkcharts.typepad.com/numbersrul...heads.html
EXCERPT: Pollsters, forecasters, and the likes were embarrassed by the Bernie Sanders upset in Michigan Tuesday night. Nate Silver called it among the greatest polling error in primary history. Now, they are struggling to explain the big miss. Recall the polls conducted close to the contest showed a Clinton lead of about 20 percent points. The actual outcome was a gap of 1.5 points, with a million votes cast. This type of miss is hard to explain because any plausible explanation must deal with these facts....