Mind the gap: Utah edition.
Year after year, Lisa Schencker of the Salt Lake Tribune had been seeing test scores that as a whole looked pretty good for her state, but she always noticed the disaggregated data didn’t stack up as well. “I suspected this was partly because of our unusually high proportion of white students (pulling up the state’s average), but I didn’t feel I personally had the statistical prowess to prove that,” she wrote me.
Recently, she asked that question during a teleconference about the NAEP math release, which again looked good for Utah. Someone in the media office e-mailed her afterward, attaching a name to her suspicions: Simpson’s Paradox. Utah’s students rank above the national average in just about every academic measure—until you look at them subgroup by subgroup. Not even the white students (the ones whose overrepresentation bring the average up) outperform their white peers elsewhere in the country.
Schencker wrote a really good story explaining this all. I only wish you could see the telling bar charts, which are only available to those who sign up to see the Tribune’s e-edition. (To all web editors, from a former graphics editor: Every chart needs to be online! They are not decoration!)
Speaking of statistical prowess: You don’t have to wait for a felicitous encounter with a national flack to acquire yours. Apply by December 21 to EWA’s statistics bootcamp, which will take place at the journalism school at Arizona State University on February 25-28. Come because participants always tell us they love this seminar, or come because of one particular statistic: The average high temperature in Phoenix in February is 70 degrees.
Recently, she asked that question during a teleconference about the NAEP math release, which again looked good for Utah. Someone in the media office e-mailed her afterward, attaching a name to her suspicions: Simpson’s Paradox. Utah’s students rank above the national average in just about every academic measure—until you look at them subgroup by subgroup. Not even the white students (the ones whose overrepresentation bring the average up) outperform their white peers elsewhere in the country.
Schencker wrote a really good story explaining this all. I only wish you could see the telling bar charts, which are only available to those who sign up to see the Tribune’s e-edition. (To all web editors, from a former graphics editor: Every chart needs to be online! They are not decoration!)
Speaking of statistical prowess: You don’t have to wait for a felicitous encounter with a national flack to acquire yours. Apply by December 21 to EWA’s statistics bootcamp, which will take place at the journalism school at Arizona State University on February 25-28. Come because participants always tell us they love this seminar, or come because of one particular statistic: The average high temperature in Phoenix in February is 70 degrees.
posted by Linda Perlstein at
4:51 PM
3 Comments:
It's great that Ms. Schencker started asking these questions.
However, this is level 1 to anyone with an even slightly wonkish streak who follows education. Any education reporter who DOESN'T get the correlation between test scores and demographics, and the resulting impact on the achievement of a school, a district or a whole school system, seriously needs to rethink his/her entire career choice, immediately.
Regarding the comment: Simpson's Paradox isn't just a matter of the relationship between test scores and demographics. Click on the link to Simpson's Paradox within Linda's post and you'll see some of the mathematics behind the paradox to which I was referring. This wasn't simply a matter of saying the state does well partly because of its high proportion of white students. This was a matter of mathematically understanding why our aggregate results seemed to contradict our disaggregated results when compared with national data. I hope that helps to clarify.
— Lisa Schencker
Yes, I know Simpson's Paradox is a broader principle, not that I could explain it myself. But it's also the principle that tripped up Washington Post columnist Richard Cohen years ago when he wrote an outraged column about how SAT scores are dropping (really because a far broader pool is taking the SAT than the college-bound elite as narrowly defined in the past, a concept he missed utterly).
Post a Comment
Considerate comments are welcome. Uncivil remarks will be deleted.
Subscribe to Post Comments [Atom]
Links to this post:
Create a Link
<< Home