My Christmas gift to you: a value-added story idea.
If you go into a typical fourth-grade class—especially a high-poverty one—on test day in March, you might find a kid who arrived at the school in January, one who arrived in February, and one who arrived three days before. You’ll find several kids who receive most of their reading instruction from a pullout teacher, and others who do so in math. There will be students who spend large part of their day in a special ed room, and some in ESL. The class might have spent a large chunk of the fall with a student teacher. A couple of kids might have been swapped from the class across the hall because they had trouble getting along with their classmates.
So how much credit, or blame, for these kids’ scores on the test should be attributed to the classroom teacher? This, in a nutshell, is the “teacher of record” problem, and chances are HUGE that your state or district has not solved it, even if it is about to make (or already makes) high-stakes decisions about teachers based on those scores.
When I went to a Vanderbilt conference on performance incentives this fall, TOR issues were the elephant in the room. In presentation after presentation, they were quietly acknowledged and just as easily dismissed. “We have not quite worked that out yet, but we’re confident in our data” was how one district official put it.
According to those in the know, it has become clear as states try to make good on their Race to the Top promises that they have no solutions to the TOR problem, if they have even considered it. If you are covering a system that is, or will be, basing any sort of meaningful decisions on value-added data, you should be writing about this. It’s a policy story that can be easily, and compellingly, illustrated with the fruits of one day’s reporting in a classroom, and given that these policies are still being shaped, you should not wait to pursue this.
posted by Linda Perlstein at
2:47 PM
11 Comments:
Hi Linda,
Actually, several school organizations have addressed the Teacher of Record issue. Tennessee has a statewide system that aligns the right teachers to students, Battelle for Kids provides a national software system called BFK LINK that provides the alignment in districts all over the country including Houston Independent, and I developed a system in Eagle County, CO that did the same thing.
It's a technical problem and states/districts who aren't paying attention to it can misalign which teachers work with which kids if they are just using their student information systems.
However this is by no means an insurmountable technical issue. By having systems that both principals and teachers access and correct you can appropriately align kids to teachers and improve the validity of the value added result.
Thanks much for raising the issue - it's an important technical component that those considering value added measures need to have in place if they want reasonably accurate estimates of teacher effectiveness.
Jason Glass
Columbus, OH
Jason, you may be able to technically match attendance to a teacher of record and work out some ratios or proportions, but you cannot assert with any confidence whatsoever that those numbers represent any facts about the students' educational experience or the teachers' effects.
This issue is one of many, many examples of how value-added models should be used for limited purposes at the broadest levels only. On the school and teacher level, the models simply cannot identify or control for relevant factors. I used my own teaching context as an example and backed up my argument with study after study in this blog post, (which was later re-posted in The Answer Sheet):
http://accomplishedcaliforniateachers.wordpress.com/2010/09/02/open-letter-ca-public-officials/
Hi David,
All measures are imperfect and value added is no exception to that. I can assert that the data captured through teacher of record systems can follow agreed upon protocols, allow both teachers and principals to inform and correct the information, and allow for a better estimation of value added results.
Let's not let perfect be the enemy of good. Value added measures have the capacity to inform and improve instruction and support better outcomes for kids.
The assertions advanced here that would say that there is some secret about teacher to student alignment and that no one is paying attention correcting that alignment is just incorrect.
Just saying it's important for everyone to get their facts straight.
Thanks much for reading and responding to my post. I appreciate the exchange.
Jason Glass
Reporters and editors are not qualified to determine value added measures have the capacity to inform and improve instruction and support better outcomes for kids. It is beyond their scope. Making that determination, as the Los Angeles Times staff did, puts the press in the position of becoming judge and jury, not just messenger. That's inappropriate and out of bounds for the press.
Jason Glass has professional involvement in education reform and has stated his position on the effectiveness in value added in assessing teachers. That's one side of a hotly debated issue. Well-informed and articulate sources for the opposing viewpoint include Richard Rothstein, Diane Ravitch, and UC-Berkeley statisticians/professors Mark Wilson and Sophia Rabe-Hesketh, among many others.
Hi Caroline,
What's really unfortunate about the debate over education now is that we try and paint each other into the 'education reform' versus 'status quo' camps. Makes for an interesting narrative, "good versus evil," "us versus them" and all that ... but I'm not sure this polarization does much that's actually good for kids.
Before you try and discredit me, why don't we try and address the point I've raised in reaction to this article. That is, Linda was stating that if you don't align the correct teachers to students, then the ability of value added to provide inference at the teacher level is limited. I completely agree. However, it's important to also note that technical solutions do exist for this issue and are being used in districts all over the country. That part Linda omitted from her analysis. I appreciate Linda's writing and point of view - I'm sure it was an innocent oversight on her part.
While I'm happy to engage in a deeper discussion about what value added is and what it's uses can be, this story is really about the teacher to student alignment component. As I've indicated here, it's an important technical problem ... for which technical solutions exist.
Thanks much for contributing to the conversation. Again, I appreciate the exchange.
Jason Glass
Columbus, OH
I didn't try to discredit you. I identified you as a spokesperson for one side of the debate over whether value-added measures are an accurate, appropriate, effective and fair way to gauge teacher quality.
My apologies for the mis-characterization Caroline. I will certainly back your assertion that there are several opinions out there about the utility of value added.
Unfortunately, most of them are in the extremes on both sides.
I'm wondering about yesterday's article on the proposal in Boston to use VAMs to supplement principals' judgement. I don't know the details of the plan but it might address a crucial issue. It is ine thing to use flawed data to supplement or complement , but it is unacceptable for test score growth to DRIVE evaluations. Once indicted by numbers, too many evaluators will follow their lead. (who would want to be evaluated by Jason in such a situation?)
They propose that teachers growth targets be set by comparing students with smlar performance. That's far different than comparing to same demographics. I doubt the true-believers in VAMs would accept that, but who cares?
It would also help principals from being crucified by crude data-driven accountability.
Perhaps we should quickly settle on that sort of compromise and call the whole educational civil war off. I'd prefer The Grand Bargain, or New Haven plan, but that's not the point.
The point is that the schemes promoted by the accountability hawks would produce an exodus of teaching talent from high-challenge neighborhood schools to schools where it is easier to raise scores, and burn out people as in the book Tested.
Hi John,
While you depart from the primary issue discussed in this piece (the proper alignment of students to teachers for value added measures) you do bring up a common misperception on value added. Frequently those opposed to or ignorant of the measure like to paint it in the extreme to say that VAM would be used as a singular data point used to infer the quality of a teacher. I can't think of anyone advancing this nonsense except those opposed to the seemingly radical notion that student results should count for anything in determining teacher quality.
Besides the obvious problem that it simply can't be used for all teachers due to the availability of assessment data in untested subjects and grades, all the VAM advocates I know argue for it to be considered as one measure among many used to reach a determination on teacher effectiveness.
A common tactic used in a debate is to paint the opposition in the extreme so as to more easily discredit them. It seems that is what's going on in your post.
In my humble opinion, I think our country is better off paying attention to teacher quality and using several data points to inform that decision rather that assuming everyone is the same and providing no real information for improvement. Or do I overstate your position ...?
Jason Glass
Concord, NH
Jason,
Can VAM explain what a teacher did (TOR aside)to elicit the scores of his/her students, ie. teach to the test, eliminate untested subjects, drill and kill, bribery, or just strong teaching?
Because if the TOR issue gets fixed, it still doesn't answer these questions for me. Or am I missing something?
Martha Infante
Los Angeles, CA
Hi Martha,
In my humble opinion, you are missing something. VAM is a statistical model - it is not the cause of anything. Coupled with high quality assessments and an accurate teacher to student alignment process, its one measure of student progress and a teacher's effectiveness within an educational system.
Taking any of the steps you mention to the extreme is foolish. Equally foolish is believing all teachers are of equal quality and that we should not be concerned with improving those failing our students while expanding the influence of the best teacher. Also foolish is not considering the ability to read or do math as important skills, or throwing up our hands saying that quality teaching and learning are phenomena beyond measurement and replication.
Thanks much for the response.
Jason Glass
Somewhere in Indiana...
Post a Comment
Considerate comments are welcome. Uncivil remarks will be deleted. Anonymous comments -- including those unaccompanied by the author's first and last name -- are not permitted.
Subscribe to Post Comments [Atom]
Links to this post:
Create a Link
<< Home