My friend the chair of Stats at UCD told me she never used them for promotion cases or merit raises.
Why? It's perfectly obvious: they are statistically meaningless.
I've heard the argument that “Well, but that's the way we've always done it.” The argument from precedent is never logical. Imagine the CIA defending waterboarding this way.
A poor teacher in the Houston Independent School District is undergoing this right now. So if anyone's reading, let it be known that it's become quite clear here at Rice that they are statistically very dubious. We're switching to what should have been done in the first place. Sure the students can evaluate, as if they were customers responding to a survey. But what needs to happen is for another teacher to go in that classroom and see the teaching.