TEA to Dump TPM?

I’ve been following the series at the Chronicle by Rick Casey on how the TEA is using TAKS scores in their campus rating system, something called the AEIS. Casey is looking at something that we educators have all had our attention turned toward lately, the Texas Projection Measure or TPM.

This is because raw scores notwithstanding, how the TEA uses the TPM, especially in borderline cases, greatly influences which of the four categories your campus will fall into.

Will the campus be rated as Exemplary, Recognized, Acceptable or Unacceptable. And how will the district be rated?

This is because, depending on how your campus is rated, it is then determined how many or how few hoops you must jump through as the school year progresses. The higher the rating, the fewer the hoops.

State Rep. Scott Hochberg (D-Houston) is also concerned with this measurement, as we read through Casey’s pieces. As he is chairman of a House education appropriations committee, he is a little worried about the jump in Exemplary campus ratings last year and wondered why that happened. As it happens, 73 of the 74 newly rated exemplary districts made that cut because of the TPM.

That is, actual student performance did not determine a district's exemplary status in 73 cases. Instead it was a statistical projection of the number of students who will eventually pass the TAKS test that did the trick.

And that is well and good if the TPM is an accurate predictor. The trouble is, that isn’t necessarily the case.

In his first piece, Casey revealed that a student could receive a passing TPM score after answering none of a TAKS test’s questions correctly. That is, a student who received a zero on a TAKS text could be scored as a student who would eventually pass the test.

Asked to defend this, Associate Commissioner in charge of the accountability system, Chris Cloudt claimed that the TPM was a “growth measure” of performance. One would assume that this was a comparison of performance from previous tests, and in doing so, one would assume wrongly. It is actually a formula applied to a student’s test score that looks back on “thousands of prior results.” It is, to use an industry term, a “fudge factor.”

The “fudge factor,” as it turns out, was devised by the national testing company Pearson Education. And when the “fudge factor” is applied to a theoretical student who scores a zero on a TAKS test, it transforms the zero to a passing score. Not based on the student’s own improvement but based on improvement of a pool of students.

It is, in short, fiction. Or better yet, it is, in short, wishful thinking.

The upside is that more and more campuses will be awarded Exemplary or even Recognized ratings, and teachers will be less beset by paperwork.

There is a downside, however. If the “fudge factor” skews all test scores upward, then schools that are actually in trouble with attaining an Acceptable rating will fly under the radar, and some necessary corrections and controls will not be applied that should be.

Politically, this could be disastrous. If public schools are viewed as having escaped poor ratings because of a state-sanctioned “fudge factor” that creates test scores that are more myth than fact, this only provides fodder to the rightwing that scream for school accountability by diverting public school funds to private, hopefully religious schools.

It gives those that raise a hue and cry for school vouchers the ammunition they need.

Ammunition that might penetrate Kevlar.

My guess, then, is that the TEA may be poised, as this article in the Austin American-Statesman suggests, to eliminate the TPM as a measure of student progress.

Unfortunately, old habits die hard as we see with what is being entertained by Education Commissioner Robert Scott, who lists three options: “the suspension of the measurement, continued use of the measurement for districts that choose to and modification of the measurement's calculation.”

Hochberg seems unimpressed:

“You don't make an invalid measure valid by doing less of it. I think we should start from scratch and develop a real measure of the progress students make in schools.”

Putting “lipstick on a pig” is how Hochberg characterized this.

Probably the only silver lining in this whole thing is that the state is about to dump TAKS as an accountability standard, replacing it with the State of Texas Assessments of Academic Readiness or . . . ahem . . . STAAR. That is, instead of 4 high-stakes tests in high school, students will take 12.

Silver lining? At least for awhile the statisticians won’t have enough data to do their magic with projections of student achievement.