With the dawn here in Texas, the most tested generation in history will sit down for the last series of Knowledge and Skills tests of the school year. It’s true you know. While the Great Generation met and mastered the tests of the Great Depression and World War II, the current generation of primary and secondary school children in Texas are, bar none, the most tested generation when you consider the kinds of tests where you “bubble” your answers inside little circles.
The only thing good about that is that at least when they go into their testing rooms tomorrow they pretty much know what is going to take place for the next few hours. I could go on and on about how tedious this is for everyone, and how disruptive it is of the curriculum calendar (all learning comes to a screeching halt), but I won’t. At least not anymore. No, I want to briefly examine what goes into test question development.
TAKS test questions are developed through a partnership between TEA, the Texas Education Agency, and Pearson Educational Measurement. Pearson develops the test questions and the TEA reviews them. Ten questions under review are usually administered to students along with their regular test questions, and student answers are analyzed. Questions that pass these rigors are then included in future tests as actual test questions that the students receive scores on. Test questions originate from individuals who contract with Pearson. I have done some of this with the Educational Testing Service so I have some experience with this. Typically one is asked to write several test questions, provide multiple choice responses, and then review several more questions developed by your peers. It’s interesting work.
Several times in the past, however, I have noticed that Pearson has allowed through a few bungled jobs. This is really through no fault of their own as they typically do not employ PhDs in their fields, unless you count PhDs in Education, and then I’ll bet that they are all over the place. The problem, as I see it, is that it is sometimes difficult to come up with a well-worded question for a concept that you are testing. Sometimes there are complexities that require a question’s words to be carefully chosen.
As an example, I cite a 5th grade science TAKS question that was asked a couple of years ago. A question for which there was no correct response.
Which two planets are closest to Earth?
A. Mercury and Saturn
B. Mars and Jupiter
C. Mercury and Venus
D. Venus and Mars
What is your answer? The question tested on the 5th grade science concept of the order of the planets in the solar system. Students in this case were supposed to reason that Earth lies in its orbital path between Mars and Venus, and therefore choice D was the correct choice. The trouble is, given the actual wording of the question, no response was correct. Had the question been reworded to “Which two planets have the closest orbital paths to that of Earth”, choice D would have been the correct one. The way it was worded, however, the two planets that are most often close to Earth are Mercury and Mars. Mercury, because, as opposed to Venus, it is most often on the same side of the Sun that Earth is, and Mars, because its orbital path brings it closer to Earth than Venus when they are on the same side of the Sun.
No 5th grader could know that, as it is an application of basic astrophysics. Only someone’s father or mother would ever be able to call the TEA out on that question. And they did, as a matter of fact. Texas, after all, has NASA within its borders. No, the question was poorly written and reviewers didn’t catch it. When it was tested with a previous year’s 5th grade class, it seemed to be a good question because 5th graders are not astrophysicists and they answered the question predictably with what they knew about the solar system.
So I think that given the specificity of questions on TAKS tests, especially the science tests, students will occasionally be visited with these kinds of erroneous questions. The problem isn’t having the wrong answers. It’s having correct questions.
The only thing good about that is that at least when they go into their testing rooms tomorrow they pretty much know what is going to take place for the next few hours. I could go on and on about how tedious this is for everyone, and how disruptive it is of the curriculum calendar (all learning comes to a screeching halt), but I won’t. At least not anymore. No, I want to briefly examine what goes into test question development.
TAKS test questions are developed through a partnership between TEA, the Texas Education Agency, and Pearson Educational Measurement. Pearson develops the test questions and the TEA reviews them. Ten questions under review are usually administered to students along with their regular test questions, and student answers are analyzed. Questions that pass these rigors are then included in future tests as actual test questions that the students receive scores on. Test questions originate from individuals who contract with Pearson. I have done some of this with the Educational Testing Service so I have some experience with this. Typically one is asked to write several test questions, provide multiple choice responses, and then review several more questions developed by your peers. It’s interesting work.
Several times in the past, however, I have noticed that Pearson has allowed through a few bungled jobs. This is really through no fault of their own as they typically do not employ PhDs in their fields, unless you count PhDs in Education, and then I’ll bet that they are all over the place. The problem, as I see it, is that it is sometimes difficult to come up with a well-worded question for a concept that you are testing. Sometimes there are complexities that require a question’s words to be carefully chosen.
As an example, I cite a 5th grade science TAKS question that was asked a couple of years ago. A question for which there was no correct response.
Which two planets are closest to Earth?
A. Mercury and Saturn
B. Mars and Jupiter
C. Mercury and Venus
D. Venus and Mars
What is your answer? The question tested on the 5th grade science concept of the order of the planets in the solar system. Students in this case were supposed to reason that Earth lies in its orbital path between Mars and Venus, and therefore choice D was the correct choice. The trouble is, given the actual wording of the question, no response was correct. Had the question been reworded to “Which two planets have the closest orbital paths to that of Earth”, choice D would have been the correct one. The way it was worded, however, the two planets that are most often close to Earth are Mercury and Mars. Mercury, because, as opposed to Venus, it is most often on the same side of the Sun that Earth is, and Mars, because its orbital path brings it closer to Earth than Venus when they are on the same side of the Sun.
No 5th grader could know that, as it is an application of basic astrophysics. Only someone’s father or mother would ever be able to call the TEA out on that question. And they did, as a matter of fact. Texas, after all, has NASA within its borders. No, the question was poorly written and reviewers didn’t catch it. When it was tested with a previous year’s 5th grade class, it seemed to be a good question because 5th graders are not astrophysicists and they answered the question predictably with what they knew about the solar system.
So I think that given the specificity of questions on TAKS tests, especially the science tests, students will occasionally be visited with these kinds of erroneous questions. The problem isn’t having the wrong answers. It’s having correct questions.
3 comments:
As a government teacher in Texas, I just chuckle each day I read your blog. Because my students are graduating, most do not take the TAKS, but I did run into this little song. I am sure you have seen it, but enjoy a refresher.
www.notonthetest.com
HILARIOUS!! No. I never saw it. Thanks!
"Don't think about thinking, it's not on the test."
I am almost in your position, C., most of my students are seniors and they are in heaven this week with a 11:00 wake-up call. My juniors and my one 10th grader, wherever he is tomorrow, are still at the mercy of "the test".
May the questions that they encounter on Thursday be well-thought out.
Now, watch out! Look! Up in the sky! It's a bird! It's a plane! It's a FLYING PIG!
I think this tests encourages a lot of kids to drop out knowing that they can't graduate without passing it. My grandson for example. only half credit to go and he dropped out. Couldn't pass English comp part of this test. He's in Iraq now. Am trying to get him to finish but he says what's the use. Please get rid of this test.
Post a Comment