Improving Achievement... Is It the Tests or the Teaching?

  • Testing improving test performance
  • 11 March, 2018
  • 9 Comments

Teacher question:

We are trying to figure out how to help our grade 2 students. Currently, we see a big gap in the percentage of students who are meeting standards in Grade K and 1 compared to the same student results in Grade 2 (more than 90% in Grades K and 1 but only 55% in grade 2) In our assessment. We allow students to have questions read in Grade K but not at the end of grade 1. Why might we see this trend consistently across cohorts? Would you be able to recommend 1 or 2 strategies that we could implement as a district (~74,000 students) to influence these data? 

Shanahan response:

Your question seems to be based on the premise that the problem is in the testing. And, that very well may be.

Scaling texts, questions, and tasks so that a second-grade test is the equivalent of a first-grade test one is a lot of work—work that few school districts usually do. It involves calculating local norms and scaling items across years so that cut scores can be set. That gets so involved that I’m curious about your purpose in doing this. If you are just trying to make sure that grading is consistent or that decisions about retention or placements are equivalent that is what it would.

The more usual purpose is that a district is trying to identify kids who are struggling so that they are on track to do well on an accountability test in Grade 3 and above. In those cases, it is important to follow kids across grade levels to determine what levels of performance are needed in the lower grades to assure later success. If you find, for instance, that 90% of the kids who are meeting standards in grade 3, had scored 75% or higher on the first-grade assessment, then that may be a good cut score to set.  

I would not at all be surprised to find out that your tests aren’t equivalent. That might be fixable by changing an administrative procedure (like whether the teacher will read directions or questions aloud or whether kids are expected to do that themselves). Test passages are more likely the issue if it is a comprehension test—how well do text selections represent different increments of performance… are the second-grade texts as much harder than the first-grade texts as the first-grade texts are above the kindergarten texts?

You definitely could spend a lot of time and resources calculating norms, adjusting tasks, and conducting cut-score determination studies to make sure that the tests are telling you what you want to know.

But you might be wrong about all of that… maybe it isn’t actually a testing problem. Maybe your tests are more equivalent than you think.

Maybe it’s the teaching.

Maybe your second-graders really aren’t doing as well as the younger kids. A lot of districts make a big deal out of giving kids a good start (e.g., beginning reading programs for phonemic awareness and phonics, early interventions like Reading Recovery, efforts to place the best reading teachers in those grades). Second-grade, not so much.

It may be wise to task (and fund) someone in your district with developing better norms and cut scores over the next couple of years. But even if you go that way, I’d strongly encourage you to hedge your bets. While resolve your testing problem—making sure that your assessment plan provides what you really want, whatever that might be—it would be sensible to try to improve achievement. Maybe 45% of your second-graders really aren’t reading well enough.

If you want to do that there are only three things you can do to improve students’ reading achievement:

1.  Increase the amount of literacy instruction and experience that your children receive.

How much reading and writing instruction do your second-graders receive?

Many schools budget a 90-minute block for reading instruction in grade 2. Is that enough? Probably not, given your results. Not all schools necessarily have to spend the same amount of time on reading instruction. Some kids are more challenged than others. I would definitely spend more than this if I were teaching kids with disadvantaged backgrounds, or second-language learners, or kids with certain disabilities. I’d spend more time because it takes them more time to get to the levels they will need.

Even when schools schedule enough time for reading instruction, teachers may not use that time as well as they could. The teacher might be reducing this instructional time by using it to read to kids or to give kids a period of time to read on their own or by just by inefficient teaching (like teaching the same lesson over and over to small groups of children).

I’m not saying that teachers shouldn’t read to kids—I would—but it shouldn’t be considered to be “reading instruction,” since we have no research that it improves reading performance for kids at this grade level. I’m not a big fan of having kids read on their own (without close involvement of teachers whose guidance can increase the amount of learning), but I wouldn’t ban it; I just wouldn’t count it as part of the 90 minutes (same reasoning here—the learning payoffs are just too small to go this way).

One of the things I don’t like about the 90-minute reading block is that it is not a 120-minute reading and writing commitment. Kids can learn a lot about reading from writing instruction and omitting writing can undermine your reading instruction efforts.

In my schools, I’ve argued for 120-180 minutes per day of reading and writing instruction. However much time you agree upon for that, I’d encourage your teachers to try to expand beyond its boundaries, by having the kids reading as part of their science and social studies classes (and by that I don't mean round-robin reading).

2. Make sure instruction focuses on teaching those things that improve children’s reading achievement.

Research has identified a number of abilities that can be taught to second-graders that can improve the children’s reading ability.

I think you need to ask yourselves the following questions:

            Are your teachers teaching decoding?

            Are your teachers teaching oral reading fluency?

            Are your teachers teaching reading comprehension?

            Are your teachers teaching vocabulary?

            Are your teachers teaching writing?

Second-graders should receive substantial amounts of instruction in each of these areas. Again, I have usually argued for devoting roughly equal amounts of instruction for decoding, oral reading fluency, comprehension/vocabulary, and writing. If that isn’t happening—if your teachers are minimalizing some of these because of their philosophies or beliefs, or because the district hasn’t adequately supported them—then I would focus district efforts on correcting these oversights.

Often teachers and principals will downplay one or another of these because they don’t match well with the tests that are to be given. If your reading test doesn’t include decoding, then perhaps they de-emphasize such teaching. The same thing happens with oral reading fluency; “if second-graders are going to be tested on silent reading then why teach oral reading?” the thinking goes. And, that can be the reason why writing gets elbowed out the picture, too. (All of those choices may serve to lower reading achievement.)

3. Make sure the instructional quality is high—that means efficient and effective.

There are many general points that can be made about quality. For instance, studies have shown how critical it is that lesson have clear purposes—purposes that both the children and the teachers are conscious of (that way kids can try to learn, and teachers can be more diagnostic, noticing whether the purposes are being accomplished). Another general example would emphasize the importance of being motivational or encouraging.

There are also quality issues that are specific to each of the instructional components noted above. Teachers may claim, for example, that they are teaching oral reading fluency, but what they actually mean is that they are engaging kids in round robin reading (taking turns reading aloud short segments of text). But that approach usually means children aren’t working with texts that are sufficiently difficult, that no one gets much reading practice, that there is no rereading to try to improve performance.

Similar points could be made for all of the areas noted above. How well are they teaching reading comprehension or decoding or vocabulary?

A careful review of current second-grade instructional practices might suggest actions that the second-grade teachers can take. However, there are things that the schools should be doing as well. For example, is student learning being monitored in the areas noted above, and if so, is extra instruction available to the kids who are not performing well? What are the schools doing to involve parents in their kids’ progress (getting them to listen to their kids read can be a big help, for example)?

I’m certainly not arguing against improving the accuracy of your testing scheme, but I would take this opportunity to give all of your students the best chance to become successful readers. That is more likely to come from increasing the amount of teaching, focusing that teaching on key areas of literacy development, and improving on the quality of that teaching.

Good luck.

Comments

See what others have to say about this topic.

Natasha Bradley
Mar 12, 2018 07:57 AM

What if our school district has implemented Core Knowledge ELA which is essentially teachers reading aloud to increase students’ background knowledge and vocabulary? Our second graders aren’t reading the material; do you suggest we turn the read-aloud into a student reader? A whole group lesson can take up to 30 minutes of a 90 minute block.

Amity Sowash
Mar 12, 2018 02:05 PM

ReadyGen material is 2-3x’s above grade level standards. This is great to create rigor in our classrooms, but not so great for allow 2nd graders to read independently. I’m teaching grade level standards with text that they can not read independently. If I ask them to find the answer in the text this presents a challenge if they can not read it. What do we use to allow them on grade level practice? I use the little readers, readworks and other resources, but honestly even some of those are above their level. Reading to understand is tough and these kids needs as much opportunity to do this as possible. I also feel students need more experiences working and testing on computers. I teach all sorts of testing strategies for paper pencil tests, but the quarterly is all on the computer. It makes it very difficult for the students to slash the trash, or underline where the got their answer. So far we have only had issues if we try to highlight the text.

Timothy Shanahan
Mar 12, 2018 02:38 PM

Natasha- no one has found that reading to second graders improves reading achievement—so that would be a bad idea for a central activity in your reading instruction.

MARGARET CONNOLLY
Mar 12, 2018 02:48 PM

In my experience as a classroom teacher, coach and now interventionist, I think teachers feel that they are "teaching" these things, but they're really highlighing these things and not taking each from a thoughtfully planned lesson throught the gradual release with explicit teaching and guiding in a small group. We, in our district keep noting a "lack of transfer" as children read and write, and our data at grade two has been steadilty growing the number of students deemed "well below grade level" on multiple measures. How do we help administrators and teachers understand that if we don't have lessons that change a student's reading behavior that day, we really aren't "teaching" a student how to read and write. We're hoping they learn to read and write.

Jamie
Mar 14, 2018 04:21 PM

One of the questions in your article is: “Are your teachers teaching oral reading fluency?”.

What would this type of instruction look like? Are you simply meaning providing opportunities for students to read out loud instead of silently?

Timothy Shanahan
Mar 15, 2018 05:09 PM

Amity--

I haven't checked but I doubt that ReadyGen's materials are that difficult. I think that program was developed to match the requirements of the Common Core State Standards and that means the text would be higher than traditional second grade materials (but more like a half grade level or so). Of course, measurement of text difficulty is not perfect so they could be overshooting your kids more than intended. In any event, what is missing, it seems to me is instruction aimed at helping the kids to make sense of these texts. You need to make sure that by the end of a lesson the students are able to read those texts well (not at the beginning, but when you finish).

In terms of kids reading independently, why not let them read independently? Let them select whatever they want to read on their own.

Timothy Shanahan
Mar 15, 2018 05:13 PM

Margaret--

For too long, teachers have "guided" or "directed" kids' reading by pre-teaching vocabulary and providing some background information relevant to the texts and then asking kids questions about what they have read. The idea was that the texts would get gradually harder and that kids would do well--reading all of the texts relatively easily and with high comprehension. Now we are asking kids to take on much harder texts, but without any more guidance than before. As you say, the instruction tends to highlight an issue (e.g., main idea, vocabulary), but without much real teaching or monitoring to see if the skill is mastered. There is a real need for well-planned instruction and repetition under these circumstances.

tim

Timothy Shanahan
Mar 16, 2018 02:42 AM

Amity—
Yes, kids need to read aloud... the texts they read from should be texts that they are somewhat disfluent with (if they can already read them well then they won’t learn from it)... they should get feedback and support from someone who will try to help when needed, etc. (teacher, parent, tutor, peer, computer)... there needs to be rereading.... and there needs to be enough of this kind of practice that kids can get good at it.

Tim

Kate
Mar 16, 2018 11:06 PM

Amity - I am a new teacher a school that uses ReadyGen and we are having the same issue. I've heard the newer version of the curriculum has switched out some of the texts for easier ones, but my school is still using the original 2013 version - which means that 2nd graders start the year trying to read Charlotte's Web (680L), 3rd graders are supposed to be reading Behind Rebel Lines (830L, mature subject matter geared toward 6-8th graders), 4th graders start the year reading The Tarantula Scientist (890L, designed for 6-8th graders). The lesson plans handed to me by admin and the grade team leader have the students read 2 to 3 paragraphs a day (preteach vocab, teacher reads one paragraph, teaches a skill, and the students reread and try to make sense of another paragraph or two) and then the class skips a dozen or more of pages of the book because the kids can't read it and there's no time. It's bizarre--students end up reading only a few sentences a day as part of this curriculum. 80% of our 3rd to 5th graders are reading a year or more behind grade level. I feel like our school, with the help of this curriculum, is making a disastrous mess out of "text complexity."

What Are your thoughts?

Leave me a comment and I would like to have a discussion with you!

Comment *
Name*
Email*
Website
Comments

Improving Achievement... Is It the Tests or the Teaching?

9 comments

One of the world’s premier literacy educators.

He studies reading and writing across all ages and abilities. Feel free to contact him.

Whoops, looks like something went wrong.

1/1 ErrorException in Filesystem.php line 81: file_put_contents(/var/www/html/storage/framework/sessions/02be8b9ac885f722892f4b217a0588be52271351): failed to open stream: No space left on device

  1. in Filesystem.php line 81
  2. at HandleExceptions->handleError('2', 'file_put_contents(/var/www/html/storage/framework/sessions/02be8b9ac885f722892f4b217a0588be52271351): failed to open stream: No space left on device', '/var/www/html/vendor/laravel/framework/src/Illuminate/Filesystem/Filesystem.php', '81', array('path' => '/var/www/html/storage/framework/sessions/02be8b9ac885f722892f4b217a0588be52271351', 'contents' => 'a:4:{s:6:"_token";s:40:"hHBkqm8vXrnZl9QxciWfTp7P3rGq5Ll3ukCJa1CU";s:9:"_previous";a:1:{s:3:"url";s:79:"http://35.162.70.160/blog/improving-achievement-is-it-the-tests-or-the-teaching";}s:9:"_sf2_meta";a:3:{s:1:"u";i:1537838338;s:1:"c";i:1537838338;s:1:"l";s:1:"0";}s:5:"flash";a:2:{s:3:"old";a:0:{}s:3:"new";a:0:{}}}', 'lock' => true))
  3. at file_put_contents('/var/www/html/storage/framework/sessions/02be8b9ac885f722892f4b217a0588be52271351', 'a:4:{s:6:"_token";s:40:"hHBkqm8vXrnZl9QxciWfTp7P3rGq5Ll3ukCJa1CU";s:9:"_previous";a:1:{s:3:"url";s:79:"http://35.162.70.160/blog/improving-achievement-is-it-the-tests-or-the-teaching";}s:9:"_sf2_meta";a:3:{s:1:"u";i:1537838338;s:1:"c";i:1537838338;s:1:"l";s:1:"0";}s:5:"flash";a:2:{s:3:"old";a:0:{}s:3:"new";a:0:{}}}', '2') in Filesystem.php line 81
  4. at Filesystem->put('/var/www/html/storage/framework/sessions/02be8b9ac885f722892f4b217a0588be52271351', 'a:4:{s:6:"_token";s:40:"hHBkqm8vXrnZl9QxciWfTp7P3rGq5Ll3ukCJa1CU";s:9:"_previous";a:1:{s:3:"url";s:79:"http://35.162.70.160/blog/improving-achievement-is-it-the-tests-or-the-teaching";}s:9:"_sf2_meta";a:3:{s:1:"u";i:1537838338;s:1:"c";i:1537838338;s:1:"l";s:1:"0";}s:5:"flash";a:2:{s:3:"old";a:0:{}s:3:"new";a:0:{}}}', true) in FileSessionHandler.php line 71
  5. at FileSessionHandler->write('02be8b9ac885f722892f4b217a0588be52271351', 'a:4:{s:6:"_token";s:40:"hHBkqm8vXrnZl9QxciWfTp7P3rGq5Ll3ukCJa1CU";s:9:"_previous";a:1:{s:3:"url";s:79:"http://35.162.70.160/blog/improving-achievement-is-it-the-tests-or-the-teaching";}s:9:"_sf2_meta";a:3:{s:1:"u";i:1537838338;s:1:"c";i:1537838338;s:1:"l";s:1:"0";}s:5:"flash";a:2:{s:3:"old";a:0:{}s:3:"new";a:0:{}}}') in Store.php line 262
  6. at Store->save() in StartSession.php line 88
  7. at StartSession->terminate(object(Request), object(Response)) in Kernel.php line 145
  8. at Kernel->terminate(object(Request), object(Response)) in index.php line 58