Standardized Testing: The Good, The Bad, and the Ugly

It’s testing season again in Pennsylvania.  Over the next six weeks, I will administer or provide materials for testing more than 300 homeschooled students in PA.  Why? Because it is the law here.  Fortunately, it is only required for grades 3rd, 5th and 8th and there is nothing in the law that says students have to score at a certain level.  That isn’t sufficient to prevent homeschool moms from getting pretty anxious about this experience ( I typically ask the kids to pray for their moms before I start testing.  That usually lightens the tension in the room).  Truth be told, even knowing all I do about the unreliability of standardized achievement tests, I still was just as anxious and curious to see my own children’s scores during their grade school days.

So, what is the value, if any, of these tests?   First, let’s consider why we give students standardized achievement tests.  Testing as an industry grew up with the advent of mass education.  When I was a classroom teacher ( in another century), I saw more than ninety students a day.  My daughter who teaches high school math can see more than one hundred.   There is no way one teacher with this many students to assess is able by observation to know where each child’s skill level is in key areas.  Hence, the standardized test.  This gives teachers a brief snapshot in time of a child’s performance.  The benefit may be that it highlights kids who are struggling or, conversely, being under-challenged, that aren’t on the teacher’s radar.  ( She’s too busy dealing with the behavior problems in the room.)  However, because it is such a short measurement of a child’s performance ( no specific skill test is longer than 40 minutes) it isn’t reliable for measuring an individual child’s performance.  If you truly want a reliable assessment of an individual child’s skill levels, you need to refer that kid for full testing which will cost hundreds of dollars and take several days to complete.  What then is the intended purpose of standardized achievement tests; such as the ITBS ( Iowas), SAT ( Stanford) or CAT ( California)?   Group assessment.  These are only good instruments for generalizing whether or not the class is making progress or failing to understand key topics.  Then teachers can adjust their lesson plans accordingly.

Unfortunately, in several states these achievement tests are used erroneously as a method of assessing individual homeschooled students.  My intention in this series of blog posts is to better inform you about how to use these tests ( if you choose to use them or are required) to help you with your homeschool program. 

1.        First, you need to know your observations of your child’s skills and understanding in your home on a daily basis is a much more valid and reliable source of data about your child’s progress than a standardized test.  At best, the test can confirm your observations or perhaps suggest an area to pay closer attention to before drawing a conclusion.

• We say a method of measuring data is valid if it actually measures the construct of interest; for example, reading comprehension.  So, (A) is a test which gives kids short reading passages that were written for no other purpose than to measure a child’s reading comprehension really measuring reading comprehension or (B)  is listening to your child read a picture book aloud and then discussing what happens in the story together a better test of reading comprehension?  Which one is more closely related to reading in the real world?  ( If you circled B, good job, you are correct!) 
• A method of measuring data is reliable if we are confident the score on a particular test is an accurate representation of that child’s true skill or comprehension.  Perhaps the room was too hot during testing, perhaps the child was sick, perhaps the question was worded in a confusing way, or the problem was presented differently than the child is use to seeing.  Where do you think it is more likely you will get a reliable measurement of your child’s true skill or comprehension?  A score from a two-hour test or the cumulative average score your child has from daily exercises in his math textbook?
  In the world of educational assessment, any data that comes from performance across time is significantly more reliable than data from a moment in time.  The problem in mass education, though, is longitudinal ( across time) studies are expensive and difficult to schedule.  But what about homeschooling?  Can you gather data about a child’s progress across time with ease?  And better yet, this child has the same teacher across time as well, so you are at an advantage from a classroom teacher who makes conclusions about children she has for only one year.

2.       Second, using a skill or demonstrating understanding in context is a far more reliable measure of achievement than taking a short test in an unfamiliar environment.  Further, the tasks children are asked to perform during testing are not in a meaningful context either.  Test designers try to come up with a task that approximates the real world, but time is of the essence.   So take those reading comprehension passages we are probably all familiar with again.  Is that how your brain works while reading?  Do you read a section of a gripping novel and then pause and ask yourself question to make sure you’ve got it?  What do you do as soon as you are confused?  You go back and re-read until comprehension is achieved.  And what kind of questions do you ask yourself anyway during reading?  Any similarity to those questions on reading comprehension tests? I didn’t think so. Finally, how often do you time yourself while reading? What do you think throwing that caveat into the pot does to reliability?

Does this resonate with you?  The best scientific data you have for assessing your child’s achievement is what you see right in front of you day in and day out.  In addition, while you are watching your child perform a task, you and your child can interact so you have access to what your child is thinking as well as what your child is doing.  That is a potent combination of information, and if you keep following this blog I’ll try to help you make sense of the data you collect in this way. 

In the meantime, the next post will deal with how to prepare your child for achievement testing so the report you do get afterwards is as helpful as possible.