SAT as Poor Indicator of School & Teacher Quality

Friday, December 12th, 2014
Comments Off on SAT as Poor Indicator of School & Teacher Quality

The New Jersey Department of Education recently released the “menu” of standardized tests that high school students can use to satisfy the archaic exit-exam policy that governs New Jersey high school graduation. The menu includes things like PARCC tests for Algebra I, II, & Geometry, along with PARCC exams in Language Arts I, II, & III. Students can also use results from PSAT, SAT, and ASVAB, among other types of commercially prepared standardized tests. Although this seems progressive on its face, will the menu really equate to a equitable process for New Jersey’s poorest children?

Because the bureaucrats at the New Jersey Department of Education have placed such a premium on PSAT and SAT by using student participation and results from those exams as factors to determine if school personnel were successful in promoting college and career readiness, it is likely that many school district leaders will encourage more students to try to use the SAT as their ticket to graduation. But what will the results from the SAT tell stakeholders about college and career readiness at a specific high school?

The answers to both questions can be found in publicaly available reports and data. Results from reports published by the College Board (See Table 11 found here) and those found in academic articles (examples here & here) on the subject suggest that, once again, SAT scores will tell the public more about the neighborhood in which a student’s school is located than that student’s college and career readiness, preparedness to graduate high school, or the quality of his or her high school. In short, the continued policy reliance on commercially prepared standardized tests to determine if a student can graduate high school will harm children from poverty.

CHARTING THE DATA

Below are charts specific to New Jersey, but are representative of the strong connection between the economic conditions of the community in which a child’s school is located and his or her performance on the SAT Math and Verbal sections.

The charts plot men SAT scores for each of the over 330 high schools in New Jersey in relation to the percentage of students in each school who are eligible for Free lunch (not free or reduced – only FREE lunch). Free lunch percentage (FLpct) is a more accurate indicator of poverty than Reduced lunch eligibility. The income thresholds for a family of four is $ 31,005 for Free lunch and $44,123 for Reduced lunch.

I color-coded the high schools according to the their District Factor Group category (DFG). In short, New Jersey used to categorize school districts based on wealth characteristics of the communities they served and the ability of those communities to fund public education. Districts were categorized alphabetically A-J with A districts located in the poorest communities and least able to fund their schools whereas J districts are located in the wealthiest communities with the greatest ability to fund their public schools. R districts are Charter high schools and V are vocational high schools. I highlighted the A districts and J districts so one can see the gap that exists not only in SAT achievement, but percentage of Free lunch eligibility.

The scale on the left side of each plot denotes SAT scores and the bottom scale on each plot denotes percentage of students eligible for Free lunch (FLpct), in decimal form. For example, a school plotted at .20 FLpct has 20% of its students eligible for Free lunch whereas a school plotted at .60 FLpct has 60% of its children on Free lunch. The correlation between Free lunch and SAT score is labeled to the right of each plot in decimal form. I drew a line of best fit through each graph to denote the slope.

Readers will notice that the J, I, & H districts have the lowest percentages of students eligible for Free lunch and the highest SAT scores on average, whereas the A, B, & CD districts have the highest percentage of students eligible for Free lunch and lower mean SAT scores.

The selective vocational high schools, those that skim the best students from a county’s high schools, also have some of the lowest percentages of students eligible for Free lunch and high mean SAT scores. The more democratic vocational schools, those that accept all students regardless of Free lunch status or prior achievement score in the average ranges on both sections of the SAT.

SATMAth1

The relationship between money and SAT Math scores is strong in New Jersey (R2=.659), as it is around the country.

Below, a similar relationship is visible between money and SAT Verbal scores (R2=.698).

Document2

The policy of allowing SAT results to satisfy a graduation requirement tells us nothing about a student’s overall achievement in high school. In fact, high school grade point average (GPA) is as good a predictor, and often, better predictor of first-year college success AND four-year college persistence than the SAT (See here and here and  here and here and here). Furthermore, high school GPA provides a more well-rounded account of a student than a score on a corporately produced standardized test. A strong argument can be made to use multiple classroom-based measures to judge high school student achievement and college readiness. Almost 1,000 colleges and universities no longer require standardized test scores or make them optional. Seems once again that NJDOE policy is caught in a time warp and not keeping up with the times.

BAD MENU

The new graduation menu currently being marketed by NJDOE bureaucrats will do nothing to improve the education of New Jersey’s poorest students. The policy will simply result in more examples of how New Jersey’s growing opportunity and economic gaps are hurting the students who need the most.

The results from the SAT will no better describe the quality of a high school on the New Jersey School Performance Reports than did the HSPA exit exam or any other standardized test.

The results WILL give the profiteering corporate-education complex and their legislative and bureaucratic supporters more ammunition to hurl at our urban centers and children as they continues to fleece the taxpayer with promises of closing the achievement gap.

In other words, nothing new to see here. Lots of NJDOE fluff and rhetoric about progressive policy improvements but no substance.

To be continued….

 

 

Christopher H. Tienken ©2012 Copyright. All Rights Reserved.