xThe Cranky Taxpayer

The Cranky Taxpayer

Fun3


Back | Home | Up | Next
Disability Abuse | SOL v Poverty | SAT Disaster | Fun1 | Fun2 | Fun3 | Do SOLs Work? | Leadership | Neanderthal Security


VDOE admits that the accreditation scores are "adjusted."  In fact, the scores are so tortured as to be meaningless.

 

It's Hard to Flunk

Note: These data embrace a spectrum of tests but VDOE boils the scores down, generally, to a single number for each subject at each school.  See 8VAC20-131-300.C.1.d.  Thus, the elementary English score is produced by averaging scores of the reading tests in grades 3 to 5.  So please be aware that the terms "test" and "score" below probably refer to results of several tests, generally at several grade levels.

The score distributions tell the story.  The math distribution is the less complicated so we'll start there.

The green diamonds are the raw pass rates, by school, on the 2013 math tests.  The mean of this slightly skewed dataset is 73.2.  The pink squares are the "adjusted" pass rates.  Note the dramatic improvement of the schools in the 60-70 percent range.

The passing score is 70.  Recall that the State both designs the tests and sets the passing score so the location of the discontinuity hardly comes as a surprise.

For English, the passing score is 75.  There was an initial problem with the database.  When I returned to the data in 2014, rather than wrestle with the entire database, I just ran Richmond.  See below.

 

How Big the Boost?

Parsing out the Richmond data from the raw pass rates and the "adjusted" pass rates for the math tests discloses the following:

School Pass Rate Rounded Adjusted Boost
Albert Hill Middle 57.5% 58 70 12
Amelia Street Special Education 55.2% 55 58 3
Armstrong High  35.2% 35 43 8
Bellevue Elementary 74.7% 75 76 1
Binford Middle 35.7% 36 37 1
Blackwell Elementary 45.4% 45 51 6
Broad Rock Elementary 76.5% 76 76 0
Chimborazo Elementary 40.9% 41 44 3
E.S.H. Greene Elementary 62.5% 63 72 9
Elizabeth D. Redd Elementary 45.0% 45 51 6
Elkhardt Middle 32.2% 32 39 7
Fairfield Court Elementary 69.0% 69 72 3
Franklin Military Academy 63.8% 64 68 4
Fred D. Thompson Middle 41.0% 41 47 6
G.H. Reid Elementary 40.5% 41 48 7
George Mason Elementary 43.4% 43 45 2
George W. Carver Elementary 70.8% 71 72 1
George Wythe High 53.2% 53 62 9
Ginter Park Elementary 36.5% 37 45 8
Henderson Middle 29.1% 29 35 6
Huguenot High 42.5% 43 53 10
J.B. Fisher Elementary 55.2% 55 59 4
J.E.B. Stuart Elementary 49.6% 50 48 -2
J.L. Francis Elementary 47.3% 47 50 3
John B. Cary Elementary 46.7% 47 72 25
John Marshall High 70.3% 70 77 7
Linwood Holton Elementary 60.6% 61 71 10
Lucille M. Brown Middle 56.9% 57 59 2
Martin Luther King Jr. Middle  25.2% 25 31 6
Mary Munford Elementary 76.6% 77 79 2
Miles Jones Elementary 45.6% 46 48 2
Oak Grove/Bellemeade Elementary 51.7% 52 53 1
Open High 82.2% 82 82 0
Overby-Sheppard Elementary 63.5% 63 64 1
Patrick Henry School Of Science And Arts 40.3% 40 44 4
Richmond Alternative  25.9% 26 23 -3
Richmond Community High 91.3% 91 91 0
Southampton Elementary 63.6% 64 74 10
Swansboro Elementary 55.5% 55 55 0
Thomas C. Boushall Middle 34.6% 35 39 4
Thomas Jefferson High 48.6% 49 50 1
Westover Hills Elementary 60.7% 61 64 3
William Fox Elementary 81.8% 82 83 1
Woodville Elementary 35.8% 36 40 4

Here we see six schools that would have flunked but were adjusted into full accreditation.  The Big Winner was Cary, with a 25 point boost.  You saw it here:  The actual math pass rate at Cary was 47%; the "adjustments" turned that into a passing 72%.  Meanwhile, Hill enjoyed a twelve point enhancement; Southampton and Holton, ten; Greene, nine; Fairfield, three.  These six schools are looking at trouble next year when the three year rolling average will include data only from the new math test.

It's hard to know what the negative boosts at Stuart and Alternative stand for.

The reading scores show even more dramatic "adjustments" with 19 of 43 schools boosted into full accreditation:

School Pass Rate Rounded Adjusted Boost
Albert Hill Middle 59.96% 60 83 23
Armstrong High  67.76% 68 79 11
Bellevue Elementary 64.52% 65 86 21
Binford Middle 41.56% 42 44 2
Blackwell Elementary 35.14% 35 41 6
Broad Rock Elementary 89.78% 90 86 -4
Chimborazo Elementary 34.60% 35 41 6
E.S.H. Greene Elementary 61.83% 62 84 22
Elizabeth D. Redd Elementary 48.81% 49 77 28
Elkhardt Middle 36.05% 36 37 1
Fairfield Court Elementary 67.83% 68 83 15
Franklin Military Academy 63.90% 64 82 18
Fred D. Thompson Middle 43.55% 44 44 0
G.H. Reid Elementary 46.81% 47 78 31
George Mason Elementary 37.69% 38 45 7
George W. Carver Elementary 76.28% 76 76 0
George Wythe High 75.90% 76 81 5
Ginter Park Elementary 30.40% 30 37 7
Henderson Middle 28.01% 28 30 2
Huguenot High 68.63% 69 84 15
J.B. Fisher Elementary 55.56% 56 78 22
J.E.B. Stuart Elementary 52.55% 53 76 23
J.L. Francis Elementary 51.47% 51 75 24
John B. Cary Elementary 55.43% 55 84 29
John Marshall High 88.52% 89 90 1
Linwood Holton Elementary 67.87% 68 83 15
Lucille M. Brown Middle 56.08% 56 57 1
Martin Luther King Jr. Middle  26.32% 26 28 2
Mary Munford Elementary 85.66% 86 86 0
Miles Jones Elementary 37.44% 37 39 2
Oak Grove/Bellemeade Elementary 40.00% 40 41 1
Open High 100.00% 100 99 -1
Overby-Sheppard Elementary 55.37% 55 79 24
Patrick Henry School Of Science And Arts 73.13% 73 80 7
Richmond Alternative  42.55% 43 29 -14
Richmond Community High 100.00% 100 100 0
Southampton Elementary 63.25% 63 82 19
Swansboro Elementary 55.91% 56 76 20
Thomas C. Boushall Middle 40.86% 41 40 -1
Thomas Jefferson High 79.01% 79 86 7
Westover Hills Elementary 61.70% 62 77 15
William Fox Elementary 84.82% 85 83 -2
Woodville Elementary 30.80% 31 35 4

The Big Winners here are Cary, with a 29 point boost and Redd, whose 49% pass rate turned into a fully accredited 77 by way of a 28 point boost.

The few one and two point negative boosts may reflect roundoff errors; maybe not.  On these data there's no accounting for the 4 and 14 point negative boosts.  All I can say is that, given the opacity and huge capacity of the adjustment process, those numbers probably are meaningless, as are so many of the other numbers.

 

The "Adjustments"

In contrast to the past, the VDOE Web site gives a fairly honest description of the accreditation calculations:

Ratings are based on the achievement of students on tests taken during the previous academic year and may also reflect a three-year average of achievement. Adjustments also may be made for students with limited English proficiency and for students who have recently transferred into a Virginia public school. Accreditation ratings also may reflect the success of a school in preparing students for retakes of SOL tests.

There is precious little authority for these shenanigans.  8VAC20-131-300, in the Accreditation portion of the Standards of Quality regulation, provides, in relevant part:

C. Accreditation ratings defined. Accreditation ratings awarded in an academic year are based upon Virginia assessment program scores from the academic year immediately prior to the year to which the accreditation rating applies. Accreditation ratings are defined as follows:

1. Fully accredited.

* * *
c. With tests administered beginning in the academic year 2012-2013 for the accreditation ratings awarded for school year 2013-2014 and beyond, a school will be rated Fully Accredited when its eligible students meet the pass rate of 75% in English and the pass rate of 70% in mathematics, science, and history and social science. Additionally, each school with a graduating class shall achieve a minimum of 85 percentage points on the Board of Education's graduation and completion index, as described in 8VAC20-131-280 B 2, to be rated Fully Accredited.

A search of the Virginia Administrative Code for "pass rate" produces the following result:

8VAC20-131-300 (show hits) ..Application of the standards.. (5)
8VAC20-630-40 (show hits) ..Program evaluation.. (1)
9VAC5-91-180 (show hits) ..Exhaust emissions standards for on-road testing thr.. (1)

The first two are regulations of the Board of Education.  8VAC20-131-300, which launched this inquiry, is quoted in relevant part above.  8VAC20-630-40 provides

Each local school division shall annually evaluate and modify, as appropriate, their remediation plan based on an analysis of the percentage of students meeting their remediation goals. The pass rate on the Standards of Learning assessments shall also be a measure of the effectiveness of the remedial program.

Hmm.  That's not much help.

8VAC20-131-5 defines "Eligible Student":

“Eligible students” means the total number of students of school age enrolled in the school at a grade or course for which a Standards of Learning test is required unless excluded under the provisions of 8VAC20-131-30 F  and 8VAC20-131-280 D relative to limited English proficient (LEP) students.

The two exceptions there relate to students with disabilities, whose participation in the SOL testing is prescribed by the applicable IEP or 504 plan, and LEP students enrolled in Virginia schools for fewer than 11 semesters. 

The regulation at 8VAC20-131-280 creates the three-year average score option and allows the accreditation to be based (in unspecified part) on the "number of students who successfully complete a remediation recovery plan."

The regulation does not otherwise define the term "pass rate."  Standing alone, the regulation would seem to use the term in its usual sense of number passing divided by number taking.  Yet the VDOE Web page (quoted above) explicitly provides for "adjustments" to the pass rate that go considerable beyond the exceptions set out in the regulation.  The VDOE procedures for calculating the accreditation "pass rate" exclude the following test records from the calculation:

• Test records marked as cheating;
• Test records marked as Grade code “TT”;
• Test records of students attending non-public schools;
• Non-passing Retest;
• Non-passing 8th grade and EOC Recovery;
• Non-passing SOA LEP; and,
• Non-passing SOA Transfer students.

Under "Accreditation Benchmarks," the procedures further provide:

• Either the current year or the three-year average must meet the current year Board-approved
benchmark to be accredited in a given subject.
• Passing Math and English Recovery tests scores for 4th through 8th grade and EOC will count twice.
• Failing recover tests scores for 4th through 7th grade count once.
When no scores are received for a given test, it will be indicated on the report as, “NS” (No score received for this subject area) and accreditation will be based on the other subject areas.
• Exclude failing 8th grade Recovery test scores, if the student has previously taken the same test in any school year (past or present).

Can you see VDOE's thumb on the scale here? 

  • Schools are encouraged to get their weaker students to cheat so those scores won't count.  Yet nothing in the regulation suggests that a cheating student is not an "eligible student."

  • Code TT is a "test taker" not in a grade and not post graduate.  Chuck Pyle tells me this could be a student who has completed the 12th grade but has to sit for the test to get the verified credits necessary to graduate.  In any case, a "TT" student can take the SOL until the computer wears out without effect on the school's accreditation.

  • Non-passing retests, apparently without any colorable authority in the regulation, can likewise be repeated without affecting the school's accreditation until the student gets a pass, whereupon the pass counts!

  • Eighth grade students flunking "Recovery" and LEP failing scores don't count against the school while the 8th graders passing Recovery retests count twice!  And, to a lesser degree, in grades 4-7, flunking Recovery scores count once but passing scores count twice.  See this.

  • And, the Big One for this year's math and reading tests (albeit expressly allowed by the regulation), if the school doesn't make the "adjusted" grade on this year's test, VDOE will use a three year average (and, as we have seen above, boost a school's score by as much as 29 points).

All these "adjustments" go in one direction: Up.  All these adjustments are totally unnecessary: VDOE writes the test and can make it as easy or hard as they want.  The adjustments only contribute opacity to the process . . . and, as we see above, create opportunities for accreditation scores that are divorced from a school's actual performance.

Your tax dollars at work.

Back to the Top

High Taxes | High Crime | Lousy Schools | Obdurate Bureaucrats

Last updated 01/05/14
Please send questions or comments to John Butcher