Tuesday, October 31, 2006

Play the Accountability Game!

This is a real hoot!

Sunday, October 29, 2006

The Problems with Too High a Standard

So why not set the highest standards in the nation?

Suppose your kids go to a low-performing school; that school scored 40 on the math test.

In order to be declared as making "adequate yearly progress", this school would need to improve its score by 5 points every year. After two years, it would be up to 50. Four years, 60. Six years, 70. Eight years, 80. Ten years, 90. Twelve Years, 100.

Bingo, isn't that what we want?

Well, let's see what happens under AYP. Last year was a good year for Hispanic kids on the math tests; they gained, on average, 3 points, faster than any other subgroup and twice as fast as White kids. Suppose they managed that level of phenomenal gains for twelve years.

In the first year, they would miss AYP, because they would need to reach a score of 45, but they would only reach 43.

Second year, they'd miss, hitting 46, but needing 50. Hmm. Now they are four points down. The school is listed in the newspaper as failing to make adequate progress.

Third year, hit 49, need 55. Down by six points - they will need to advance by 11 points next year to avoid going into 4th year sanctions, where the principal can be fired.

Fourth year, they hit 52, but too bad; the state starts to assume control of their school, because they needed to hit 60. And the situation is looking really dire; next year, to avoid going into the terminal stage of sanctions, they will need to improve by a whopping 13 points.

Fifth year, they hit 55. They've improved by about as much as anyone in the state over five years, but the school goes to the final stage of restructuring, because they would have to hit 65 to avoid seeing "all or most of the staff fired", and have the school reopen either as a charter school, or under an approved private management firm, or under the control of the state.

Now, the situation is much worse than this, because these have been historically high gains for this group of special ed kids. Last year, on average, saw English Language Arts scores DECLINE by three tenths of a point across the state. It was a good year for math, with an average 1.5 point gain.

But guess what? The average "improvement target" last year for English was 5.3, and for math was 8.6. For subgroups, the improvement required to avoid making "inadequate progress" was 17.9 for Special Ed Math, 14.0 for Limited English Speakers on English Language.

Any further determinations of "adequate progress" are simply meaningless when the required gains in a year reach than a few points. A school could have top 10% performance in the state for three years running and still reach fifth year sanctions - firing all or most of the staff, turn over control to the state.

Look at it another way. A school that started at 90 is expected to go from 90 to 100 in 12 years. A school that started at 40 is expected to go from 90 to 100 in 2 years, after they have already gone from 40 to 90 in ten years.

Unrealistic expectations to meet a seemingly noble goal can do as much harm as good. They can be used to justify a sense of urgency for questionable, untested, or perhaps even counter-productive reforms. And they sap any sense of achievement along the way.

We must have realistic goals and timeframes as a starting point for serious education reform.

Tuesday, October 24, 2006

A National Hero

Copy of a letter to the editor in the Orlando Sentinel.

October 15, 2006

Surprisingly, it's sanction time at our little FCAT A school here at R.J. Longstreet Elementary in Daytona Beach.

After having received five A's in a row on the FCAT, but simultaneously having failed the annual progress goal of the federal No Child Left Behind Act, we are under a sanction called "corrective action."

We are now being visited by district level "support personnel" who will monitor how we are teaching writing and math. Of course, we are teaching writing quite well, because more than 80 percent of our students got a 3.5 grade-point average or better on the writing section of the FCAT.

And our math scores were high enough to help us earn the five consecutive A grades. The problem for Longstreet originated in our district's response to the federal request for us to set up an annual progress goal.

The goal we set was one we haven't reached yet. In hindsight, knowing what we know now, we could have set a lower goal and been fine. But we didn't.

So now come the sanctions. One sanction option was to reconstitute our entire school, which would have meant getting rid of the teachers who were responsible for teaching more than 80 percent of our kids to do wonderfully on the FCAT.

When you consider that more than half our school's students come from backgrounds of poverty, our teachers performed a miracle. They were able to successfully teach the majority of our students despite these dismal odds.

I guess it didn't make good political sense to get rid of great teachers, so now we are receiving scrutiny by people who may not have the skills of those teachers over whose shoulders they are assigned to look.

What is happening is pandemic across the nation. Public education is under siege from state and federal politicos who are transforming what was once an arena of pure educational learning into a corporate state of testing.

This allowed well-connected publishing companies to gorge themselves on public school dollars in a frenzied testing environment that has been sold as "accountability" to the unsuspecting public.

Local administrators are intimidated and fearful of losing money for their districts, maybe even losing their jobs if they resist.

You can hear them saying submissively, "We just have to play along."

R.J. Longstreet Elementary School is so much better than the A repeatedly assigned to it by evaluation processes that can't meet minimal standards of reliability and validity in the real world of accountability.

It is a safe school, a community-involvement award winner, a place where kids enjoy coming to learn, socialize and play each day.

Its parents are supportive and caring.

Its biggest enemies are these tests and those behind these predatory programs that starve those in need and lavish the money saved on their corporate accomplices.

This current administration is working hard to change the face of the world into its own corrupt image, and it is succeeding in public education, the only place where a defense could have been mounted to defeat it.

R.J. Longstreet Elementary is fighting to maintain its right to educate rather than become of victim of the testing craze.

We wear our badge of sanctions proudly, knowing we are succeeding despite misguided actions.

Bill Archer is a school counselor at R.J. Longstreet Elementary. He lives in Daytona Beach.
Copyright © 2006, Orlando Sentinel | Get home delivery - up to 50% off

Accountability standards for corporations

Read this Bloomberg article about accountability standards for corporations.

U.S. Treasury Secretary Henry Paulson said he is considering recommending changes to the 2002 Sarbanes-Oxley corporate governance law because its restrictions have overwhelmed some American companies.

While the ``net result'' of stricter reporting standards for executives has been positive, Sarbanes-Oxley has also contributed to ``an atmosphere that has made it more burdensome for companies to operate,'' Paulson said in an interview today from Washington.

``We're going to need to look at how we can address some of these issues,'' Paulson said. ``This is something we're giving a lot of thought to.''

The remarkable thing is, you could substitute NCLB for Sarbanes Oxley, and American companies for American public schools, and arrive at the same dialog. Except, in the case of Sarbanes Oxley, our federal government is siding with corporations, and with NCLB, they think they got that right and it is the parents and teachers who've expressed concerns who are wrong.

Monday, October 23, 2006

Central Planning Mainframe Mindset

Standardized testing harks back to the 1950s, in the days of punch cards and mainframe computers. You'd submit a request for processor time, carefully punch out all the cards, submit them, and wait. Some time later, an operator would put the cards in the queue, and report results back. Programmers would hope the result wasn't that they had gotten one card out of order and they had to start over.

All that changed beginning in the late 1970s for computers. By the time I started programming computers, we could run Basic programs on a teletype. Before Jimmy Carter left office, I had a computer with 64K of RAM, a color display, a hard drive, and a printer. These days, you can get more computer power on your desktop than was available to all of NASA in the early days of Apollo. And for your home computer, you can get education software that gives your kids instant feedback on their learning process.

But not so for standardized tests. Kids fill out computer forms that have hardly evolved since the days of punch cards. They do long comprehensive tests that give little opportunity to help the child that took the test, rather than short quizzes on material they are currently learning. And then, they wait. The tests are gathered with the same level of security and reliability that we get at the voting booth, and carted off to a central bureaucracy, where they are tabulated.

And then the kids go home for the summer.

By September, some of the results come back to the school. Here in Massachusetts, the AYP Press Release of failed schools goes out in mid-September, or roughly four months too late for a school to make effective use of the data for high level planning purposes for the year.

Our principle says they think they can make good use of some of the disaggregated data from the test results they got back from central planning.

I bet they could have gotten a lot more use out of that if they'd just had a series of micro-tests during the course of instruction.

Meanwhile, here we are, at year 2.5 - we just found out that our school missed AYP this fall, a half a year after the tests that could have told us, and four months after the school could have used the information to adapt curriculum or classroom configurations for this year. So we'll be going into the third year before the principle has a lot of options for adjusting things.

In the fourth year, the principle can get fired, so I suspect we will see some changes next year.

High on the list, if the Principle is desperate to hold onto his job, will be retention of a few third-graders who appear headed for failure in 4th grade MCAS testing. One of the dirty secrets of No Child Left Behind is that the easiest way to improve test scores is to identify kids likely not to hit the proficient threshold, and hold them back.

I remember back when I went into 7th grade, there were a couple kids who'd been held back a year or two. They had a hard time fitting in the desks, and were always in trouble for disrupting the classroom. I was about a foot shorter than they were.

I read in today's Washington Post that as many as 14% of Florida 3rd graders have been held back a year.

If they're holding back third graders in Florida, I bet Florida's test results are coming back a couple months sooner than Massachusett's test results do.

Wednesday, October 18, 2006

Dissonance: The Broad Prize .vs. Massachusetts AYP

( And the answer is - the Broad Prize got it right, and Massachusetts AYP formula requires arbitrarily high performance gains for a school to be labeled "adequate")

A couple other blogs have posted about the Broad Prize.

Boston won the Broad Prize this year, but 65% of schools in Boston were identified by the state as failing to make Adequate Yearly Progress.

I've demonstrated that only a handful of schools in Massachusetts matched the scores that will eventually be mandated by the state to avoid being labeled as inadequate schools and subjected to No Child Left Behind sanctions.

Today, I decided to ferret out why 65% of Boston's schools have failed AYP, while only about a quarter of schools across the state have failed.

To do this, I looked closer at Massachusett's improvement_targets.xls spreadsheet. This spreadsheet shows CPI scores for each subgroup in each school across the state, as well as "Improvement Targets". The improvement targets for each subgroup are (I am told - the actual formulas are not included in the spreadsheet - only current year data is included. Wonder why?) require that subgroup to improve from the baseline scores to 100% proficient by 2014.

That means, for example, if baseline score in 2002 was 60, then the school is mandated to improve by 40 points over 12 years, or 3.3 points each year.

OK, so we are told about 25% of schools across the state failed to make AYP. But if you calculate whether a school actually made it's improvement target, I find that 1216 subgroups out of 5054 made it for English Language Arts; and 1287 out of 5054 made it for math. 522 out of 5054 made BOTH.

So about one out of ten subgroups made it across the state. That is, NINETY PERCENT OF SUBGROUPS FAILED TO MAKE THE PROGRESS MANDATED BY THE FORMULA.

So how come only 25% of schools are identified for improvement? After all, a school is supposed to be identified for improvement if any subgroup fails either math or ELA.

And the secret is that in addition to the individual school's improvement target, there is also a state threshold. If your CPI beats either, your school is called "adequate" for that year. The state thresholds started at 53.0 for math, and 70.7 for english, in 2002. Both hit 100 in 2014. So, as of spring 2006, the state cut scores were 68.7 for math, 80.5 for english.

What this effectively says is that any school that scored above the state average in 2002 has been given a "get out of AYP Free" card for a few years. These schools can be deemed as making adequate progress even if their test scores don't improve - for the time being. But meanwhile, schools that started at or below the state average have to improve by leaps and bounds every year.

So - there you have it. Why do 65% of Boston's schools fail AYP while only 25% fail statewide? The answer is that 90% fail statewide, if you use the calculation that actually requires the school to make progress.

Without any knowledge about the Broad Prize's selection criteria, we can say that it is more likely that the Broad Prize uses a more rational and more accurate means of evaluating excellence and improvement of urban public school systems than No Child Left Behind does.

Monday, October 16, 2006

Loss of local control

One of the biggest advantages of small town schools is the autonomy they have enjoyed. All the parents know all the teachers - they can talk with the teachers every day. The principal comes out to the parking lot to talk every day at pick-up time. Our school has been accountable to parents since before the newfangled idea of accountability was invented. The school shares a regional high school with several other towns, but the district superintendent has traditionally delegated responsibility to the principle and to local control.

No Child Left Behind shifts accountability. Administrators must comply with federal mandates as interepreted by state bureaucrats, and if they don't, they can be fired. This goes for staffing, for curriculum, and for complying with mandated standardized test scores. In particular, if a district fails to make "adequate yearly progress", as most schools in Massachustts inevitably will, any administrators that haven't following the mandates of state and federal bureaucrats are likely to lose their jobs in the 4th or 5th year. Same goes for teachers.

So, now the superintendent is actively intervening in local affairs - saying the state made him do it - out of fear of losing his job.

We want the state and federal government to get out of the way and let our local school continue to do its job. Local accountability is more important than accountability to federal and state bureaucrats.

Explaining No Child Left Behind to Totalitarians

Just as the People’s Republic of China has its nine-year Compulsory Education Law of 1985, the United States has a pre-eminent law as well. The U.S. law is known as the No Child Left Behind Act of 2001 (NCLB).

NCLB has significantly impacted state educational systems and local school districts. The law’s requirement for standards, assessments, corrective actions, public report cards and adequate yearly progress has altered many state testing programs.

This is in a report written by Harcourt Assessment. Later in the report, we find the explanation of sanctions:


Perhaps the most significant part of NCLB is that schools must help each group and the aggregate make progress or face the consequences. Schools not making adequate yearly progress have to take corrective steps ranging from offering parents public school choice to restructuring schools. If schools are able to intervene and achieve adequate yearly progress for two consecutive years, they can have the sanctions removed.
The consequences for not meeting adequate yearly progress for two consecutive years are that schools must go through the following steps of corrective action for school improvement:

Public school choice

If a school does not make adequate yearly progress for two consecutive years, parents can choose to have their child attend a different school.

Supplemental services
If a school does not make adequate yearly progress for three consecutive years, the district must provide public school choice and provide supplemental services (e.g., tutoring) to low-achieving students, disadvantaged students, students with disabilities and English language learners.

Corrective action
If a school fails to make adequate yearly progress for four consecutive years, the district must implement certain actions such as changing the curriculum, getting a new principal or replacing staff.

If a school fails to make adequate yearly progress for five consecutive years, it must restructure the school (e.g., state takeover, hire a private company).
Once a school makes adequate yearly progress for two consecutive years, it is considered improved.

American public schools have traditionally been locally controlled. In towns like ours, we used to have accountability - just a conversation with teachers and the principal away. No Child Left Behind puts state and federal bureaucrats ahead of parents and school committees.

Thursday, October 12, 2006

Adequate Yearly Progress mandate: every public school in Massachusetts must match Boston Latin's MCAS scores

Boston Latin is an exceptional school. It screens potential students with an exam and only accepts the highest scoring kids from across Boston.

Boston Latin is also one of the few schools in Massachusetts to approach the "100% proficient" level mandated by No Child Left Behind. All schools must reach 100% "proficiency" by 2014, or they are subjected to sanctions.

I asked the Department of Education how many schools across the state had reached the test scores mandated for 2014, and did not get a reply. So I decided to review the data myself, looking at the state's spreadsheet , and following the advice the DOE gave me, to read this document to learn how Massachussetts implements its AYP

I sorted schools based on their Math and English Language Arts (ELA) CPI data - this is the score that has to eventually reach "100%". And there, at the top of the list, was Boston Latin - one of a handful of schools across the state to come close to 100% proficient in both English Language Arts and Math scores.

The state allows an "error band" that means schools will only need to achieve proficiency somewhere between 95.5% and 97.5% proficient, but even allowing for that error band, only about two out of every 100 subgroups tested scored as high as 95.5% proficient for either Math or ELA. A school can fail if any subgroup does not reach the mandated proficiency level - subgroups include low income, special ed, black, hispanic, white, and asian. I didn't bother to try to calculate how many schools could pass the whole gamut of subgroups, but I'd say a reasonable first approximation is to guess that each group cuts the odds of making the grade in half. So if a school is over the 95th percentile for one of Math or English in aggregate, and it has three subgroups, guess something like 2% divided by 2 to the 4th, or say 2/16th of one percent of schools that the state would label as "adequate", if the standards that will be mandated in 2014 were applied to 2005 test data.

Oops. Boston Latin composite for ELA was 98.4, and their group size was big enough for the 2.5% error band. I guess even Boston Latin is an inadequate school, according to the state and federal mandates.

The state and federal government mandate that every school across the state of Massachusetts must achieve standardized test scores that will be difficult even for the best schools in the state, and applies sanctions to any schools that fail to do so.

Everyone would like that to be possible, and to strive towards that goal, but as a mandate that labels any school that fails to do so "inadequate" and applies radical sanctions - transferring control over local schools to state and federal bureaucrats - Houston, we have a problem.

Monday, October 09, 2006

I live in a small, median town pretty far away from Beacon Hill and Washington, or at least so we thought when we moved out here. But in the past year, our local public school has taken a beating, all inflicted by bureaucrats in Boston and Washington.

We've suffered a perfect storm of budget cuts and mandated changes in curriculum, and on top of that, the federal government, with the encouragement of our state Department of Education, says our school is "inadequate", even though our school has above average standardized test scores.

Our school has been an attractive "choice" school drawing kids from surrounding towns; but it seems this sort of "choice" is out of favor in the rarified atmosphere of Beacon Hill and Washington, where "public" schools are supposed to become lowest common denomimator drill factories (preferably run by corporate managers), and "Choice" means either a charter school or a private school.

We want the Department of Education to be more responsive to parents, more respectful of teachers, and more critical & independent of the federal government.

We want our school back.