Staying on sidelines put to test in voucher program audit

Long valuable as an instrument to determine whether government follows the law and to render a judgment on how well it does, the Louisiana Legislative Auditor retains this value because the office investigates impartially process and output of policy. Thus it’s an unwelcome development on the occasions where it begins to allow to slip into its products preferred policy positioning, as happened in a recent audit of the Louisiana Student Scholarships for Educational Excellence Program.

This audit probed the implementation of the program that allows families whose children are in underperforming public schools to receive state money to attend a participating, either public or nonpublic, qualifying school of their choice. It made several recommendations for programmatic changes, either by law or by Department of Education regulation, that addressed implementation issues including that DOE oversee more closely that participant schools keep better financial and client records, that DOE provide better metrics to match school resources to capacity in serving voucher students, and that DOE institute specific criteria for determining whether schools should continue participation, for consistency sake and accountability purposes.

In this case, the LLA produced a “performance audit,” wherein the agency not only renders an accounting of how well the law was followed, but also may insert recommendations for program operation, either directly to the agency running the program or (in a “matter for legislative consideration”) as a legal change to be wrought by its boss, the Legislature. And it’s here where problems may creep in, for in recommending these things the LLA is tempted to substitute its own policy-making judgment for that of the implementers or creators of the program.

For example, DOE disagreed with the assessment of specific criteria needed for continued program eligibility, pointing out it had applied an internal standard that any two-year participant school with at least 10 tested students that had on those tests an average proficiency score of less than 25 percent that would end up restricting seven schools as a result. The LLA in response averred these criteria did not address complete ejection from program participation.

But in fact Bulletin 133, which summarizes regulations promulgated by DOE and authorized by the Board of Elementary and Secondary Education concerning this program, in its Chapter 9 is specified eight different criteria with which a provider must comply to continue in the program. It does not contain performance data as one of these, although it does allow for DOE to enforce “criteria for participation” (for example, allowing DOE to limit the number of slots based upon performance). Accordingly, because statute punts the question to BESE and BESE issued these regulations, this is not a question of DOE not wanting to do or not clarifying, it’s an issue of BESE nor the Legislature from wishing to establish numerical targets.

Now, the LLA could have phrased this as a “matter for legislative consideration,” as in “The legislature may wish to consider establishing quantifiable performance criteria based upon the scholarship cohort’s proficiency scores for program continuation” or something along those lines. And, that may not be a bad idea although structural aspects of the program may make computing usable aggregate scores for each and every provider impossible, hence the discretion given DOE. But the approach of the LLA in the audit was to opine that, since public schools both traditional and charter faced sanctions as a result of poor performance based upon a quantifiable scoring method, a similar kind of method should be used in this circumstance for nonpublic schools as well.

Unfortunately, that statement reflects more of a policy preference of the LLA – you have to compute some quantitative performance-based standards and decide whether to kick schools out on that basis – than presentation of a policy option. If BESE has decided to eschew use of specific performance-based criteria for whatever reason or even not to address whether a participant can be removed, that’s its call and is subject to political debate over whether that is the optimal policy. But the LLA is not to take sides in this debate.

Yet even the “matter for legislative recommendation” route contains some potential pitfalls along these lines. In the report, the LLA also makes one of these, after a brief review of eligibility for provider participation. It notes that for public school participants they have to meet a certain academic grade that defines “acceptability,” but no such standard exists in law or regulation for nonpublic providers in this program. Statute only reads that a participating school meet non-public school approval (including a curriculum at least equal to that in rigor for public schools) by BESE, comply with nondiscrimination standards, and adhere to a 20 percent enrollment cap of slots requested if having been in the program fewer than two years.

Again, this puts the LLA perilously close to substituting policy content for evaluation. It could be that policy-makers already have decided that the concept of “acceptability” does not translate exactly or well here, that, as families will have a choice, the market will reward schools that do a good job with more students and those that do not will get fewer or zero, with “acceptability” measured by criteria families think important (which likely will have a lot to do with their students’ or families’ children’s test scores). It’s another policy debate worth having (and one which was briefly broached in the legislative review of the report last week), but the point is the LLA should not insert itself into that by, with such a ruling and its phrasing, conveying a kind of imprimatur to one side of the argument.

Whether unrealized lapses or otherwise, the LLA needs to take great care in not exceeding its statutory mandate to examine the “financial accountability, legal compliance and evaluations of the economy, efficiency, and effectiveness of the auditee’s programs or any combination of the foregoing.” This particular report gives an example of where in this regard it could have exercised more caution.