by State Auditor Doug Hoffer Evidence-based. Evidence-informed. Data-driven. Results Based Accountability. PIVOT. Vermont policymakers frequently use these terms to indicate that new policies and spending decisions are the result of rigorous, fully-informed analysis.
Too often, though, the promise of these terms is undercut by bad data, incomplete information, and misleading representations from officials.
When State government does performance measurement well, Vermonters are able to determine whether state programs are working well and if their tax dollars are being efficiently spent. When it’s done poorly, though, the information can tell a less-than-complete or misleading story, leading to bad decisions and wasteful spending. At worst, it can be a total waste of time.
Here are some examples we’ve encountered in our work of State agencies presenting less-than-helpful portrayals of government performance:
- The City of Burlington’s 2021 Tax Increment financing report counted 150 Burlington High School teachers and staff as jobs “created” by the program, when they have merely moved temporarily out of the PCB-impacted BHS and into the empty Macy’s department store building. The Vermont Economic Progress Council, the state agency that administers the TIF program, acknowledges that “there is no way to verify the TIF District-reported numbers” (p.13). Nevertheless, they continue to report whatever the towns submit to them.
- In Measuring Up: Examining performance reporting in the Vermont Annual Outcomes Report, we found the State’s annual outcomes report depicted the percent of Vermont covered by “state-of-the-art telecommunications infrastructure” at a clearly implausible 96.2% (it was based upon antiquated 4/1 download/upload speeds). The report also represents subtle changes in certain measures as meaningful when they are within the margin of error (e.g. fluctuations in the percent of adults who smoke cigarettes), meaning that there has not been any statistically significant change. Yet these outcomes are portrayed as meaningful trends.
- The Legislature’s Joint Fiscal Office (JFO) analyzed an Executive branch report touting the success of the “new worker relocation grant” program and found it relied on unrepresentative and misleading data to derive its most important conclusions. Nonetheless, the report remains on the Agency of Commerce and Community Development (ACCD) website with no mention of the JFO analysis, which effectively rendered the report of little to no value.
- Our audit of the Agency of Digital Service’s IT project management found that a key performance measure, % of on-target projects, is calculated based on the percentage of projects in “green” status. Vermonters would assume that means the project has gone smoothly and is expected to be completed on time. However, ADS considers some projects to be “green” even with significant cost increases and completion date extensions. Their use of “green” status is based upon revised cost and schedule expectations, masking how well the project has met initial expectations.
- Our evaluation of the State’s Ten-Year Telecommunications Plan found that it included no performance measures, leaving legislators and the public without any measurable basis for determining whether the Public Service Department is doing a good job furthering Vermont’s broadband, cell service, and emergency communications goals. The response from the Commissioner of Public Service was that they didn’t include performance measures because the statute doesn’t require them to. Nothing prevents the department from adopting useful measures of their performance – in fact, it should be considered a no-brainer in this day and age.
- Performance reporting for the ACCD’s Vermont Training Program makes it difficult to know the extent to which taxpayer funded training has benefited the trained employees. The methodology includes several measures of questionable usefulness, and the data excludes almost half of all the trainees. (See pages 5 and 6 for details.)
- Our audit of the Dam Safety Program found that the Department of Environmental Conservation failed to regularly monitor dam conditions, so there is no assurance that they are accurately calculating and correctly reporting the percentage of dams in poor condition. This summer’s floods highlight the importance of providing the public accurate information in this area.
- Our review of ten nurse recruitment and retention incentive programs found that none had performance measurement in place to determine whether the incentives were having any effect on the size of Vermont’s nursing workforce.
When we encounter these weaknesses in government accountability, we offer constructive suggestions to the relevant agency. Sometimes they adopt our recommendations, but not always. When they do not, it is Vermonters who lose out. Vermonters fund State government and are the ones who are impacted by its performance. They deserve accurate, clear, and timely performance measures.
Sincerely,
Click here to see our complete report on performance reporting in Vermont.