by Timothy McQuiston State Auditor Doug Hoffer today released a report on Vermont’s statewide health information network. Hoffer concluded that despite millions being spent on it, the state has no idea whether or not the network has improved health outcomes. The Vermont Health Information Exchange (VHIE), is managed by the Vermont Information Technology Leaders, Inc (VITL). The state has paid VITL over $38 million since 2005. Almost one-third of this amount ($12.3 million) was expended in fiscal years 2015 and 2016 through grants and contracts with the Department of Vermont Health Access (DVHA).
The objective of Hoffer’s audit was to assess whether and how the state evaluated VITL’s activities and measured VITL’s performance in fiscal years 2015 and 2016 grant and contract agreements executed by DVHA as of June 30, 2016. For certain agreements, DVHA shared oversight responsibilities with the Agency of Administration (AOA).
The audit found that DVHA and AOA oversaw VITL by obtaining monthly status reports and required deliverables and holding regular meetings with VITL.
Nevertheless, there were oversight deficiencies. In particular, the State did not sufficiently oversee the building of a clinical data warehouse by VITL, which stores parsed data from the VHIE to use for analysis and reporting. Also, the state never explicitly included the clinical data warehouse as a deliverable in the agreements with VITL and did not define expected functional and performance requirements.
Thus, the state is not in a position to know whether the clinical data warehouse is functioning as it intends.
“DVHA’s agreements with VITL included limited performance measures for its work, and neither DVHA nor AOA used mechanisms called for within the agreements to develop quantifiable performance measures after the agreements were finalized,” Hoffer concluded in the report. “Without quantifiable performance measures, the state’s ability to judge VITL’s efforts and gauge success is significantly inhibited.”
Also, until recently, DVHA was authorizing payments to VITL even though its invoices did not always include detailed substantiating information, such as specific hours by individual and project.
DVHA recently took action to address this issue.
For example, in its most recent grant agreement (for fiscal year 2017), DVHA requires VITL to submit invoices that break down the total amount billed into budget categories and be accompanied by detailed accounting information.
In 2016, DVHA also received and reviewed VITL’s detailed accounting data from July 1, 2014 to December 31, 2015. DVHA has questioned the allowability of some costs in this time period, but as of early September 2016 had not reached a final conclusion.
The audit also found that while the state’s agreements required VITL to report on performance measures related to the quantity of its work (“how much”), the agreements lacked measures to assess the quality (“how well”) and impact (“is anyone better off”) of VITL’s work.
The agreements also lacked performance targets.
As a result, the state is unable to adequately assess the performance of VITL and to demonstrate the value of the VHIE.
Hoffer said DVHA and VITL have begun to address these issues. For example, DVHA has agreed to fund an impact assessment by VITL that will assess the impacts of VITL’s work on those health care organizations that also participate in DVHA’s Blueprint for Health initiative. The state also has begun a process to develop targets to be used in future VITL agreements.
“We made a variety of recommendations to DVHA, such as adding quality and impact measures in future agreements with VITL,” Hoffer said in the report.
The auditor, however, does not have enforcement authority.
The report also includes a section on nationwide challenges associated with health care organizations sharing electronic health records, such as insufficiencies in health data standards. “These challenges must be overcome to have an efficient health information exchange in Vermont that can be utilized to achieve the state’s goals of improving population health, improving quality of care, and reducing health care costs,” Hoffer said.
While the departments in question generally agreed to address the auditor’s concerns, DVHA Commissioner Steven Costantino and Deputy Administration Secretary Michael Clasen were offered a trenchant response: “The State Has Begun to Address Oversight Deficiencies, but Has Limited Measures in Place to Evaluate Performance.”
Source: Auditor of Accounts. 10.3.2016. The full report is available at the state auditor’s website: http://auditor.vermont.gov/sites/auditor/files/VITL%20Final%20Report%2010.3.2016.pdf