Vermont Business Magazine Vermont State Auditor Doug Hoffer has released a new investigative report analyzing the performance of the Vermont Department of Labor’s primary job training programs. These Workforce Innovation and Opportunity Act (WIOA) Title I and Title III programs receive about $7 million from the federal government each year and are administered by the Vermont Department of Labor.
“Job training programs are extremely important because they help people realize their potential,” said Auditor Hoffer. “When done effectively, individual Vermonters can make more money, get off public assistance, and enjoy a more fulfilling worklife. As the largest source of training funding in the state, we felt it was important to analyze how the WIOA programs are serving Vermonters with an eye to maximizing the impact of these federal dollars.”
The report evaluates the percentage of the total eligible population being served, the employment and earnings outcomes for program participants, the impact of training services compared to general job-seeking assistance, and the performance of Vermont relative to other states.
In each area, the report recognizes that performance could be negatively affected by the Department of Labor’s stated priority of serving Vermonters with high economic need and barriers to employment.
The report finds the following:
- Across all categories of eligible Vermonters, WIOA funds serve a relatively small percentage of potential participants.
- While some groups experienced increased employment and earnings, across all programs a significant portion of participants had lower earnings, lost employment, or remained unemployed when compared to preprogram status.
- Participants who received training services had higher quarterly earnings than those who only received general job-seeking assistance.
- Compared to other states, Vermont’s performance falls in the bottom half for most measures.
Hoffer added: “The Department of Labor’s commitment to serving people with high economic need and barriers to employment is commendable. Several of the findings in this report speak to the positive impact that these types of programs can have, and we know that the Department’s dedicated staff help many Vermonters advance their careers each year. However, our analyses also revealed limitations in the programs’ reach and impact. We hope the information in this report helps the Department and the Legislature improve these job training services so more Vermonters can earn livable wages and fulfill their potential.”
Executive Summary
The Workforce Innovation and Opportunity Act (WIOA) is a federally funded workforce development program that offers 1) education and training services to increase participants’ employment and earnings and 2) services to employers to help them access and retain skilled workers. The Workforce Development Division at the Vermont Department of Labor (VDOL) administers the Title I and Title III WIOA programs in Vermont. This report is focused on Title I and Title III services provided to jobseekers.
Examples of services for jobseekers across WIOA Title I and Title III programs
Given the centrality of this large amount of funding ($6.8 million annually) to the Workforce Development Division’s mission to coordinate “the education and training of our workforce for Vermont’s current and future job opportunities,” the State Auditor’s Office (SAO) initiated this inquiry to examine the performance of Vermont’s WIOA Title I and Title III programs. We focused our inquiry in four areas:
1. How many people are served by WIOA relative to the level of need in Vermont?
While we did not have data to quantify the total eligible populations for each program, we drew on other sources as proxies to estimate the number of potential participants. Based upon the limited information available, we conclude that the Title I and III programs serve a small percentage of the eligible population of Vermonters.
Summary of the target populations for WIOA Title I and Title III programs
2. Are participants connected with quality jobs?
WIOA is designed to get participants “into high quality jobs and careers.” In our sample, 58% of participants in the Adult program had higher earnings or gained employment following WIOA, while 42% had lower earnings, lost employment, or remained unemployed when compared to preprogram status. In the Dislocated Worker and Wagner-Peyser programs, the majority (61% and 59% respectively) had lower earnings, lost employment, or remained unemployed after exit.
Trends in quarterly earnings two quarters after program exit
Across all programs, a significant portion of participants had no earnings or earnings below $4,999 two quarters after exit. This amounts to less than $20,000 a year, which is less than the annualized pay of a full-time minimum wage earner. Participants’ earning potential is impacted by many factors, some of which are outside of the control of the VDOL, and certain participants face significant barriers to employment (e.g., English language learner, health status, involvement in the criminal justice system).
3. Which program components are most effective?
The Title I programs offer participants both career services (e.g., job search assistance, case management) and training services (e.g., on the job training, occupational skills development). Among participants in the Adult and Dislocated Worker programs, participants who received training had higher quarterly earnings following program exit. However, a notable percentage of participants (18% to 20%) did not complete their training and approximately half of participants who found employment were not working in a field related to their training.
4. How does Vermont’s performance compare to other states?
Vermont’s performance varies by program and metric but falls in the bottom half for most measures. Meanwhile, Vermont’s average Title I participant cost is higher than comparable states. Title III Wagner-Peyser spending is more similar to comparable states.
Areas for Further Consideration
• Evaluate proactive outreach strategies to increase enrollment.
• Explore opportunities for additional data collection and evaluation to inform program design.
• Review opportunities for which a federal waiver may increase flexibility to address Vermont-specific needs.
Signed into law in 2014, WIOA is the primary federal workforce development law, funding programs for jobseekers, workers, and employers. This report is focused on Vermont’s Title I and Title III WIOA programs for jobseekers, which are administered by the Workforce Development Division at the VDOL.3 Title I programs provide intensive career services and training for three groups: eligible adults, dislocated workers, and youth (ages 14-24). Title III funds “Wagner-Peyser” employment services, which are available to all job seekers. Services are offered primarily through the VDOL’s twelve regional Career Resource Centers. While states have some flexibility in how they design their WIOA programs, states must adhere to numerous federal requirements regarding participant eligibility, program design, performance metrics, and more.
Table 1. Examples of services for jobseekers across WIOA Title I and Title III programs
In Program Years 2018 and 2019, Vermont spent an average of $6.8 million of federal funding annually for Title I and Title III programs.
Many of the VDOL’s programs, particularly in Title I, serve individuals with high economic need and those facing barriers to employment. Among participants who exited from the Adult program from April 2019 to March 2020, 92% were low income,4 75% received public assistance, 43% had a disability, 39% were long-term unemployed, and 28% were ex-offenders. Participant characteristics vary considerably by program. Among participants who exited from the Dislocated Workers program, 24% were low income, 10% had a disability, 11% were long-term unemployed, and 9% were ex-offenders.5
Vermont’s WIOA Title I and Title III Program Performance
To analyze the performance of Vermont’s Title I and Title III programs, the SAO drew from a combination of data published by the United States Department of Labor (USDOL) and the VDOL, as well as our own analyses of deidentified participant-level data among those who exited the program in calendar year 2018. Since the VDOL tracks performance throughout the year following program exit, we focused on this time period to avoid the potential impact of the COVID-19 pandemic. We sought to answer four questions:
1) How many people are served by WIOA Title I and Title III programs relative to the level of need in Vermont?
2) Are participants connected with quality jobs?
3) Which program components are most effective?
4) How does Vermont’s performance compare to other states?
1) How many people are served by WIOA relative to the level of need in Vermont?
One way to understand a program’s impact is to examine how many people are served relative to the total number of people who are eligible to participate. While we did not have data to quantify the total eligible populations for each program, we drew on other sources as proxies to develop conservative estimates of potential participants.
Adult jobseekers: Vermont’s unemployment data provides some insight into the number of jobseekers in the state that may benefit from WIOA services, particularly Wagner-Peyser. The VDOL reports on the number of people that collect unemployment each year.6 The U.S. Bureau of Labor Statistics also tracks discouraged and marginally attached workers who are not considered part of the labor force (because they have not looked for a job in the past four weeks) but have indicated they are available to work and have looked for a job in the previous 12 months.7 Taken together, unemployed persons, discouraged workers, and marginally attached workers represent a significant group of Vermonters that could benefit from Wagner-Peyser.
Wagner-Peyser also serves participants who are currently employed; approximately 20% of participants who utilize Wagner-Peyser services are employed at the time they seek services.8 Therefore, Figure 1 underestimates the total pool of potential participants for Wagner-Peyser.
Identifying the pool of potential participants for the Adult program is particularly challenging due to the large number of priority groups. Given the focus on participants who receive public assistance and low-income adults, we looked at enrollment in the Reach Up program to provide a reference point. In federal Fiscal Year 2018 (October 2017 to September 2018), 2,583 adults were enrolled in Reach Up.9
Dislocated workers: At both the federal and state level, employers are required to notify the VDOL in the event of mass layoffs.10 In Program Year 2018 (July 2018 to June 2019), the VDOL received notices from 15 employers who were planning to lay off a total of 880 employees.11 The data collected by the VDOL is not a comprehensive list of all the layoffs in the state and skews towards larger organizations,
12 but it serves as a starting point to understand the number of dislocated workers.
Disconnected youth: Drawing on the US Census Bureau American Community Survey, the Social Science Research Council estimates that in 2018, Vermont had 6,300 youth (16-24 years old) who were not in school and not working, also called disconnected youth. Nationally, approximately one third of disconnected youth live in poor households.13 The WIOA Youth program prioritizes serving youth with barriers to employment, with a focus on low-income and out-of-school youth. Taken together, this suggests there are approximately 2,100 young Vermonters who may be eligible for the WIOA Youth program. See sidebar for more details.
Based upon the limited information available, we conclude that the Title I and III programs serve a small percentage of the eligible population of Vermonters. There is reason to believe that the eligible population for Wagner-Peyser and Dislocated Worker programs is significantly understated for the reasons noted above. It seems likely that not all eligible Vermonters necessarily need these services to successfully participate in the labor market. Nonetheless, there may be opportunity for the VDOL to reach a greater number of participants in need of support.
States work with the USDOL to develop negotiated performance targets for each metric, informed by a statistical model that intends to account for historical performance, anticipated participant demographics, and economic conditions. Wage records are the primary data source for determining employment and earnings. In Program Year 2018, Vermont met or exceeded the negotiated targets.
However, we encountered three challenges when interpreting this data. First, using wage records as the primary source to determine employment status provides limited information about job quality and, in some cases, may be misleading. For example, in our analysis, a participant in the Adult program had quarterly earnings of $58 after exit and was considered employed. While earnings are one indicator of job quality, the lack of data regarding hourly wages, number of hours worked, benefits, and participants’ progression along a career pathway limit our ability to assess whether participants are connected to high quality jobs.
Second, in some instances, the large gap between target and actual performance raises questions about whether all targets provide a meaningful basis of comparison. For example, as shown in Figure 3, the targets for quarterly earnings in the Adult program are significantly lower than actual performance, and the gap between target and actual performance has grown. The flat target earnings from 2017-2019, when compared to actual earnings, suggest the model underestimates participants’ earning potential, does not reflect the VDOL’s ability to improve service delivery, and is divorced from Vermont’s real experience.
Third, the VDOL only reports performance at the program level which can mask how sub-groups are impacted within each program. For example, in Program Year 2019, the USDOL reported that 67% of all Vermont Youth participants were employed or in an education or training program two quarters after program exit, but the rate was only 50% among Youth ages 22 to 24. The median quarterly earnings two quarters after exit was $4,435 for adults with a disability compared to $6,969 among all adult participants.16 This type of sub-group analysis could inform how the VDOL directs resources.
Nonetheless, earnings data is important because the Workforce Development Division strives to connect participants with career pathways that provide a livable wage.17 As such, we conducted a more detailed analysis of earnings data among participants who exited their WIOA program in calendar year 2018. We looked to see whether participants found jobs that paid a living wage and whether participants found jobs that paid as well or better than their prior employment.18
We first examined participants’ median quarterly earnings after program exit to see how they compared to the Vermont Livable Wage. 19 The median wage for participants in the Dislocated Worker program was above the Livable Wage, but below for participants in the Adult and Wagner-Peyser programs.
Among participants who exited in 2018
Examining the distribution of earnings provides a more complete picture of how all participants fared, including participants with no earnings. Across all programs, a significant portion (20% to 34%) of participants in our sample had no quarterly earnings two quarters after program exit, and 14% to 29% had quarterly earnings below $4,999 (annualized to less than $20,000 per year). 20 Low earnings could be a result of low wages, late-in-the-quarter employment starts, or of less than full-time employment.
We also examined how participants’ earnings two quarters after exit compared to their preprogram earnings. Participants fell into five categories:
1) Participants with no preprogram earnings that gained employment two quarters after exit
2) Participants with preprogram earnings that had higher earnings two quarters after exit
3) Participants with preprogram earnings that had lower earnings two quarters after exit
4) Participants with preprogram earnings that had no employment two quarters after exit
5) Participants with no preprogram earnings and no employment two quarters after exit
In the Adult program, 58% of participants in our sample had higher earnings or gained employment following WIOA, while 42% had lower earnings, lost employment, or remained unemployed. In the Dislocated Worker and Wagner-Peyser programs, the majority (61% and 59% respectively) had lower earnings, lost employment, or remained unemployed after exit.
Among participants who had both preprogram earnings and earnings two quarters after exit, earnings increased in the Adult and Wagner-Peyser programs but decreased in the Dislocated Worker program.
While the increased earnings in the Adult and Wagner Peyser programs are still below the Vermont Livable Wage, for some participants a low paying job can still be an important step on the pathway to a Livable Wage. Furthermore, participants’ earning potential is impacted by many factors, some of which are outside of the control of the VDOL. For example, if a highly skilled participant in the Dislocated Worker program was laid off due to a manufacturing business closure, there may not be similarly high paying job opening in their region.
3) Which program elements are most effective?
A variety of services are offered across the Title I programs, but they fall into two broad categories: career services and training. In Vermont, career services are offered primarily by the Workforce Development staff at the regional Career Resource Centers and include services such as job search assistance, case management, and development of an Individual Employment Plan. Training services include occupational skills development through classroom training and on-the-job training. In our sample, all participants received career services, but only a portion received training. In the Adult and Dislocated Worker programs, participants who received training generally had higher earnings two quarters after exit than those who did not receive training.
The VDOL is required to track whether participants who receive training attain a credential (e.g., certification, licensure, degree) and/or make progress towards a skill or credential, called a measurable skills gain. In our sample, 74% of Adult participants who received training attained a credential and 84% had measurable skills gain. In the Dislocated Worker program, 68% of participants who participated in training attained a credential and 82% had measurable skills gain.
Outcomes among participants who received training are encouraging. However, we identified two places where training funds may not be fully utilized. In the Adult program, 20% of participants in our sample did not complete their training program and only 54% of participants were employed in a field related to their training. In the Dislocated Worker program, 18% did not complete their program and only 48% were employed in a field related to the training. We recognize that a range of factors, which are not necessarily negative, may explain these trends. For example, a participant may have left their training program because they found full-time employment and/or may not have been able to find a job in the occupational field for which they received training. However, at an average cost of close to $2,000 for training per participant, these figures raise questions about whether all of the training funds offer a good return on investment. Collecting participant feedback about what aspects of training were most valuable and, if applicable, why participants did not complete a training program could help the VDOL make more informed decisions about how to allocate training funds.
The Youth program emphasizes providing participants with work experience; no less than 20% of Youth WIOA funds must be allocated to work experience. In our sample, 82% of Youth participants had some type of work experience, including summer employment and on-the-job training. However, we found that participation in work experience was not associated with higher rates of employment or participation in education and training programs two quarters after exit. Although only 23% of youth participants in our sample attained a credential, credential attainment was associated with positive outcomes; 79% of youth participants who attained a credential were employed or enrolled in an education or training program two quarters after program exit compared to 55% of those without a credential.
4) How does Vermont’s performance compare to other states?
Vermont’s performance varies by program and metric but in comparison to other states’ performance, Vermont falls in the bottom half for most measures:
Meanwhile, Vermont’s average cost per participant in WIOA Title I programs is higher than the national average and, in the Title I programs, higher than all but one comparable state.21 Title III Wagner-Peyser spending is more similar to comparable states.
Several factors may contribute to the difference in performance across the states. For example, differences in regional labor market conditions and demographics can lead to differing results. States’ decisions about program design also impact service delivery and resource allocation. Some states deliver their Title I and Title III career services by contracting with third parties while other states, like Vermont, administer the majority of their Title I and Title III career services via State employees. In addition, some states have waivers from the federal government to alter their WIOA programs. Lastly, Vermont’s commitment to serving participants with the highest needs may contribute to their high cost per participant and lower performance. Nonetheless, Vermont may benefit from examining successful practices in other states.
Areas for Further Consideration
The goal of this investigative report is to better understand the performance of Vermont’s WIOA Title I and Title III programs. Recognizing that the VDOL operates within the context of both federal requirements and the strengths and weaknesses of Vermont’s economy, we offer the following areas for further exploration:
• Evaluate proactive outreach strategies and partnerships to increase enrollment among participants who may benefit from WIOA services. For example, coordinating more closely with Community Action Agencies could be one way to identify families with barriers to employment who would benefit from WIOA services, particularly in the Adult and Youth programs.
• Analyze outcomes among dislocated workers who are identified via the WARN Act and/or the Vermont Notice of Potential Layoffs Act but not enrolled in the WIOA Dislocated Worker program to determine whether there is need for more intensive and/or different services and training among this population. Some dislocated workers may have found other appropriate employment or opted to retire. However, if a need for additional support is identified, conduct more intensive outreach to engage with a greater percentage of dislocated workers.
• Explore opportunities to collect additional data and conduct more rigorous evaluation activities to inform resource allocation and assess program performance.
• Develop formal mechanisms to collect participant feedback regarding which program elements are most useful or effective. For example, a survey of participants could provide helpful feedback about why participants did not complete their training program or whether skills from their training program helped prepare them for their employment (even if it was not in the same field as their training).
• Review opportunities for which a federal waiver may increase flexibility to address Vermont-specific needs.
While outside of the scope of this inquiry, a future area of study may include how the VDOL trains their staff to best serve WIOA Title I and Title III participants.
Investigative Report Note: An investigative report is a tool used to inform citizens, the Legislature, and State agencies about issues that merit attention. It is not an audit and is not conducted under generally accepted government auditing standards. Unlike an audit, which contains formal recommendations, investigative reports include information and possible risk-mitigation strategies relevant to the topic that is the object of the inquiry.
Footnotes
1 Congressional Research Service. The Workforce Innovation and Opportunity Act and the One-Stop Delivery System. Updated January 21, 2021.
2 State Workforce Development System Report of Findings & Recommendations: Act 69 (2017), Sec. E.1 January 15, 2018.
3 At the national level, Title I and Title III of WIOA are administered by the US Department of Labor (USDOL). The USDOL funds Title I and Title III programs through formula grants to the states.
4 Low-income is defined in the Workforce Innovation and Opportunity Act. 29 .U.S.C § 3102 (36) (2014).
5 PY19 Vermont State Data Book, Table II-1 and Table III-1.
6 VDOL, Economic and Labor Market Information Division. Statistical Record of Vermont Unemployment Insurance Program, Calendar Years 2000 to 2019. December 2020. See Table 6, “First Payments.”
7 U.S. Bureau of Labor Statistics, All Marginally Attached Workers for Vermont. Retrieved from FRED, Federal Reserve Bank of St. Louis. September 2021.
8 PY19 Vermont State Data Book, Table VI-1.
9 Reach Up is Vermont’s Temporary Assistance to Needy Families program. In Federal Fiscal Year 2018, Reach Up provided cash assistance, case management, and support services to Vermont families whose income was generally below 50% of the Federal Poverty Level.
Vermont Department for Children and Families. Evaluation of Reach Up. January 2019.
10 Under the Federal WARN Act, employers with 100 or more employees must notify states in the event of mass layoffs (500 or more persons or 33 percent of the employer’s active workforce). Under the Vermont Notice of Potential Layoffs Act (21 V.S.A. § 413), employers must notify the state if they anticipate layoffs of 50 or more employees within a 90-day period. For more information, see the VDOL website.
11 When the VDOL is notified about layoffs, they provide Rapid Response services to support affected employees. Rapid Response, which is funded through the WIOA Dislocated Workers Funds, can include on-site support to help employees connect with unemployment insurance and to provide information about the Career Resource Centers. The VDOL alerted the SAO that the list of layoffs provided to the SAO may be incomplete.
12 The Bureau of Labor Statistics tracks the number of dislocated workers by region; however, the small sample size in Vermont does not allow for state-level analysis.
13 Lewis, Kristen. A Decade Undone: Youth Disconnection in the Age of Coronavirus. New York: Measure of America, Social Science Research Council, 2020.
14 USDOL, “Workforce Innovation and Opportunity Act.” Accessed August 12, 2021.
15 The USDOL’s use of the term “unsubsidized employment” refers narrowly to employment for which an employee or employer doesn’t receive a direct cash payment to support wages. Many of these jobs receive substantial indirect subsidies because the pay rate is low enough that the employee is eligible for the Earned Income Tax Credit, Medicaid, Food Stamps, and other entitlement programs. The Youth program measures whether participants are 1) employed, 2) in an education program, or 3) participating in training activities.
16 PY19 Vermont State Data Book, Tables II-25, II-29, V-22, and V-24.
17 Vermont Workforce Development Division Annual Report, 2020. See page 7, Goal 2.
18 Consistent with the USDOL methodology, we averaged preprogram earnings from two and three quarters prior to program entry as the baseline for comparison. We did not include the Youth program in this analysis given the program’s dual focus on employment and educational opportunities and the fact that many young people may be entering the workforce for the first time.
19 We opted to use the 2018 Livable Wage data to align with dates in our sample.
20 As noted above, when wage data is not available, earnings are entered as zero which may skew the data.
21 Vermont is a “single-area” state; the VDOL looks to other single-area states to compare program design and performance. The other single-area states are New Hampshire, North Dakota, South Dakota, and Wyoming.
Source: MONTPELIER, VT – Vermont State Auditor 9.29.2021