Section 6: Evaluation
Step 1: Evaluation Planning
Monitoring and evaluating health improvement effortprovides accountability, while also allowing you to share successes with others, learn from your mistakes and make changes to the course of action as necessary. Evaluation is the systematic collection, analysis, and reporting of information about a program in a way that enables practitioners to learn from their experience. Programs and initiatives need to be evaluated based on goals and objectives set during the planning process. There are different levels of evaluation and each offers different types of evaluation information.
Formative Evaluations, which are used in the program planning stages to ensure that the needs of the intended audiences are properly
addressed and that the appropriate materials and procedures are used to implement the program.
Process Evaluations, which track and report a program to help understand what is going well and what isn't, and to make decisions
about how to improve the design and implementation of the program in the future.
Outcome or Impact Evaluations, which determine if the program met its stated goals and objectives. Outcome evaluations also measure
specific effects and/or changes (both intended and unintended) to determine if the program made a difference in the workplace.
PDF Data Dashboard
PDF 10 Steps to Evaluation
Step 2: Evaluating Reach
Tracking employee participation and progress is a basic and key way to evaluate your efforts.
- Track attendance of participants in stress management programs
Baseline
- List current options for employees through worksite and identify number of employees using each option (ie participation in classes). Examples:
- Number of classes and participation in these programs
- Availability of educational materials on emotional health
- Number of emotional health-related policies
- Number of environmental strategies
- Number of partnerships with community resources for stress-free lifestyle
- Determine costs of current company emotional health programs such as:
- Capital investment in building or facilities such as break rooms
- Staffing, equipment, and space
- Conduct survey of employee satisfaction with current workplace culture
Process
- Reassess barriers to emotionally healthy workplace
- Document steps taken and progress toward implementing each intervention selected
- List numeric goals in each form of intervention within a designated time period (e.g., 12 months from startup):
- Employee reach (e.g., number of educational pamphlets distributed)
- Employee participation (e.g., number of desired participants in classes)
- Describe timeline for implementation of each planned intervention (e.g., length of time and timing of tasks to develop, initiate, and conduct a mass campaign)
- Create a baseline budget for new interventions including classes, instructors, classroom space, printed and online educational material, etc
- Identify opportunities for new partnerships with community groups who provide programs
- Reassess employee satisfaction
Outcome
Step 3: Evaluating Effectiveness
The effectiveness of programs depends on the intensity of program effort and the use of multiple interventions. A rule of thumb is that the more programs implemented together as a package or campaign, the more successful the interventions will be.
Baseline
Determine levels of employee stress from employee health survey or health risk appraisal measures. One of tools you can use to assess how many of your employees at risk of mental health issues is your Health Risk Appraisal. You can utilize your HRA data to see actual numbers of employees who are experiencing high stress or reporting depression and translate those actual numbers into percentages of your workforce. Changes in this percentage over time will give you a big picture indication of whether you are addressing the problem effectively. This is a long-term problem, though, and will take time and a variety of complementary approaches. Specific, interim measures can help determine if you are on the right track with your efforts, and which components of your activities are the most effective.
- Determine baseline percentage of employees with health conditions (program participants versus non-participants) affected by stress including heart disease.
- Determine employee knowledge, attitudes, and beliefs about stress and mental health issues
- Evaluate employer’s current knowledge of the health benefits of good emotional health
- Measure employee’s knowledge about mental health issues
- Assess employee awareness of existing workplace programs, policies, and benefits
- Determine the number of employees who are thinking about making changes or state that they want to change their stress levels.
Process
Periodic repeats of baseline measures
Outcome
Assess changes in employee stress levels such as:
- Changes in health conditions such as heart disease assess the effects of lowered stress
- Compare changes between program participants before education and other programs are initiated and after operation of these programs
- Assess changes in employee knowledge, attitudes, and beliefs
- Assess changes in employee awareness of existing workplace programs, policies, and benefits
PDF CDC Issue Brief
PDF Chenoweth Evaluation
Step 4: Evaluating Cost and Productivity
Healthier employees are less likely to call in sick. Companies can sometimes assess sick day use as the most direct measure to determine whether health programs are increasing worker productivity.
Baseline
- Determine the average number of sick leave days per employee over the previous 12 months for health conditions affected by stress or depression
- This measure may be less useful if there has been a large increase or decrease in numbers of employees over the past 12 months
- Determine the costs for worker absenteeism including costs of replacement workers, costs in training replacement workers, and loss and delay in productivity
- Determine time employees spend during working hours participating in stress related worksite programs
Process
- Re-assess the average number of sick days per employee at the first follow-up evaluation
- Periodic repeats of other baseline measures
- Re-assess the average number of sick days per employee at the first follow-up evaluation
Outcome
- Assess changes in the average number of sick days per employee in repeated follow-up evaluations
- Assess changes in time employees spend during working hours participating in stress related worksite programs
- Assess changes in costs from baseline
Health care cost measures for mental health
In contrast with the worker productivity costs described above, health care costs are measures of the direct medical expenses of providing employee health care and preventive health programs.
Baseline
- Determine current health care use and costs, including pharmaceuticals, counseling, and inpatient and outpatient care, for mental health diseases and conditions.
- Determine the current health care use and costs for clinical services (if covered), including counseling, prescribed supplements and other medication, outpatient and inpatient care for employees
- Determine the health care use and costs of program participants before education and other programs are initiated and after operation of these programs
Process
- Periodic repeats of baseline health measures
Outcome
- Assess changes in health care use and costs from baseline
- Assess changes in the type, use, and costs of employee health benefits related to clinical services (if covered)
- Compare health care use and costs of program participants before education and other programs are initiated and after operation of these programs
Website: Depression Calculator Shows Cost of Untreated Illness
Website: Depression Calculator
PDF Welcoa ROI Calculator
PDF CFO Framework for Mental Health