In this section:

Section 6: Evaluation

Step 1: Evaluation Planning

Once a company has conducted assessment and planning of nutrition programs, and developed the specific tasks of implementation for these programs, it is time to develop the evaluation plan. The evaluation plan should be in place before any program implementation has begun.
Metrics for worker productivity, health care costs, heath outcomes, and organizational change allow measurement of the beginning (baseline), middle (process), and results (outcome) of workplace health programs. It is not necessary to use all these metrics for evaluating programs. Some information may be difficult or costly to collect, or may not fit the operational structure of a company. These lists are only suggested approaches that may be useful in designing an evaluation plan.

These measures are designed for employee group assessment. They are not intended for examining an individual’s progress over time, which would raise concerns of employee confidentiality. For employer purposes, individual-level measures should be collected anonymously and only reported (typically by a third party administrator) in the aggregate, because the company’s major concerns are overall changes in productivity, health care costs, and employee satisfaction.

In general, data from the previous 12 months will provide sufficient baseline information and can be used in establishing the program goals and objectives in the planning phase, and in assessing progress toward goals in the evaluation phase. Ongoing measurements every 6 to 12 months after programs begin are usually appropriate measurement intervals, but measurement timing should be adapted to the expectations of the specific program.

One of the best ways to tools you can use to assess how many of your employees are using tobacco is your Health Risk Appraisal. You can utilize your HRA data to see actual numbers of employees who are using tobacco and translate those actual numbers into percentages of your workforce. Changes in this percentage over time will give you a big picture indication of whether you are addressing the problem effectively. This is a long-term problem, though, and will take time and a variety of complementary approaches. Specific, interim measures can help determine if you are on the right track with your efforts, and which components of your activities are the most effective. 
 

Step 2: Evaluating Reach

Tracking employee participation and progress is a basic and key way to evaluate your efforts.

  • Track attendance of participants in tobacco cessation programs

Baseline 

  • List current tobacco treatment options for employees through worksite and identify number of employees using each option. Examples:
    • Number of classes and participation in these programs
    • Availability of educational materials
    • Number of tobacco-related policies
    • Number of environmental strategies such as removal of ash-trays or smoke-free areas 
    • Number of partnerships with community resources for nutrition such as American Cancer Society, local hospital or health department
     
  • Determine costs of current tobacco programs such as:  
    • Staffing, equipment, and space
    • Alternative treatments
    • Medical/Prescription costs 
  • Conduct survey of employee satisfaction with current workplace supported tobacco programs

Process 

  • Reassess barriers to employee cessation
  • Document steps taken and progress toward implementing each intervention selected
    • List numeric goals in each form of intervention within a designated time period (e.g., 12 months from startup):
      • Employee reach (e.g., number of educational pamphlets distributed)
      • Employee participation (e.g., number of desired participants in cessation classes)
       
    • Describe timeline for implementation of each planned intervention (e.g., length of time and timing of tasks to develop, initiate, and conduct a mass campaign)
    • Create a baseline budget for new interventions including classes, instructors, classroom space, printed and online educational material, etc
    • Identify opportunities for new partnerships with community groups who provide nutrition programs (e.g., ACS, local health department, local hospital, etc.)
     
  • Reassess employee satisfaction regarding workplace supported programs.

Outcome 

  • Measure reductions in the number and type of employee barriers to tobacco cessation 
  • Assess changes in workplace tobacco cessation programs
    • Measure changes in the number of program options for employees through the worksite and changes in employee participation using each option before and after the program or campaign. Examples:
      • Number of new programs developed and offered to employees and participation in these programs
      • Number of new educational materials developed and made available to employees
      • Number of tobacco-related polices developed and implemented compared to baseline
      • Number and type of new environmental support changes made 
      • Number of new partnerships with community groups created to enhance access and opportunity for employees to be smoke-free 
       
    • Assess changes in program costs from baseline
      • New capital investments made
      • Increases in staffing or materials needs due to new program offerings
      • New incentives or changes in existing incentives based on employee participation
      • Changes in costs for benefit coverage 
       
    • Assess changes in survey responses for employee satisfaction following implementation of a workplace supported programs and compare with baseline

PDF Sample Encounter Form 
 

Step 3: Evaluating Effectiveness

 
The effectiveness of tobacco programs depends on the intensity of program effort and the use of multiple interventions. A rule of thumb is that the more programs implemented together as a package or campaign, the more successful the interventions will be.


Baseline  

  • Determine employee tobacco use from employee health survey, health risk appraisals or nicotine/cotinine testing. Examples include:
    • Determine the percent of employees who report tobacco use or exposure to secondhand smoke 
    • Determine the percent of employee who test positive for cotinine/nicotine
     
  • Determine baseline percentage of employees with health conditions (program participants versus non-participants) affected by tobacco use including heart disease, cancer, and lung disease/COPD.
  • Determine employee knowledge, attitudes, and beliefs about tobacco use.
    • Evaluate employer’s current knowledge of the health benefits of tobacco cessation 
    • Measure employee’s knowledge about tobacco 
    • Assess employee awareness of existing workplace programs, policies, and benefits
    • Determine the number of employees who are thinking about making changes or state that they want to change their tobacco habits

Process 

  • Periodic repeats of baseline measures

Outcome  

  • Assess changes in employee tobacco use such as:
    • Changes in the percentage of employees reporting tobacco use or exposure to secondhand smoke 
    • Changes in the percentage of employees testing positive for nicotine/cotinine 
     
  • Changes in health conditions such as asthma, lung disease, blood pressure, and even cholesterol levels can assess the effects of tobacco cessation 
    • Compare changes between program participants before education and other programs are initiated and after operation of these programs
     
  • Assess changes in employee knowledge, attitudes, and beliefs about tobacco
  • Assess changes in employee awareness of existing workplace programs, policies, and benefits

PDF Tobacco Cessation Questionnaire
PDF Tobacco Exposure Assessment
Website: Tobacco Technical Assistance Consortium 

Step 4: Evaluating Cost and Productivity

Productivity 

Healthier employees are less likely to call in sick. Companies can sometimes assess sick day use as the most direct measure to determine whether health programs are increasing worker productivity.

Baseline  

  • Determine the average number of sick leave days per employee over the previous 12 months for health conditions affected by tobacco use such as pneumonia, bronchitis, and asthma     
  • Determine the costs for worker absenteeism including costs of replacement workers, costs in training replacement workers, and loss and delay in productivity
  • Determine time employees spend during working hours participating in tobacco related worksite programs

Process  

  • Re-assess the average number of sick days per employee at the first follow-up evaluation
  • Periodic repeats of other baseline measures

Outcome  

  • Assess changes in the average number of sick days per employee in repeated follow-up evaluations
  • Assess changes in time employees spend during working hours participating in tobacco related worksite programs
  • Assess changes in costs from baseline

Health Care Cost Measures

In contrast with the worker productivity costs described above, health care costs are measures of the direct medical expenses of providing employee health care and preventive health programs.

Baseline  

  • Determine current health care use and costs, including pharmaceuticals, counseling, and inpatient and outpatient care, for the diseases and conditions affected by tobacco behavior.
  • Determine the current health care use and costs for clinical tobacco services (if covered), including counseling, prescribed supplements and other medication, outpatient and inpatient care for employees
  • Determine the health care use and costs of program participants before education and other programs are initiated and after operation of these programs

Process  

  • Periodic repeats of baseline health measures

Outcome  

  • Assess changes in health care use and costs from baseline
  • Assess changes in the type, use, and costs of employee health benefits related to clinical tobacco services (if covered)
  • Compare health care use and costs of program participants before education and other programs are initiated and after operation of these programs

 PDF ROI Calculator
Website: Well Steps ROI Calculator
Website: American Health Insurance Plans Business Case for Smoking Cessation  

Website: American Cancer Socieity Workplace Solutions 
PDF Productivity