Rationale

How do you know the staff development you design is getting the results you intended?
In the previous module, you learned about SMART objectives. The designer must be proactive in determining how specific objectives will be measured to assure that staff development really makes a difference.

Broward County HRD Department requires that the designer of the staff development activity align the evaluation to the specific objectives.

DEFINITION

Evaluation is a process of reviewing and analyzing data in order to make informed decisions about the effectiveness of a staff development event.

Historically as staff developers we have documented our work, rather than evaluating it. We have counted the numbers of participants, reported the calendar or schedule of programs, calculated the cost and sometimes the reactions of the participants.

Review the shift in thinking about evaluating staff development.

Shift in Staff Development Evaluation

It is important for the evaluation to consider the context, process and content of a staff development initiative, as all may be important to the results and impact on student achievement or performance. Staff development evaluations of the past often focused only on process.

You may remember the NSDC Standards you reviewed in the Introductory Module were organized by context, process and content. If you are not familiar with the terms context, process and content in relation to staff development. Click on the links to learn more.

Reflect on the design you are producing for this course.
Consider: How can you move beyond the simple process evaluations often used in the past to the more extensive evaluation of results or impact required today? 

To learn more about thinking of staff development planning and evaluation in terms of the content, process and context read Tom Guskey’s brief article on Backwards Planning. After reviewing the article, close the window and return to the module.

Now let's look at how the thinking about the evaluation of adult learning opportunities has evolved to the new requirements outlined by Broward County.  

History of Evaluation

What are your ideas about the kind of evaluation needed to measure your staff development project for this course? Understanding the evolution of thinking about evaluation may be helpful as you plan for the type of evaluation you will need to use.

Kirkpatrick’s thinking suggests there are four levels of evaluation:

Click on each to learn more about this pioneer’s thinking about evaluation.

Guskey suggests there are five levels of professional development evaluation:

Click on each to learn about this contemporary thinking about professional development evaluation in educational settings.

The Broward County evaluation model synthesized the work of Kirkpatrick and Guskey and the HRD Department identified three possible levels of evaluation:

Click on each to review the Broward County Schools’ HRD Department definition.

The bottom line for staff development evaluation today is to assure that the development of teachers, administrators, and other educators is focusing on education’s bottom line: improved student performance.

To understand the current thinking about staff development evaluation that links to student performance, read the National Staff Development Council’s explanation of their evaluation standard. Other links for additional resources are provided at the end of the article. Select one or two articles of interest to you for further development of your understanding.

Submitted Activity 3-A
Respond to the following questions and submit email or US Mail or Pony Mail your responses to your learning facilitator.

At which of the three levels defined by Broward County HRD Department have you typically evaluated staff development activities you have designed and why have you done so?

What new ideas have you gained about impact evaluation: linking the staff development you design to improved job performance or student outcomes?

Do you think it is feasible to link staff development results to job performance or student outcomes?

Why or why not?

 


Data Collection

To determine if the desired changes occurred, the designer must identify what data to collect and the most appropriate process to gather the data.

Broward County HRD Department provides examples of common data collection methods for each of the three levels of evaluation which may provide ideas to the designer considering the most appropriate approach for data collection for your project for this course.


Think about the design project you are developing for this course.

  • What are the desired changes resulting with the learners?
  • What might be the most appropriate process to gather the data?

Designers may benefit by viewing an example. Using the example of the Broward County Upward Bound Leadership Team Training,

  • One of the desired changes: more effective leadership team meetings
  • Possible data collection method: an observation by a trained observer at a school’s leadership team meeting several months after the training using a checklist to observe the changes in job performance among the team members.

Click on Review icon to see an example.

Individual approaches or combinations may be planned for gathering evaluation data by the designer. PDF files are provided if you wish to print out the information. Be aware that in Adobe Acrobat you may zoom in and out of a document. After printing a specific PDF hit the back button on your browser to navigate back to the main module. For other ideas for data collection for your project, click on each of the following options to learn more:

To further your understanding of evaluation of staff development, complete the following activity. Think about and write your response on a separate sheet of paper. Then click on expert opinion and compare your answer with the expert. There may be more than one correct answer.

Now review your specific objectives for your design project.
Consider: How will you know if the desired results have been accomplished?
Consider: How will you gather data to show these results?