You've designed what you believe is an excellent program. Now it's time to "put up or shut up"...meaning it's time for you to pilot the program and show its effectiveness to the organization. You've followed all the "rules" to create the perfect pilot program - participants, setting, observation, etc. Now it's "show time."

At the start of the program, tell participants that:

• They are the pilot group for the program.
• You are looking for feedback for this program.
  Comments will be analyzed to determine changes to the program.
• Honest and candid feedback is important.
• They will complete a concise, short evaluation after each module for immediate feedback.
• They will participate in a oral program debrief, no longer than 30 minutes, at the end of the program.
• They will be asked to complete the “standard” evaluation form as they proceed through the entire program.

Be sure to caution participants that you don't want feedback during the presentations - only when you request it. If they want to jot ideas down immediately, they should use the "standard" evaluation form. I don't know about you, but without this warning participants feel that they can audit and discuss the effectiveness of the program during the program rather than "experience" the whole program. Don't be derailed.

During the program, distribute a concise, short evaluation after each module or segment to collect immediate feedback.

After the program, the debrief should be done by the instructional designer or someone who has not facilitated the program. The facilitators should never do the debrief or be present during the debrief. In fact, sponsors, or other people who have a vested interest in the class, should not be present. You want to provide an environment where participants are comfortable being frank and providing valuable feedback. The participants of the pilot debrief should be told that the debrief:

• Will be tape recorded.
• Should last no longer than 30 minutes.
• Responses will be kept confidential.
• Results will be summarized in a written report.

That being said, keep your word. Record the debrief so you can keep the feedback to 30 minutes. Keep on track with your questions and be willing to collect additional information after the 30 minutes, if participants have more to add; however, allow participants to leave if they have shared what they wanted, or needed, to say.

Again, remember all people with "ownership" in the program should not be present in the room during the actual debrief.

After the debrief with participants, conduct a debrief with the individuals who have "ownership" of the program (facilitators, designers, observers, stakeholders, sponsors). Consider conducting a debrief similar to the one facilitated for participants with questions specific to the group. Remember, you must maintain the confidentiality of the participants in this meeting - at all costs.

Also during the "ownership" debrief, schedule a revision meeting for a day and time that will work for all those involved. Ensure that at least a half-day is scheduled to thoroughly review the debrief feedback from participants and discuss next steps. Before adjourning this debrief, discuss any next steps needed to be completed by the group prior to the revision meeting.

During the Revision Meeting:

• Review the results of the summarized, oral debrief written report.
• Discuss any other feedback from other interested parties, especially those resulting from the "ownership" debrief.
• Determine next steps.
• Commit to follow up dates for completion of the next steps.

Pilots help ensure that effective and successful programs are facilitated in your organization. Taking the time and providing the resources to conduct a thorough pilot for new and/or revised programs can ensure that participants are getting what they truly need to move the organization forward. The costs are too great, if effective and successful programs are not required. Take the steps to ensure effective and thorough pilots are executed in your organization. It pays!