Summary of Client’s Challenge

The International Municipal Signal Association (IMSA) is globally recognized as the leader in public safety certifications, having issued over 100,000 certificates in public safety. IMSA wanted to increase rigor in their certifications and provide radically updated training to their members as they prepared to take their certification exam in a specific discipline, such as Work Zone, Signs and Markings and Traffic Signal 1. To build workforce capacity in these areas, Change by Design offered a multi-level approach.

In the initial discussions with IMSA, Change by Design identified the initial products that would meet their budgetary requirements and build the foundation for later training development. These products included:

  • Legally Defensible Assessments/Exams

  • Standard Operating Procedure Workflows

  • Detailed Course Outlines

To best ensure that assessments/exams were legally defensible and rigorous, Change by Design performed the following activities in close collaboration with key subject matter experts:

Job/Task Analysis

Conducted a job/task analysis (J/TA) for each discipline, identifying all tasks, sub-tasks, and steps in each process in which they were engaged. In this JTA, we also identified all essential knowledge and tools that would be used to enable accomplishment of each step for each sub-task.

REPRESENTATIVE SURVEY     

Surveyed the greater group of qualified individuals in the U.S., Canada, and Mexico to validate each J/TA. We analyzed the results to determine - for each skill area - the weighting of number of questions out of 100 to be asked to cover it. The results were produced in the “exam blueprint”.

DEVELOP A CURRICULUm (dacum)

Built a DACUM (Developing a Curriculum) chart for each discipline. This DACUM is a means of researching and analyzing a particular job resulting in a chart listing duties, tasks and related job information. It enables the sequencing and ratifying of the following elements of each task:

  • Difficulty: Extent of difficulty of each task

  • Importance: Extent of how essential each task might be to the accomplishment of the job

  • Frequency: How frequently the tasks must be performed in the conduct of the job

AUTHORING

Conducted item-writing sessions that served as the draft test items using an item capture software.

DEVELOP EXAM FORMS

Developed two exam forms for each discipline, which means that there would be two sets of test items covering the same material but were different questions and responses. We then beta-tested the exam forms with the broader qualified community.

ANALYSIS/REVIEW

Conducted an item analysis/review to throw out poorly performed questions and determine the passing score for each discipline exam.

Concurrently, after the conduct of the J/TA and development of the DACUM chart, the Change by Design team proceeded to engage in the development of standard operating procedure workflows, branded to IMSA branding standards. Standards included:

  • Prerequisite Knowledge

  • Specific Tools and Resources Used

  • Skills, Abilities and Attributes

  • Numbered Overarching Steps for Each Workflow

  • Color-Coded Workflows: Actions ordered and numbered, with all decision points and alternative choices identified.

Once we had developed these SOPs, we turned our attention to the design of each course. For each of the ten disciplines on which we worked, we built a detailed course outline that served as the guiding document for the development of a course for each discipline.

COURSES FOR EACH OF 10 DISCIPLINES

IMSA then decided to invest in the course materials and hired Change by Design to build them. Altogether, the Change by Design team built 10 discipline courses for a total of 45 hours of instructor-led training. Working closely with subject matter experts, we developed these courses in branded PowerPoints slide decks, providing clear instructions on what to say and when to say it, alignment to federal standards and guiding documents, and skills-focused activities that required a deep understanding, through the instruction, of all related knowledge. We further provided a participant guide for each course that included all slides and places for them to take notes.

Contact Change by Design today if you need exams that are legally defensible for granting certifications, as well as custom instructor-led (ILT), virtual instructor-led (vILT) or eLearning trainings built and deployed through your learning management system (LMS).