How Organizations Leverage e-Learning, Assessments and Competency Management

A Focus Report by The MASIE Center For the exclusive use of e-Learning CONSORTIUM Members

April 2002

PO Box 397 Saratoga Springs NY 12866 USA (518) 350-2200

How Organizations Leverage e-Learning, Assessments, and Competency Management

INTRODUCTION
We asked the question . . . . How Do Organizations Leverage e-Learning, Assessments, and Competency Management? In response, a Working Group of CONSORTIUM members was formed to share their experiences and to learn from one another. In this report, we present two case studies of CONSORTIUM member organizations that detail the unique combination of processes, approaches, tools/systems/technology, content, vendors, resources, and models they use to leverage e-Learning, assessments, and competency management. As a Working Group, we found it helpful to establish a foundation for these case studies . . . What are Assessments, Competencies, and Competency Management? How do organizations use them? Competencies are characteristics that result in superior performance. Organizations establish and use competencies as a performance benchmark (behaviorally-based performance standards and outcomes). Competency Management is a process used by organizations to identify desired competencies and competency gaps and to manage how and when those competencies will be developed. Assessments, whether in the form of surveys, tests, 360 feedback, or performance evaluations, are a means of evaluating knowledge, skills, and/or competencies. By determining what a person knows, what a person can do, what a person can accomplish (behaviors and results); organizations can identify and close knowledge, skill, and competency gaps. The simplest form of assessment is a self-assessment. At the opposite end of the spectrum is performance observed, evaluated, and rated by experts based on an established set of standards. So . . . How Do Organizations Leverage e-Learning, Assessments, and Competency Management? The success of an organization depends on the ability of each of its employees to effectively perform and meet goals. Organizations are using e-Learning, assessments, competencies, and competency management to support the achievement of performance goals individual performance, which leads to organizational performance, and which, ultimately, helps achieve business goals. Specifically, organizations are using the web, Learning Management Systems, test engines, HR systems, and e-Learning content to assess, view, plan, update, develop, deliver, manage, track, and report the competencies, gaps, development, progress, and performance of their people. The experiences of the following two organizations illustrate this. For Texas Instruments worldwide sales force to remain effective, they needed much more technical competence than had been required of them in the past. So Texas Instruments developed their own competency program built upon assessments and delivered online. They built their own Learning Management System (LMS), devised a way to rapidly produce their own online content (in as little as 24 hours!), created their own test engine for building assessments, developed their own competencies, created a performance evaluation process and tools, put together customized management reports, and added a Back Office so that

The MASIE Center e-Learning CONSORTIUM

Page 2

How Organizations Leverage e-Learning, Assessments, and Competency Management their Sales & Marketing managers could manage and track the competency assignments of their own employees. An international manufacturing company wanted to standardize the work processes of its 6,000-member field workforce located in field offices throughout North America. They also wanted to achieve faster, time to performance for new hires. So, they developed and implemented a performance-based qualification system in order to achieve these goals. The underlying principle of a qualification system is that individuals are observed and evaluated by a master performer doing their job tasks under normal working conditions to ensure they are meeting specific standards of performance. The qualification system consists of job models, performance models, qualification instruments, qualification events, an evaluator/master performer, individuals whose performance is being evaluated, a website, eLearning, a Learning Management System, and an HR system. Over the next several pages, you will read the details of these case studies as these companies share: Their business situations Their solutions a unique combination of processes, approaches, tools/systems/technology, content, vendors, resources, and models. How they use and leverage e-Learning, Assessments, Competencies, and Competency Management How they developed and launched their solutions How effective they have been Their challenges and the lessons they learned along the way

We also suggest that you read our report How Organizations Use Learning and Technology for Leadership, Management, and Succession for more insights into how organizations use eLearning, assessments, and competency management for leadership, management, and succession. We hope that you learn a great deal from the experiences detailed in these case studies. Our sincere thanks to the champions from each of these organizations for sharing their experiences with us and with all of you. Our thanks to Connie Latson for serving as editor/writer of this report. Sincerely, Elliott Masie & Helen Rinaldi The MASIE Center e-Learning CONSORTIUM

The MASIE Center e-Learning CONSORTIUM

Page 3

How Organizations Leverage e-Learning, Assessments, and Competency Management

Building a Competency-Based Development System

Case Study Focus: Building a Competency-Based Development System Target Population: Texas Instruments Sales & Marketing organization, which includes more than 1,000 people worldwide Organization: Texas Instruments (TI) Business: Manufacturer of global semiconductors, educational and productivity solutions, and sensors and controls. Designer and supplier of real-time signal processing solutions. Champion: Clark Hise, Training Manager Business Situation: The complexity of TIs products has increased significantly over the years. For their worldwide sales force to remain effective, they needed much more technical competence than had been required of them in the past. Bringing the sales force to one central location or sending a subject matter expert to train them would be difficult to do and would take them away from their primary responsibility of serving customers. What TI needed was a flexible, rapid, and low cost solution for increasing their competencies. Business Solution: TI faced a build or buy decision. After careful analysis of Learning Management Systems (LMS), authoring tools, test engines, etc., they decided that building rather than buying was the best solution for them. Thats because at that time (1996-1999), they could not find tools that could effectively meet all their requirements. Whats more, it was much more expensive to buy a solution that did not meet their needs than to build one that did. So, TI developed their own competency program built upon assessments and delivered online. Solution Components: TI built their own Learning Management System (LMS), devised a way to rapidly produce their own online content (in as little as 24 hours!), created their own test engine for building assessments, developed their own competencies, created a performance evaluation process and tools, put together customized management reports, and added a Back Office so that their Sales & Marketing managers could manage and track the competency assignments of their own employees. In addition to these internally developed systems and content, they use third-party content from Telecommunications Research Associates (TRA), PeopleSoft for their HR system, and TEDS for Instructor-Led Training. How TI Uses Their Competency-Based Solution: A Test Engine generates a series of Assessment questions, tracks responses, provides feedback, and produces a score on how well the individual met the learning objectives. Assessments are used to test a persons knowledge and mastery of required competencies. Question types include multiple choice with one correct response, multiple choice with multiple correct responses, and true/false with provisions for fill-inthe-blank and for matching. Three types of assessments are used: A pre-test, which can only be taken once, is used to determine an individuals existing knowledge. If the individual passes the pre-test, they receive credit for their demonstrated knowledge. If they don't pass the test, they know where to focus their learning.

The MASIE Center e-Learning CONSORTIUM

Page 4

How Organizations Leverage e-Learning, Assessments, and Competency Management A post-test, which can be retaken as often as desired, assesses whether or not the individual met the learning objectives of the course. The post-test questions are the same as in the pre-test. A final exam is used to demonstrate mastery of required competencies. A database contains all the questions from the courses pre-test, post-test, and quizzes. For each learning objective associated with a specific competency, two to three questions are randomly selected from the database. There are three levels of Competencies for the Sales & Marketing sales force: Level 1 competency: Identify and qualify an opportunity Level 2 competency: Demonstrate features and benefits Level 3 competency: Implement a design

TIs competency program has evolved over time. Initially, they focused most of their efforts on the achievement of Level 1 competency because it addressed the needs of their largest population. They later expanded their focus beyond Level 1. Then, they evolved from only using online tests to observing and evaluating performance -- requiring that the sales force demonstrate their ability to apply their knowledge and competencies in a customer situation. For example, a Level 2 candidate must deliver a one-day workshop to customers to demonstrate proficiency in this competency. Mentors are members of the sales force who are proficient in a particular set of competencies. Using a Checklist, the Mentor assesses whether or not a candidate demonstrates required competencies and documents his observations to support that evaluation. For example, a Level 2 candidate must show that they: Know the content Can deliver an effective presentation to customers Are able to address customer needs Can answer customer questions Know the technology Can use and demonstrate the tools Can instruct customers on using the tools Use time effectively Operate at both a strategic and detailed level

The level of competency that the sales force should achieve is determined by Sales & Marketing Managers. Using an internally-developed Back Office environment, managers assign competencies and courses to their direct reports and produce reports to see who has not completed courses or is not progressing through the competency program at the appropriate pace. Content is the most time critical component of TIs competency development solution. 90% of their content has been internally developed and 10% externally developed. Few courses are blended.

The MASIE Center e-Learning CONSORTIUM

Page 5

How Organizations Leverage e-Learning, Assessments, and Competency Management Because of the increasingly technical nature of their products and their speed to market, TIs Internally Developed Content needs to be created and deployed rapidly. For this reason, the authoring tools and process they use is a combination of: Presentation slides (PowerPoint) Synchronized audio (Real Player, Windows Media Player, Flash) Audio transcript embedded into the notes section of PowerPoint slides Conference calls Video PDF

These tools not only enable them to create and deploy content in as little as 24 hours, they provide an added benefit of accommodating different learning styles. Externally Developed Content is used to teach technical subjects, such as telecommunications, networking, and semiconductors. Working with the Sales & Marketing organization, they identified the subjects needed and then contracted with third-party online vendors, such as Telecommunications Research Associates (TRA) (www.worldwidelearn.com), for those courses. The Online Training Storefront is TIs internally developed Learning Management System (LMS). All courses (internal and external) are accessed through the Online Training Storefront, even though some vendor courses are hosted on the vendors server and some on TIs own LMS. When a person enrolls in an externally-hosted vendor course, an email is generated/sent to the vendor who registers the person and emails them back with a password for accessing the content. When a course and/or assessment is completed, the results are stored in the vendors database, an email with the data is sent to TIs LMS, and then the individuals training history is updated in TEDS. TEDS is a third-party Learning Management System (LMS) that TI uses to manage their Instructor-Led Training courses (www.teds.com). In 1998-1999, TEDS did not support eLearning, so TI developed their own LMS to support their online learning needs. All training history (both online and instructor-led training) is stored in TEDS. PeopleSoft is the HR system TI uses to maintain information on employees who they are, the TI organization they belong to, their job function, etc. Today, a data feed from PeopleSoft enables them to keep their learner tables in TEDS current. In the future, they plan to integrate TEDS, their Online Training Storefront, and PeopleSoft. They are developing a customized interface to facilitate this integration. The Results: Thanks to senior-level management support, TI has achieved 100% completion of the online competency program by all regions of the Sales & Marketing organization in 1999. Participation has been higher than 90% in subsequent years. They made available over 40,000 hours of training in 1999 and close to the same amount in 2000. In 2001, the number of hours delivered decreased because most people in Sales & Marketing had achieved their competency goals. Sales & Marketing has more fully embraced e-Learning. They average 30-40 hours of training per year, compared with only a couple of hours in other parts of TI. In fact, the number of courses taken last year exceeded their expectations. They expected $200,000, but $500,000 of unassigned courses were actually consumed.

The MASIE Center e-Learning CONSORTIUM

Page 6

How Organizations Leverage e-Learning, Assessments, and Competency Management

Individuals thought they knew more than they actually did. By having a way to validate prior knowledge and knowledge gained through training, they were able to demonstrate a large increase in the base knowledge of the Sales & Marketing organization. The number of people who were able to identify and qualify opportunities rose from hundreds to thousands, while the number of people who could demonstrate product features and benefits rose from tens to hundreds. As a result of the expertise theyve developed from working with the Sales & Marketing organization and the tools they now have in place, Clark is expanding his efforts to other parts of TI. In 1998-1999, Sales & Marketing funded 80-90% of his costs. Today, although Sales & Marketing remains one of his top three customers, they account for only 15% of his cost recovery. How TI Developed this Solution: Clark worked directly with top Sales & Marketing Management to understand and fulfill their requirements. How Competencies Are Developed: Competencies are jointly developed by TIs product experts and Sales & Marketing Managers based on what an individual in a specific job is expected to know and be able to do. Content is then internally developed to support these development needs. How Checklists Are Developed and Maintained: Checklists are maintained by the same Mentors who observe and evaluate the sales force. Mentors work directly with the various product groups to keep both their knowledge and the Checklist current. How the Test Engine Was Built: To create their assessments, they evaluated several vendors but found that either they didnt have the functionality TI was looking for or they were too costly. Eventually, TI found a vendor with a test engine that had most of the functionality they needed, so they hired the developer of the vendors test engine to build a Test Engine for TI. It cost $40,000. How TI Developed Internal Content: TI explored using third-party authoring tools, but found them too time consuming and costly for producing the type of rapid product training they needed. By the time a course was developed, the information was already out-of-date. Content development is a blended effort, with 90% of development accomplished by content experts from the product groups and the remaining 10% done by Clark's group. To facilitate the process, Clark provides product/content experts with a process, guidance, and tools, including a template to use in developing the content, test questions, and training agenda and an online how to document. For the remaining 10%, Clark adds a contextual frame of reference, learning objectives, questions to check comprehension, a call to action, etc. This process reduced the cost and enabled him to focus on the infrastructure. It also eliminated the need for product/content experts to know instructional design. When courses are developed, they build in a feedback loop between learners and product/content experts. So, when a learner does not understand a test question or thinks it is invalid, the feedback form links them directly to content experts who, in a matter of minutes, can either validate or change the information.

The MASIE Center e-Learning CONSORTIUM

Page 7

How Organizations Leverage e-Learning, Assessments, and Competency Management How TI Developed Their Online Training Storefront: Back in 1997-1998, TI explored third-party Learning Management Systems (LMS), but they could not find a single LMS vendor that could effectively meet all their needs: Provide competency management, support multiple formats, assign curricula to subsets of the organization, track courses to the degree that their Sales & Marketing organization required, etc. They also discovered that these systems cost hundreds of thousands of dollars, and neither Clark nor Sales & Marketing had the budget to fund such an investment. So they determined that building rather than buying was the best solution for them. Clark's team started with a system built in Cold Fusion using an Access database. As requirements evolved, one person on the team was given the responsibility of developing and maintaining the infrastructure. To be consistent with other software development efforts within TI, subsequent implementations of the Online Training Storefront have been built in ASP using a SQL database. Clark's group has now released its fourth generation Online Training Storefront. One member of Clarks team became an infrastructure manager and developed their Online Training Storefront using Cold Fusion. The current system they are using is third generation. An assistant helps maintain it. TI built a feed from their Online Training Storefront to their accounting system in order to handle training charge backs. At the end of each month, they run a batch job. When they begin using PeopleSoft, it wont be necessary to run the batch job. How TI Launched this Solution: The first-generation Cold Fusion/Access Online Training Storefront was released in 1998 on an existing server with a 10 MB connection to TI's network. The group now has multiple servers, each handling specific tasks, with 100 MB connections to TI's network, increasing the capacity and response time for their worldwide users. Solution Costs: The cost to internally develop TIs Online Training Storefront, including the test engine, was $40,000. It took about six months to develop. The internal cost of running and maintaining this system today is $100,000 per year it takes one full-time and one part-time person to maintain it. The cost to develop their online courses is $500 to $2,000 per content hour (compared with $5,000 to $35,000 per content hour quoted to them by vendors). The standard charge for internally-developed content is $25/content hour. Because of the large volume of online content that the Sales & Marketing organization was expected to complete, Clark was told he could only charge $10/content hour for online courses included in the competency program.

The MASIE Center e-Learning CONSORTIUM

Page 8

How Organizations Leverage e-Learning, Assessments, and Competency Management The Challenges and Lessons Learned: The following is a list of some of the challenges they faced and the lessons they learned along the way. Challenges Build or buy decisions Lessons Learned Once you make a decision, its very difficult to change. So, as you go through the thought process, consider both short-term and long term implications carefully. For TI, it was better to build. They have since compared what they developed to what is offered by outside vendors annually and have still not found any product that could match the capabilities and cost of their internally developed system. TI developed a rolling program they released a set of courses each quarter and notified management of their availability so that mangers could assign courses to their sales force members quarterly. They also provided a reporting capability for managers so they could monitor activity. For senior-level people, Clark informed them that he would be in their office on a particular day and time. He then hired a video crew and arrived to videotape them. For junior-level people, who preferred not to be videotaped, he gave them a headset and tools and let them capture it themselves. Gaining senior-level management support (Senior Vice President of Sales & Marketing) and tying the bonuses of Sales & Marketing Managers to the completion rate of their direct reports was the key. Naturally, this resulted in 100% participation across all 4 TI regions.

Getting Sales & Marketing employees to work on development throughout the year, rather than wait until December.

Content creation -- Even though developing content was very important, it became a lower priority than many other things that content experts had to do. So the challenge was getting them to make time to record the content. Completion of courses by learners

Adding Value: In 1999, a derivative of the internal Online Training Storefront was created and launched as the external Online Training website http://ti-training.com. Through this website, courses with appropriate content have also been made available directly to tens of thousands of TI customers and anyone who is interested in TI products. A secure space has also been created so almost all of the content made available to TI's internal sales force can also be accessed by its reps and distributors. Therefore, the same content is being re-used for multiple audiences, which dramatically increases the return on the content expert's time and effort. Looking Ahead: There are other areas of TI that haven't considered implementing online competency programs. Clark hopes to encourage and sell them on using the Online Training Storefront and PeopleSoft so they can develop, implement, and track the progress of their employees' knowledge and skills. In October of 2003, TI will integrate their Online Training Storefront with PeopleSoft making it a corporate-wide and corporate-supported package.

The MASIE Center e-Learning CONSORTIUM

Page 9

How Organizations Leverage e-Learning, Assessments, and Competency Management

Implementing a Performance-Based Field Qualification System

Case Study Focus: Implementing a Performance-Based Field Qualification System Target Population: Field workforce of 6,000 people distributed throughout North America Organization: An international manufacturing company Business: Manufacturing Champion: Training Manager Business Situation: This company's workforce is distributed throughout North America in multiple field offices. Projects have long life cycles and involve a fairly technical and broad range of skills. As more employees enter the company's workforce, it is necessary to provide detailed job performance and criteria, along with focused development plans. The company faces a fundamental challenge: to offer higher, value-added solutions to a wider range of customers. Part of the solution requires constant product evolution and innovation; another part of the solution requires overall reduction in costs: the cost of getting a new hire up to speed, the time to productivity for new hires or new product introduction, and the overall work efficiency. Business Solution: Driven by the need to sustain growth, maintain customer satisfaction, and reduce overall time and cost for employee development, this organization embarked on developing and implementing a performance-based field qualification system. The underlying principle of this type of system is that an individual is observed and evaluated by a "master performer" doing their job tasks under normal working conditions to ensure they are meeting specific standards of performance. A "master performer" is a person identified by their branch as an exemplary performer of these tasks. This type of field qualification system would enable the organization to achieve its goals of faster time to performance and the standardization of the work processes of field operations. Solution Components: With senior management support, the organization embarked on developing a performancebased field qualification system. It consists of job models, performance models, qualification instruments, qualification events, an evaluator/master performer, the individual whose performance is being evaluated, a website, e-Learning, the OnTrack for Training Learning Management System, an HRIS system and the help of a third-party vendor. How This Company Uses Its Field Qualification System: Qualification Instruments are checklists designed for use by an Evaluator. The instruments detail the expected step-by-step job performance upon which a field employee will be evaluated. Qualification Events are developed for each job. Each event consists of a series of tasks that lead to qualification. Events begin with less complex tasks of an entry-level person and progress to more complex tasks of an advanced-level person. The Evaluator is a person identified by their branch as a "master performer" or exemplary performer of these tasks. The qualification instruments detail the specific things that the evaluator must look for the candidate to do (ie, critical

The MASIE Center e-Learning CONSORTIUM

Page 10

How Organizations Leverage e-Learning, Assessments, and Competency Management steps they need to demonstrate, common errors they might make, etc.). If a candidate deviates from the expected performance steps, the evaluator, as a master performer of these tasks, is in a position to evaluate the circumstances and reasons that the candidate deviated and determine whether or not they should still be qualified. The instrument requires them to document and justify what they observed and the basis for their decision to qualify/not qualify the candidate. The evaluator must sign off on the qualification instrument. The information will be entered into the organizations Learning Management System (LMS), which is called OnTrack for Training. Although not yet fully implemented, their OnTrack for Training Learning Management System (www.dksystems.com) allows them to track and integrate their training and development programs. It also enables the evaluator to go into the organizations online educational services system, pull up an individual and record that they have been successfully qualified. A supervisor can look at the status of any individual's qualifications, what has not yet been accomplished, the evaluators comments, etc. Note: They are replicating the signature process in their LMS so in the future an evaluator can sign off electronically too. When a new field employee (ie, a specialist or engineer) is hired, they are entered into the organizations HR system. Two emails are automatically generated - one to the new specialist and another to their supervisor - to welcome the new employee, explain the system/process and provide documentation about it, and notify both individuals that a development plan for the new employee is due in 30 days. They also offer an orientation to the qualification system that explains the whole process. Qualification instruments are given to employees when they are hired so they can see what is expected of them, what they'll be measured on, how long it should take them, etc. The employee can review their qualification path from entry through advanced level and see all tasks in which they need to become qualified over the next six months. This drives training because it lays out the courses and on-the-job-training that are offered and required in order to achieve qualification (mostly technical skills). Quarterly reviews are used to check progress and make any adjustments. A website serves as a document repository for information resources related to the program and provides easy access and easy maintenance. It also lists the courses that are offered and cover the content they'll be tested on. From here, they can also print out a copy of the actual qualification instrument that the master performer will use to evaluate them.

How This Organizations Culture Contributed to Adoption: It was a challenge to integrate this process into the organizations culture, because its culture encourages people and branches to be innovators and somewhat self-directed. It is not one in which home office provides detailed procedures to its branches. Using a "push" approach to roll out standard practices and processes would not be the right approach. So they used a "pull" approach - they demonstrated the value and efficiencies to convince them of the benefits to the system.

The MASIE Center e-Learning CONSORTIUM

Page 11

How Organizations Leverage e-Learning, Assessments, and Competency Management The Results: Users are extremely supportive of the process and see how it benefits the company. New employees think its great and look forward to using it because it shows them the tasks they have to perform, how they will be evaluated, the training that supports the process, etc. Supervisors believe it makes setting goals easier, because it defines the technical aspects of a job and the discernible events that will achieve them. Senior management supports the process because it addresses performance criteria and productivity measures. Due to organizational changes, implementation progress was slowed. They are still in an implementation phase so they have not yet realized the full benefit of this system. However, senior management believes this to be a key strategy, so it will be implemented throughout all field divisions to develop the technical skills of their people. How This Organization Developed The Solution: This company began developing this solution with the help of a third-party vendor in 1998. They spent about one and one-half years in development. Below are the details of the development process: What They Learned From the Initial Survey An initial survey was used to gather information from field personnel and their supervisors on what the jobs and tasks were. In this process, they discovered a huge difference between what field personnel said they do (their job performance definitions and skill requirements) and what their own supervisors and their home office believe they do. So they embarked on a project to develop clear definitions of job duties, tasks, responsibilities, etc., for all key field personnel. Working with the Third Party Vendor to Develop the Process/System They worked with a third party, who helped them build a system to document whether or not a person could perform a qualification task to a specific set of standards. The vendor applied their process/discipline of doing job task analysis, defining qualification systems, creating qualification instruments, and then piloting and validating those instruments. Creating Job/Performance Models To create job/performance models, they brought in field personnel and asked them to define their primary job areas and identify key tasks performed, measurements for determining that they did tasks correctly, expected outputs, typical efficiencies, enabling knowledge and skills, etc. Once the job/performance model was created and validated, another smaller group of incumbents examined it and determined how the work was typically chunked and performed (ie, which tasks are performed together to produce a discreet activity that can be measured and recognized as good output). Field personnel also helped develop qualification "events" for each job.

The MASIE Center e-Learning CONSORTIUM

Page 12

How Organizations Leverage e-Learning, Assessments, and Competency Management Developing the Qualification Instruments Evaluators or "master performers" developed each qualification instrument -- a checklist used by evaluators/master performers to qualify an individual. It included things such as process steps that an evaluator monitors as a person executes tasks, non-negotiable performance criteria, etc. The Role of Technology Initially, this entire process and resulting documents were largely paper-based. They have now put all their paper documents online and created a website to serve as a document repository for easy access and easy maintenance. They also built an online tracking system to track an individual's completion of these qualification events and an administrative process for tracking, monitoring, and reporting. Training Although the bulk of training delivered by the education services department within this organization is still brick and mortar, (classroom-based, instructor-led, ) they are also doing more and more with technology to lower the total cost of training (an internal metric) to the organization. This organization uses WebEx (www.webex.com) as a Virtual Classroom for delivering product update training. They also use it for advanced desktop application skills training and employee orientation. They have several technical process training events in development which will also be delivered with this tool. They have been using traditional asynchronous CBT/WBT to deliver proprietary technical training in several areas, and they also use some off-the-shelf content to train employees on IT technologies and desktop application skills. They recently rolled out a blended solution for Employee and Management Development training (soft skills) that combines asynchronous web-based modules and follow-up skills practice with classroom sessions. The content is Leadership for Results from AchieveGlobal (www.achieveglobal.com). Much of this content has been linked to the qualification profiles to close knowledge and skill gaps leading to field performance. Training Development Development of training program content typically involves a content Design Team of Master Performers who work with instructional systems designers from the education services curriculum and qualification group to develop a detailed design of their training solution. They nominate (and can potentially be) members of the Development Team. A Development Team, consisting of subject matter experts, provides the specific training content and reviews and approves the final training materials. A Pilot Team, made up of members of all previous teams, attends the initial delivery and provides feedback on the effectiveness of the training solution. How This Organization Launched this Solution: This qualification system and process was not mandated by the companys senior management so the idea and its value had to be sold to district managers and managers. They initially rolled it out to target field locations. They provided several days of training for both management and field personnel and provided implementation guides. Their initial target population was 800 to 1,000 field specialists and engineers (service and systems) because they knew their quickest gains would be made with these field groups. They intend to expand to all key non-management field positions.

The MASIE Center e-Learning CONSORTIUM

Page 13

How Organizations Leverage e-Learning, Assessments, and Competency Management Solution Costs: An investment of $500,000 was made to develop the field-based qualification system. The Challenges and Lessons Learned: The following is a list of some of the challenges they faced and the lessons they learned along the way. Challenges Adoption Lessons Learned New employees like it because they know what is expected of them, how they'll be evaluated, etc. Most want aggressive plans so that they can become fully capable in their roles. Supervisors believe it helps to achieve goals. Some managers had to be convinced of the need for a master performer because it meant these individuals were "off the job" when they were evaluating someone. The best way to achieve organization-wide adoption is through incentives or by requiring adherence (ie, tie compensation and promotion to qualifications, integrate into performance appraisal process, etc.). However, the company has not tied qualification into a compensation system separate from current goal setting and appraisals. Having a strong local champion with a dotted line reporting relationship to the district manager is the best way to overcome obstacles and give the initiative the visibility and compelling goals needed for change.

Cultural Change

Looking Ahead: The organization is in the second phase of implementation. In recent meetings, senior management indicated that the field qualification system remains a key strategy that will be implemented throughout all field divisions to develop the technical skills of their people. The organization looks forward to realizing additional benefits of this system. The next opportunities for business improvement are the reorganization of curriculum and training support alternatives to make learning more accessible and to reduce time and costs.

The MASIE Center e-Learning CONSORTIUM

Page 14