Audience Analysis Keith A.

Doering Colorado State University

1|Page

Kevin Costner's character in The Field of Dreams heard a voice telling him "if you build it, he will come" ("Field of dreams (1989) quotes," n.d.). Taking that approach in the development of an instructional system would almost always result in failure. A systematic approach is required to ensure adequate resourcing of resources which are often limited. Understanding the learner and their needs is a critical step because their "knowledge will be important in designing instruction that is effective and interesting to learners" (Smith & Ragan, 2005, p. 58). In order to capture the training needs of my office, I first had to understand the requirements of the office. The office performs the Training Evaluation function for our base and assumes multiple responsibilities. The Training Evaluators are in the GS-1750, Instructional Systems Specialist job series as classified by the United States Office of Personnel Management. According to the Positional Classification Flysheet for Instructional Systems Series, GS-1750 one of the jobs a GS-1750 can be used for is Instructional Program Evaluation. In this capacity, an individual will "use such techniques as: needs analysis; statistical validation of tests; comparative studies of the progress of various categories of students; and observation of instructional methods including the use of training devices, equipment and facilities" (Positional Classification Flysheet for Instructional Systems Series, GS-1750, 1991). There are six GS1750 positions in the office. In addition, there is a data technician and four military personnel who take on special projects. When developing the learner analysis tool (Appendix 1), the primary focus was on the GS-1750 Training Evaluators to develop the questions to understand the needs for the training program. The Core Document for government positions identifies primary duties and required Knowledge, Skills and Abilities (KSAs) for specific jobs. The 1750 Core Document (Hicks, 2005) for the Training Evaluation office expands upon the requirements outlined in the United States Office of Personnel Management document. Duty 2 in the core document of the office states individuals properly devises solutions for which precedents do not exist, develops new approaches, and makes major modifications to existing guidelines." One of the
2|Page

standards of measurement for that duty is for the training evaluator to develop and implement evaluation devises to nearly always ensure proper and accurate statistical analysis of assigned specialty areas". Duty 4 outlines that an evaluator will "properly develops and administers surveys to obtain feedback from graduates and their supervisors. Conducts comprehensive analysis of statistical results to determine quality of educational and training programs." The last duty which ties directly to the development of this training program is duty 5. Duty 5 requires an evaluator to prepare Evaluation Reports for final review and approval by the supervisor. Each of the duties outlines in the core document is directly tied to KSAs for the duty position: Knowledge of training evaluation administration, policies and procedures. Ability to establish assessment program objectives and to assess progress toward their achievement. Knowledge of learning theory, the principles, practices, and techniques of instruction, instructional design and materials development. Ability to analyze and evaluate in order to accomplish special training evaluation projects or studies and develop timely and economical solutions. Ability to effectively communicate orally, in writing and work harmoniously with all levels of personnel. Knowledge of training resources programs, principles and policies. Knowledge of the principles and techniques required to evaluation instructional programs.

The KSAs served as the basis for specific questions in the analysis tool.
RESULTS

Nine of the ten individuals assigned to the office responded to the survey which was administered via SurveyMonkey. The responses follow:

3|Page

4|Page

NOTE: One respondent answered Dont know. The individual had previously answered that they had two years of evaluations experience.

5|Page

NOTE: One respondent answered Dont know. The individual had previously answered that they had two years of evaluations experience.

6|Page

NOTE: One individual responded that Class room with extensive, repetitive, hands-on buttonology and another wrote Either self-paced or get into the system and play around. Classroom settings arent always effective in learning how to navigate data bases and websites. Upon review of the responses to the analysis tool, I realized I could have asked a few more demographics questions to help me fine-tune the training program. Although my primary focus was for the training evaluators, the recent addition of additional special projects military personnel might would allow me to tailor training to their needs vice developing a project for an evaluator. As a result, I would have added an additional question for the respondent to identify their duty position. Analysis of the results of that question would allow me to better understand the needs of the office and possibly tailor individual training based upon job function. The results have also forced me to rethink how I was going to develop the training program. The original plan was to create a library of self-paced PowerPoint presentations to guide the office through different components of SurveyMonkey; however, five of the eight who responded to the question about delivery method preferred a classroom setting over self-paced. As a result, I might have to develop short lessons for a classroom setting to cover "buttonology" and then give individualized assignments upon completion of the classroom work. The limiting factor with this approach is finding time where the entire office is available for a classroom lesson.

7|Page

Going back to the Field of Dreams analogy, there might be some truth if the vision is modified to if you build them, they will come. In simple terms, avoid developing a training program with the mind-set of one system (or approach) will work for everyone because a cookie-cutter approach to education and training is never successful.

8|Page

Appendix 1

Keith Instructional Design Project

I'm enrolled in an Instructional Design and Program Development course as part of my Master's program. My project this semester involves building a training program for the office. My goal is to put together a SurveyMonkey training package which will allow everyone in the office to use SurveyMonkey to its potential in order to meet our evaluation needs. The following questions will give me some data required to develop the training program. Thanks in advance for your help, Keith

9|Page

Keith Instructional Design Project

1. How many years of experience do you have working in a training evaluation office?
j Less than 1 l m j 1 l m j 2 l m j 3 l m j 4 l m j 5 l m j More than 5 l m

2. What is your comfort level with computers and technology?

Very Comfortable
j k l m n

Somewhat Comfortable
j k l m n

Not Comfortable
j k l m n

Please specify if not comfortable

5 6

3. Do you have any experience with using HTML codes?

j Yes l m j No l m

4. Do you have any experience with creating surveys in SurveyMonkey?

j Yes l m j No l m

5. Which component of SurveyMonkey do you feel is the most important for you to learn
j Creating a survey l m j Editing existing surveys l m j Collecting surveys l m j Analyzing responses l m j Building reports l m

Other (please specify)

10 | P a g e

Keith Instructional Design Project

6. Which component of SurveyMonkey do you feel is the least important for you to learn
j Creating a survey l m j Editing existing surveys l m j Collecting surveys l m j Analyzing responses l m j Building reports l m

Other (please specify)

7. How much time do you have in an average week to take the training?
j Less than 1 hour l m j 1-2 hours l m j 3-4 hours l m j More than 4 hours l m

8. Which training environment best describes how you prefer to learn?

j Classroom setting l m j Self-paced l m j Trial and error (figure it out on my own) l m

Other (please specify)

5 6

11 | P a g e

References Field of dreams (1989) quotes. (n.d.). Field of Dreams. Retrieved from http://www.imdb.com/title/tt0097351/quotes Hicks, G. (2005). Air force core personnel document (CPD) (United States, Department of Air Force) Smith, P. L., & Ragan, T. J. (2005). Instructional design. Hoboken, NJ: J. Wiley & Sons. United States, Office of Personnel Management. (1991, May). Positional Classification Flysheet for Instructional Systems Series, GS-1750. Retrieved from http://www.opm.gov/policydata-oversight/classification-qualifications/classifying-general-schedulepositions/standards/1700/gs1750.pdf

12 | P a g e