Project Management for Computer Based Training Development

: This article deals with the management of instructional development projects for computer-based training (CBT), and is primarily aimed at project managers working with a team of instructional developers for a corporate client. Two issues are discussed: a) estimating the size of a CBT project, and b) performing a cost-benefit anal ysis. These issues are important for projecting costs, tracking performance and justifying development expenditures. This is a fictionalized case study. The methodology, examples, concepts and esti mates are composite sketches drawn from several projects, based upon the author's exper iences while working as a CBT project manager. Actual figures and clients have been inten tionally obscured to protect the proprietary rights of all parties involved. The reader should be cautioned that the article presents only one approach to project development and estimation. Wholesale application of the approach described is not recom mended, as every project will introduce novel interactions of resources and variables which mandate different treatment. Still, it is hoped that the reader will draw upon the ideas presented to refine project management approaches already used.

Instructional development teams in academic, business and industrial settings are selecting computer-based training (CBT) as the primary delivery medium in increasing numbers. Organizing large scale computer-based training projects requires traditional instructional development expertise, but also introduces novel problems for the project manager, given the labour intensive nature of such development efforts. What are some of the factors a project manager must consider when estimating the size of a proposed project? How can cost benefit analysis be used to justify development expenditures? What milestones should be observed for obtaining client approval during extended projects?
These questions haunt CBT project managers, who must coordinate the activities of several development team members, balance resources, and interface the needs of a client with the capabilities of a development team and the limitations of this medium. Anyone who has attempted to manage a large scale CBT development effort realizes that clients can be elusive, content volatile, resources limited, communication strained, and ultimately, how politically bound any development effort can become. Clarity of purpose and process can help a project manager avoid common pitfalls, and deliver a product which meets expectations on both sides of the financial fence.

ESTIMATING PROJECT SIZE
A common, and potentially misleading myth is that one hundred hours of development time are required to create one hour of computer-based instruction. A number of factors intervene to lengthen or streamline a CBT development effort, and alter unit-cost estimates. In addition, an "hour" of CBT is a meaningless label, given that trainees will progress through instruction at widely varying rates. More reasonable estimates can be obtained by eliciting accurate information from the client concerning the training required, and then estimating development efforts by task. The process of estimating projects can be logically broken down into three phases: client information; development stages and deliverables; and detailed task, time and cost estimates.

Phase I: Client Information
During Phase I, a number of avenues should be pursued with a client to clarify expectations and share understanding. A project manager's desire to "get on with the job" should be tempered at first to allow a thorough exploration of the scope and nature of the project. What types of information should be gathered?

Trainee Expectations
• perceived relevance of the content to job performance • perceived competence in the training components • motivation or desire to undertake training • level of computer phobia and keyboard skills • organizational pressure, mood or tone Current training approaches. In many projects, CBT is selected to replace or augment current training. A detailed understanding of current training approaches will supply the project manager with valuable information about the commitment of the organization to training, the resources available for development, and the "tone" of the instruction.
Content delineation. A crucial step for the project manager is to determine specifically what content is covered in existing training. Detailed information is required, and can be collected from a number of sources. Sample several sources to determine the reliability of information; printed documents will sometimes only vaguely resemble the actual content considered in a training event.
Client expectations. Clients may have expectations about computer-based training which impact the size of the project. Clarifying expectations early can help avoid later misunderstandings and costly revisions.
Trainee expectations. Often, useful information can be obtained from potential trainees. Identifying and accommodating trainee fears and desires can help in setting the tone and level of instruction.

Phase JJ: Development Stages and Deliverables
During the second phase of estimating a CBT project, client information is used to divide the project into major segments or stages. Of course the division between phases is artificial. The actual development effort will likely be characterized by continual dialogue and phase regression. Nevertheless, at this point you are considering what major development stages will need to be pursued to complete the project, and what output (deliverable) will mark each step. Developers commonly concern themselves (and the client) with only the final delivery of the course. In an extended project, this can be a mistake. Intermediate delivery of tangible products, even if somewhat artificial, provides evidence of progress and gives the client an opportunity to be involved in critical development stages. Formal review and approval of intermediate deliverables also makes the client accountable for development decisions, should revisions be required which extend the project or escalate the cost.
In a typical CBT development project, the following major stages and deliverables might be employed. By in large, they closely resemble the major steps involved in any instructional development project.
Front-end analysis. During this stage, information gained from the client is articulated in a needs assessment, audience analysis, environmental analysis. statement of project goals, and a preliminary workplan and project schedule. A front-end analysis document is delivered to the client for approval. Des(!?I1. Typically, the design stage includes a detailed task analysis of the content, identification of instructional units, specification of all objectives, an outline of design specifications (sample frames, menu and branching structure, colours, remediation), and outlines of supplementary materials. These steps can require a significant amount of "invisible" development time. Completion of the design stage is marked by the delivery of a detailed curriculum design document for review and approval. The client should be encouraged to review the curriculum design document carefully, as changes are relatively easy and inexpensive at this stage of development, but become increasingly more cumbersome as the project proceeds.
Writing. With most large projects, the development of courseware is broken into two steps: writing the courseware (using a word processor), and authoring the courseware (transferring the courseware to computer). Editing is much more efficient with a word processor than most authoring systems and languages. Since writing will usually involve revisions, courseware is first written and approved before authoring begins. Each unit and supplementary materials should be submitted to the client for review and approval. If possible, the client should be encouraged to include trainees in the review process.
Authoring. Approved written versions are entered into the final computer-based version of instruction. If the design specifications were precise, and the writing composed with frame constraints in mind, this is a relatively mundane and time-consuming task. The project manager should work closely with the CBT author to ensure that project specifications are followed. The completion of authoring is marked by the delivery of the CBT version of the courseware to the client.
Implemelltation. The implementation stage of development includes pilot testing and perfonning final revisions on the courseware. Clients vary in their commitment to pilot testing materials, but the project manager should encourage this step if possible. Embarrassing errors can often seep through to this version of the materials, especially given the number of individuals who have handled it. A pilot test can often catch mistakes before the product is released. A pilot test report should be delivered to the client upon completion, detailing revisions following the pilot test.

Phase 11/: Detailed Task. Time and Cost Estimates
The third phase of estimating a CBT project involves breaking the major stages identified in Phase II into discrete tasks. The tasks are smaller and easier to estimate. Certainly, some tasks may be overlooked and estimates will be subject to error. Still, because of their size, errors will be small, and unless they are systematically wrong in the same direction, the overall estimate will be reasonable. Stress with the client that estimates are tentative, and changes to the project will impact the estimates.
Project management time should be included as a discrete task in estimates. This is a separate administrative category covering time devoted to managing development team members, interacting with the client, writing reports and the like. A percentage of time, say, 10%, can be estimated over the duration of the project for the project director, although the percentage will increase with the complexity of the project and the number of individuals involved. Internal and client reviews should also be included as tasks, as this will be valuable infonnation for estimating the duration of the project.
In order to complete the cost estimate and a tentative schedule, the following infonnation is needed for each task: -Starting date; -Completion date; -Duration (in working days); -Hours (billable time needed to complete task); and -Billing rate (amount per hour).
When estimating starting and completion dates, keep in mind that several tasks may occur simultaneously, while some will not be able to begin until an earlier step is completed. For instance, authoring a unit cannot begin until writing has been approved, but task analysis, audience analysis, needs assessment and learning environment analysis may occur in any order. In addition, several individuals may attack different tasks, compressing timelines considerably. Table 3 was adapted from an actual project to illustrate the appearance of a task, time and cost estimate. The rates and tasks were altered to simplify the example and preserve the confidentiality of some of the figures. The actual final product included seven CBT units and an instructional guide.
Although estimates will need to be updated periodically to reflect changes in the project, the task, time and cost estimate provides a useful fonnat for reviewing perfonnance, setting target delivery dates, and refining cost and material estimates.
For scheduling large projects, a number of helpful programs are available to the project manager. Notably, "Project Scheduler" for the IBM micro series, and "VisiSchedule" for Apple series computers were found to be useful tools. Both are tedious when initially setting up the components of a schedule, but easy to update and revise.

COST-BENEFIT ANALYSIS
It's no secret that CBT development is expensive. In many cases, the apparent instructional benefits of the medium are sufficient to justify development expenditures. Trainers and developers look to CBT to provide consistent, flexible, individualized and effective learning opportunities for trainers ... and indeed these things are possible. Still, others in our "bottom-line" society are more interested in cost factors related to training, including travel expenses, opportunity costs, and development expenses. The purpose of the following approach to cost-benefit analysis is to address these concerns --to compare conventional classroom training costs to CBT development and delivery over a multiple-year span. This type of analysis does not address the comparative effectiveness of the two approaches. Rather, cost-benefit analysis in this instance asks "Can CBT development costs be favorably amortized?" The assumption is that unless an obvious and critical problem exists, cost will be used to justify the adoption or rejection of CBT.
Several variables can influence a CBT cost-benefit analysis positively, some of which have direct cost implications, and some of which influence costs indirectly. Considering direct cost influences, CBT is a reasonable alternative when large numbers of trainees must be accommodated, flexible scheduling is required, trainees are widely distributed geographically, and the client has existing computing facilities which can be exploited for training.
When considering indirect influences on costs, CBT development is an attractive alternative when learning requires practice or repetition, consistent training is required, and when the content to be delivered is structured or logical.
A cost-benefit analysis is performed by first outlining the major cost variables associated with conventional training. lbis information should have been gathered during Phase I of estimating the project size, and Phase III likely provided most of the information necessary to complete an analysis of training costs associated with CBT training. Any missing information can be easily obtained by contacting the client.

Sample Positive Cost-Benefit Analysis
In order to illustrate the process, the same project as earlier will be referenced. Two analyses will be performed: one projecting classroom training expenses, and a second projecting computer-based training expenses. The problem. Corporation Amoeba is a commercial operation with 100 stores in a wide area. Store level management personnel (store managers, co-managers and department managers) rely upon two key financial documents to describe performance monthly. Successful retail performance depends upon skillful analysis of the financial documents. Approximately 500 individuals require training in the first year, with ongoing training requirements resulting from 10% turnover per year, and growth of 10% new locations per year. Approximately five individuals per location require training. For the purpose of the analysis, the following estimates will be used. With the introduction of CBT, some additional variables are introduced. The analysis is based upon the following figures.

Cost Benefit Analysis Observations
The preceding analyses were constructed to illustrate specific points, rather than reflect reality. In any actual cost benefit-analysis, variables will differ significantly. Still, from this example, a number of useful observations can be made.
In this example, CBT development and equipment purchase was fmanced by diverting funds from travel, accommodation and opportunity cost categories. In other situations, where these figures are not as dramatic, costs will be amortized over a longer period of time.
At the end of the three-year period, the corporation will have equipment which can serve other functions, although this is often a highly political issue. In other cases, a client may already possess computers which can be used for training, significantly reducing the amortization period.
The above example assumed that the length of training could be reduced from two days to one, due to the efficiency of computer-based training. The efficiency issue is not well established, and may be more folklore than fact. Nevertheless, in the projects referenced for this article, the CBT instruction was much more efficient than conventional classroom intervention, and resulted in approximately fifty percent (50%) time savings. Development personnel speculated that this was due to the intensity of planning which occurred during the project, rather than any inherent characterisitics of the medium employed.
The figures used in the analysis closely approximated actual costs from CBT development projects. By manipulating a few of the figures represented, it should be evident that cost justification of this type is a function of available equipment, employee wages, the number of trainees involved and the length of the amortization period.
These are factors for your consideration, and generalized tools you may find useful as a project manager. This is not a fill-in-the-blank form for landing CBT contracts. It should, however, provide a level of guidance for planning large scale projects, and considering whether CBT can be justified on a cost basis.