Planning for Integrating Teaching Technologies

Canadian Journal of Learning and Technology

Volume 30(2) Spring / printemps 2004

Planning for Integrating Teaching Technologies

Mandie Aaron

Dennis Dicks

Cindy Ives

Brenda Montgomery

Authors
Mandie Aaron, Ph.D. is an Educational Consultant In Montreal, Quebec.

Dennis Dicks, Ph.D., is an Associate Professor of Educational Technology at Concordia University in Montreal. Contact: 1455 de Maisonneuve W. Montreal, QC, H3G 1M8. Telephone: (514) 848-2424 x 2026. Email: djd@vax2.concordia.ca

Cindy Ives, Ph.D., is a Faculty Development Associate at McGill University in Montreal. Contact: cindy.ives@mcgill.ca

Brenda Montgomery, M.A., is a teacher at Selwyn House School. Contact: rilling@selwyn.ca

Abstract

Teaching technologies offer pedagogical advantages which vary with specific contexts. Successfully integrating them hinges on clearly identifying pedagogical goals, then planning for the many decisions that technological change demands. In examining different ways of organizing this process, we have applied planning tools from other domains - Fault Tree Analysis and Capability Maturity Modeling- at the school and college levels. In another approach, we have examined attempts to broadly model the integration process at the university level. Our studies demonstrate that the use of a variety of tools and techniques can render the integration of teaching technologies more systematic.

Résumé: Les technologies d'enseignement offrent des avantages pédagogiques qui varient en fonction des contextes particuliers. Pour réussir leur intégration, il faut clairement cerner les buts pédagogiques et planifier les nombreuses décisions que ces changements technologiques exigent. En considérant différentes manières de mettre en œuvre ce processus, nous avons utilisé des outils de planification provenant d'autres domaines - l'analyse de l'arbre de défaillance et la modélisation de stabilisation des capacités - tant pour les écoles que les collèges. Dans une perspective différente, nous avons étudié les tentatives cherchant à modéliser de façon large le processus d'intégration au niveau universitaire. Nos études démontrent que l'utilisation d'une variété d'outils et de techniques peut rendre l'intégration des technologies d'enseignement plus systématique.

Technologies for Teaching

Supporters claim that the use of information technologies in education will increase communication among students and teachers, provide access to resources that may otherwise not be available, and encourage "authentic" learning as students access "real-world" data not provided by textbooks (Schrum, 1995). Given that most students almost anytime, anywhere can access various forms of information technology - MP3, cell phones, PDAs - it does not make sense to exclude this part of their experience and ability (Tapscott, 1999) from the educational part. Together, these challenges to the instructor's monopoly on sources of learning (Dicks, 2000) can serve as a catalyst for an examination of pedagogy, perhaps moving practice from a didactic to a more collaborative approach (Becker & Ravitz, 1999; Dexter, Anderson, & Becker, 1999).

Attempts to integrate information technologies (IT) into teaching and learning have a history as long as the technologies themselves. They range in scale from simple classroom applications to projects covering entire school systems, sometimes focused on education at a distance, but increasingly promoting a `blended' format: access at school, at work, at home, on the road. To a large degree, the logic driving technology integration has shifted from serving a circumscribed clientele (e.g., Continuing Education) to providing competitive advantage at the institutional level. In any case, as Ives (2001, Chapter Two) points out, the literature suggests that simply adding technology, simply investing in hardware and connectivity, will not produce the promised benefits. Technology is by nature disruptive, and so, demands new investments of time, money, space, changes in the way people do things, new skills and so on. While the classroom innovator may make do, at the level of a school, college or university, technology integration will involve broad groups of stakeholders: faculty, staff, technicians, librarians, administrators. "Make do" will not do. Bates (2000) suggests that project management, instructional design, team-based course development and other academic and administrative techniques perfected in distance education environments, are crucial to the success of technology integration in a broader institutional context. He insists that success requires an appropriate match of educational technologies to strategic mission. Change on this scale requires coordinated planning.

In this paper we will examine several different approaches to planning for the integration of teaching technologies, drawing on literature from a wide range of disciplines. We adopt a very broad sense of teaching technology to include any computer application intended to facilitate teaching or learning. Our objective is to use our research findings to inform planners, particularly at the school, college or university level, about specific techniques for analyzing institutional needs, and about broader perspectives for modeling the change process.

Planning for Technological Change

Technology planning approaches generally fall into one of three categories: top-down, initiated by administrators; bottom-up, driven by the people delivering a product or service; or mixed, involving a bit of both. Top down approaches may ensure adequate resources, but risk involving the grass-roots only superficially or even engendering resistance. In a bottom up approach, the grass-roots may be better placed to understand and implement innovation, but enthusiasm may not compensate for a lack of physical and political support.

Instructors determine to a large extent what happens in the classroom (Hooper & Rieber, 1995). Fuller's (2000) research suggests that instructor acceptance plays a critical role in the successful use of computers in the classroom. However, other players - school boards, university administrators, government agencies - tend to control goal -setting, working conditions, performance evaluation, and the resource allocation that shapes these activities. By compelling instructors to collaborate with people outside the classroom, technology can be perceived as a threat to the private practice of pedagogy. Effective planning means getting the top and bottom to communicate across their subcultures to build a shared set of ends and means.

Effective planning combines the advantages of the top-down and bottom up approaches, where leadership provides the mandate, the resources, and the coordination, yet recognizes the importance of local acceptance and empowerment. In one form of this approach, leadership provides the structure of reform, the grassroots work out the details. The State of Indiana presents an example of this type of planning, by dictating the general guidelines for the technology plan, but leaving the details to each school (Indiana Department of Education, 2002). Similarly, the California K-12 Education Technology Master Plan for Educational Technology lists recommendations for schools including acquisition of resources, equitable distribution, professional development, and evaluation of the plan (California Department of Education, 2004).

A limited body of evidence indicates that planning for teaching technology adoption does lead to better results, beyond funding, equipment acquisition and other resource allocations (Brush, 1999). Strategic planning tends to be limited to these resource issues, so it needs to be broadened to include adjustments to organizational culture (Ives, 2002, p.148), thereby encompassing a broad range of factors which potentially affect the success of integration initiatives.

Factors Affecting the Integration of Teaching Technologies

In summarizing research on attempts to introduce teaching technologies in schools, Leggett and Persichitte (1998) and others have identified sets of factors (TEARS: Time, Expertise, Access, Resources and Support) which reportedly have influenced the adoption of teaching technology. Aaron (2001), in research at the college level, has modified this to the "SPECTRA" list (Support, Perceived need, Expertise, Communication, Time, Resources, Access) by adding the key ingredient, communication. In a rigorous review of the literature on the integration of teaching technologies in universities, Ives (2002, p.45) identified a long list of factors. Table 1 reproduces the data from Aaron and Ives.

Table 1.
Factors affecting teaching technologies integration in college and university contexts.

All of this information derives from qualitative reports on respondents' concerns, so the relative importance of these factors can only be assessed by frequency of citation, rather than a quantitative metric. Thus, the percentages indicate the relative frequency of responses for each type of factor. The college data comes from a study conducted by Aaron (2001) discussed in the Fault Tree Analysis section later in the text. The university data comes from an analysis of 136 articles published in the period 1995 - 2001 (Ives 2002, p.45). The factors have been listed according to Aaron's "SPECTRA' categories, on the left side of the table. Generally, the college and university samples tend to express similar types of concerns. This issue will be taken up again in Table 6.

Identifying these factors in a particular institution should help planners prepare for the integration of IT in that context. Ideally a planner would like to know not only which factors concern the target group, but also orders of priority, so that resources could be optimally marshaled to effect change. How might a planner obtain this type of information?

We have experimented with two different types of analytic tools for determining the factors likely to influence adoption of IT in a specific context: Fault Tree Analysis and Capability Maturity Modeling.

Fault Tree Analysis

Fault Tree Analysis (FTA) originated in safety engineering as an analytical tool for examining systems in terms of events and other types of impediments likely to prevent the attainment of the system's goals (Wood, Stephens & Barker 1979). Understanding possible sources of failure within a system provides opportunities for pre-emptive action to increase the likelihood of success.

Early on, educators adapted the FTA approach for the examination of educational systems (Stephens, 1972) and several such applications followed (Jonassen, Tessmer & Hannum 1999), but only on a modest scale. In this context, FTA offers two types of advantages: a) the process facilitates communication about proposed innovations in terms all stakeholders can understand, and b) FTA provides a practical outcome in the form of a graphical display of the people and events involved in the proposed innovation, and the relationships among them. Both these features should contribute to the accuracy and authenticity of the data collected on the factors influencing the adoption of significant change. On the down side, FTA demands substantial time and effort, such that the validity of the analysis will be threatened if the scale of the system limits participation by the stakeholders.

FTA thus holds promise as a way of involving stakeholders in the planning of technological innovation, identifying where barriers to change may arise, and indicating which players could be engaged in seeking alternatives to overcome these barriers.

A Field Trial of FTA

Aaron has conducted a major field trial of the FTA method in a college contemplating teaching technology initiatives (Aaron, 2001). The College, part of Quebec's province-wide network of first-level post-secondary institutions, offers a mix of pre-university and professional programs to 7500 full-time and 3500 part-time students, in an urban setting. It describes its mission in terms of attaining excellence through both innovative educational approaches and the application of appropriate technology. At the time of the study, information technology took the form of centralized faculty resource rooms (5 for 543 faculty) equipped with a few networked PCs. In addition, the College provided about 150 administrators, program directors and faculty with personal network access. Academic and IT administrators expressed interest in moving faculty towards web-based delivery of courses on campus, and in establishing a presence in distance education. They thought an FTA might serve this cause. Subsequently, Aaron and the College established an FTA Team consisting of two representatives of the academic administration, two representatives of technical services, 4 faculty, and Aaron, the FTA researcher.

FTA procedures formulated by Stephens (1972) have been elaborated by others, including Jonassen et al. (1999). Aaron further modified the procedures to fit the time constraints of the FTA team members and to permit the participation of faculty members (the focal system). The next section summarizes the steps followed in this particular Fault Tree Analysis.

FTA Field Test Methods

As its first task, the FTA Team agreed upon a mission statement: briefly, that by the end of the academic year, the college would develop and implement a small set of web-based credit courses, and increase faculty awareness of potential uses of new teaching technologies.

Following the steps outlined below, the FTA Team then speculated about events which might prevent the achievement of this goal, and, through a long series of meetings and e-mail exchanges, structured their ideas in the form of a Fault Tree. Towards the end of this work, the Team invited all 543 faculty members (the focal system) to evaluate, by means of printed survey, a draft version of the Fault Tree (response rate 14%). Throughout this study, over roughly 8 months, the researcher collected several other types of qualitative data regarding the activities and attitudes of the FTA Team:

  1. Defining the system: 4 members of the administration and the researcher met to define the system boundaries, goals and constraints (e.g., technology integration in courses offered by College faculty).
  2. Mission analysis: The FTA team defined the types of events that would indicate successful accomplishment of the goal (e.g., increased use of Internet in classes by faculty).
  3. Identification of undesired events: The FTA team identified events that would indicate the goal was not being reached (e.g., no increase in faculty use of the Internet in classes).
  4. Fault hazard analysis: The FTA ranked all undesired events by degree of importance, and by level (goal, function or task).
  5. Contributing failure analysis: Members of the FTA, using the integration literature, identified as many failures (e.g., no computer access) likely to contribute to undesired events as possible. These were categorized as "under/partially under/not under" system control.
  6. Specification of logic gates: The FTA determined whether failures were linked by AND or OR gates, concluding that all were joined by OR gates and that for simplicity the gates would not be included in the final Fault Tree.
  7. Develop Tree to next lowest level until complete: Steps 5 and 6 were repeated until no further contributing events could be identified.
  8. Validate Fault Tree against the system: In a first validation, the FTA confirmed each of the events, and their lateral and hierarchical relationships; a second process is outlined in Step 11.
  9. Label Fault Tree: For easy reference, the researcher labeled all events A, B, C etc.; AA, AB, AC, etc. contributing to A; and so on.
  10. Quantitative evaluation: Because of the tree's limited scope, the FTA deemed quantification (e.g., Stephens, 1972) was not necessary.
  11. Validation by the focal system: As part of a survey on integration, faculty were asked to comment on the relevance and likelihood of a sample section of the tree.
  12. Validation of the FTA process: In collaboration with the FTA team, the researcher assessed whether the FTA process contributed to an understanding of the College's environment.

Fault Tree Analysis Outcomes

The Fault Tree constructed by the FTA Team identified a total of 228 events that might result in failure to reach the goals of the teaching technology initiative. Generally, the 75 faculty responding from the focal group endorsed the failure events in the FTA Team's prototypic tree. Figure 1 illustrates how a Fault Tree inter-relates failure events, for a small sample from this very large set. Each of the boxes in the figure represents a failure event identified by the FTA team. Each event is seen as leading to the occurrence of events higher in the hierarchy. In more formal practice, the events would be joined by logical operators (e.g., and/or gates). Inserting gates was judged too complex a task for the population in this field trial.

Aaron has coded the 228 failure events into what she has called the SPECTRA categories (Table 1). Basically her analysis takes the list of factors identified by Leggett and Persichitte (1998) and others, then adds Communication to reflect her finding that this element tends to be under emphasized in technology integration studies.

Because the college and university factors have been derived in very different ways, from different types of sample, comparing them stretches the imagination. However, the samples weigh the factors similarly, except for communication, which ranks far above others in the college data, and figures much more importantly than in the university data. Even if the climate and culture factors in the university data are combined, this "communication" category (15%) still ranks third, well below the first two, and comparatively lower than "communication" in the college data. Apparently people working in this college have high expectations regarding communications about impending change.

Capability Maturity Modeling

We have also experimented with Capability Maturity Modeling (CMM) as a tool for exploring how an organization might react to innovative change. Here again, we borrow from engineering practice, in this case from the software industry (Carnegie Mellon Software Engineering Institute, 2002). CM models have been used in other domains, such as systems engineering and human resources. In this case, we have used the CMM concept to examine the readiness of a set of schools to embark on eaching technology initiatives.

A Capability Maturity Model assesses the readiness of an organization to assimilate change by evaluating its practices within a defined and standardized framework. Each of the five maturity levels of this hierarchical framework includes methods for determining the status quo and for making a transition to the next level. These methods focus on the organization's mastery of different Key Process Areas (KPAs) at the respective maturity levels.

Table 2 displays the five levels of a generic CMM and their basic characteristics, adapted by Montgomery (2003) from Carnegie Mellon Software Engineering Institute (2002). As an organization advances through levels 2 to 5, the characteristics generally accumulate. Each level, except the first, has a set of KPAs defined in terms of goals and "best practices" (see Table 4) which govern the organization's ability to successfully manage change.

Table 2.
The five levels of a CMM (Capability Maturation Model) and their basic characteristics

In education, many factors complicate the process of innovation. Technological, social and pressures from the work world encourage educational institutions to evolve. Not a domain that readily accepts change, education reacts gradually to these external pressures. We can see this gradual approach in the way successful instructors adopt a reflective outlook on their own practices (Becker & Ravitz, 1999). This suggests that educational cultures can pass through different phases of maturity regarding change, ready to move forward, backward, or maybe not at all. A Capability Maturity Model attempts to document these phases.

A Field Trial of CMM

Whereas Fault Tree Analysis examines possible areas of resistance to change in an organization in an open-ended process, CMM compares the organization's readiness for change with the set of guidelines outlined in Table 2. Like FTA, the CMM approach offers the advantage of allowing all stakeholders to participate in the assessment exercise. Accordingly, Montgomery (2003) has conducted a collective case study of six K-12 schools in various stages of planning for teaching technology integration, using CM modeling with a view to developing a TI-CMM (a "technology integration" version of the CMM tool). Three private, one private parochial, and two public schools agreed to participate.

By means of structured interviews, examination of plans and records, and teacher surveys, Montgomery gathered data in all six schools regarding the technology planning process, availability of equipment, network infrastructure, teachers' attitudes and skill levels, and examples of technology use. She examined each school's technology plan, the teachers' attitudes toward technology use, and samples of technology projects; interviewed the Principal or Head of school, and the technology coordinator or IT Director. She analysed this information within the generic CMM framework in order to develop descriptions for the KPAs, goals, and best practices at each level of the TI-CMM. A central premise of the TI-CM Model is that the purpose of technology in schools is to support teaching and enhance the learning process.

Separate protocols for the Principal and for the technology coordinator guided the interviews. The questions dealt with issues that define each of the maturity levels, including documentation, use of technology, professional development, availability of resources and infrastructure.

Teacher interviews employed two instruments. The Technology Implementation Questionnaire (TIQ) (Wozney, Venkatesh, and Abrami, 2001) examines instructors' use of technology and their reasons for integrating or not integrating technology into their classrooms. Montgomery used the TIQ to determine the extent to which instructors actually use technology and whether or not they see it as beneficial and cost-effective. The TIQ provides a benchmark to assess the state of technology integration in each school at an instant in time in comparison with the plans that were developed and implemented by each school's administrator.

The second instrument, the Teacher Technology Activities Questionnaire designed by Montgomery, asks instructors for examples of how they use technology in the classroom. Montgomery also asked each school to provide copies of its technology plans or relevant documents. Five of the six schools could comply with this request.

Capability Maturity Modeling Outcomes

In her study, Montgomery modified an established planning process to create a Technology Integration Capability Maturity Model for K-12 schools. While no model can perfectly represent the complexities of an educational institution as it adopts new technology, the TI-CMM provides precise goals and guidelines for a staged implementation plan, grounded in research by educators with principles differing from those of commercial organizations.

Use of the case study method permitted a grounding of the TI-CMM in data collected from real schools on the goals, key process areas, and practices which define each level of the maturity model. Further, the collective case study sampled a range of schools in the expectation that each school would represent differences in practice, planning methods, and in the amount and use of available technology. This provided insights into issues at each level of the TI-CMM and allowed for the development of a generic TI-CMM useful for all K-12 schools.

Table 3 displays examples of how the five levels of a generic TI-CMM might be manifest in the context of teaching technology integration. Table 3(a) shows changes in teacher/student access to technology, and how they use it, as an organization progresses through the `capability' levels. Table 3(b) shows changes in the roles of the project champion and technical support personnel and in professional development activities for teachers as an organization progresses through the `capability' levels. Each level except the first has a set of Key Process Areas (KPAs) defined in terms of goals and "best practices" which govern the organization's ability to successfully manage change. Table 4 provides examples of KPAs for each level in the TI-CMM in the context of teaching technology integration. The column headings indicate broad types of process; the last heading indicates the types of process which have to be developed to move from a given level to the next one. Generally, the higher levels incorporate the key processes already present in lower ones.

Table 3 (a).
Levels of the TI-CMM with regard to physical resources


Table 3(b)
Levels of the TI-CMM with regard to human resources


Table 4.
Examples of key process areas for each level in the TI-CMM


Data from the sample schools embarking on teaching technology projects indicate the TI-CMM can be useful in identifying the strengths and weaknesses in the schools' planning activities, and suggests concrete steps schools can take to move forward.

Table 5 indicates where the six schools in Montgomery's sample would fit in her TI-CM model, based on the data she collected. The table shows examples of how technology has been integrated into teaching activities. None of the schools would place at the highest levels of the maturity hierarchy, though two displayed most of the characteristics of the second highest level. The concentration of the schools in the lower TI-CMM ranks should not surprise us since they have only recently embarked on programs that entail pervasive change.

Table 5.
Positions of the six sample schools on the TI-CMM

We can get an idea of the issues these schools face from the data that Montgomery's survey collected from the 117 participating teachers. Response rates varied from 18% to 58% across schools. The responses, coded by Montgomery (2003), have been grouped according to Aaron's (2001) SPECTRA categories in Table 6. The last three columns compare the relative frequency of responses for each SPECTRA category in this school data [S]; and in the college [C] and university [U] data presented earlier in Table 1.

Table 6.
Teacher comments on 117 questionnaires on teaching technology integration

Teachers in schools, colleges and universities broadly agree in ranking Support near the top in importance, Time, Perceived Need and Access near the bottom, and Expertise in the middle. The college sample ranks Resources lower, and Communication higher than the school and university samples.

The case study data provides further insight into the relative importance of varios factors favouring technology integration. One is the vital role the technology "champion" plays in schools. In the SoftWare-CMM, and the Human Resources-CMM, too much reliance on an individual person relegates an organization to a level 1 rating. Similarly, schools relying on individual teachers to encourage technology use will rank at level 1 in the TI-CMM. However, promoting an innovation within a culture typically resistant to change requires a person appropriately placed within the administrative structure and with personality traits that allow him or her to effect change. So reliance on an influential `champion' does become an important factor in the middle levels of the TI-CMM, as formalized plans and processes take shape.

Presence of a comprehensive implementation plan constitutes another key factor. Although five of the six schools in this study had some form of a technology plan, even if provided by the School Board, none of them included detailed plans for professional development, discussion of needs assessments, the format of potential training, or the desired outcomes.

Secondly, the technology plans lacked overall pedagogical goals for technology integration. Most respondents agreed that technology could be useful, but could not list its advantages in the classroom. At all levels, the TI-CMM points out the need for specific goals to direct activities and provide benchmarks for measuring progress.

Models for Conceptualizing Change

Fault Tree Analysis and Capability Maturity Modeling can serve as formal methods of conducting the `Needs Analysis' phase of institutional planning for technology integration, to answer the questions, "What are current conditions?" and "What has to change?" The answers should help planners chart a course of action to meet their objectives. Unfortunately, in our experience, planning typically proceeds on the basis of assumptions, rather than firm data on needs. Further, planners may not be in a position to choose wisely among the variety of approaches available.

In another line of research, we have examined alternative ways of conceptualizing and dealing with the task of integrating teaching technologies. Ives' (2002) review of the research on efforts to integrate teaching technology in universities revealed several models of how instructors behave in this context, including the Barriers model (Ertmer, 1999), Facilitating Conditions (Surry & Ely, 1999); the Technology Acceptance model ( Davis, 1989), the Technology to Performance Chain model (Goodhue et al. 1997), and Diffusion models (e.g., Rogers, 1995). She suggests that this work points to the need for integration initiatives to be carried out on an institution_wide scale. She has found five different approaches to institution-wide planning, and categorized them into three broad types: Planning Models, Design Models and Implementation Models. Table 7 briefly characterizes the three types, with specific examples of each type.

Planning models try to create efficient business plans, largely through top-down actions intended to manage resources and change work-culture, e.g., Knowledge Media (Daniel, 1999) and Postindustrial (Bates, 2000). Design models work toward learner-centred instruction through a mix of top-down / bottom-up actions designed to change the operating environment and change instructors' goals, e.g., Virtual Campus, (Paquette, Ricciardi-Rigault, de la Teja, & Paquin, 1997) and Conversational (Laurillard, 2002). Implementation Models seek competitive advantage by distributing resources to enable grass-roots instructors to change the way they teach, e.g., Ubiquitous Computing, (Brown, 1999).

Table 7 shows that the different planning models focus on different types of goals, different types of strategies and tactics, and therefore, lead to different degrees of completeness in attending to the SPECTRA factors. Consequently they tend to favour some members of the participant organizations over others (Ives, 2002, Table 20).

Table 7.>
Brief characterizations of the three types of technology integration models

To remedy this, Ives has proposed a modification of Reigeluth's Educational Systems Development approach (ESD) (Reigeluth, 1995) as a means to effectively integrate the perspectives of three key subcultures in educational institutions: faculty, support staff and administrators. Her ESDesign model focuses on designing for learning as a process that all participants can share, offering them a methodology to address two central questions: Why do we need teaching technologies? What choices do we have for fulfilling this need?

In extending the scope and scale of ESD, Ives adds the key element of strategy formation, so that her ESDesign model consists of four spheres of planning activity: Strategy, Design, Implementation and Evaluation (Figure 2). To emphasize how they interact and overlap, the spheres can be imagined as rising off the page in three rather than two dimensions. Key processes are indicated within each of the spheres.

Figure 2. The ESDesign model displayed as four spheres.

In the context of teaching technology integration, this model of change management offers special advantages in that it deals explicitly with the design, development, and evaluation of activities intended to improve learning. Consequently, the model directly addresses the factors that, the literature shows, affect attempts to integrate information technologies into teaching and learning.

The ESDesign model deals with factors affecting technology integration in the ways outlined in Table 8. They emerged as recommendations in Ives' literature search and case studies (modified from Ives, 2002, Table 22, p.146). Ives has organized the factors according to the level of control or "degree of freedom" that a faculty member can exert in dealing with each factor. The simple entries in the Actions column of the table disguise a very complex, integrated approach to institutional change. Table 8 (a) lists actions that can be to address first order factors, the "extrinsic" ones largely under the organization's control.

Table 8 (a).
Actions that can be taken within the ESDesign model to address extrinsic factors


Table 8 (b) lists actions that can be taken to address the respective second order factors, the "intrinsic" ones largely under the control of an individual faculty member. Most of the key factors lie outside the direct control of instructors yet profoundly affect the amount and quality of support they will need to change their way of teaching. Providing this support requires coordinated activity among the many parts of the institution which have grown up around the basic teaching enterprise: IT services, physical plant, libraries, academic management, and so on. Even with this support in place, changing the relationship between the instructor and the instructed requires new actions and expectations on both sides.

Table 8 (b).
Actions that can be taken within the ESDesign model to address intrinsic factors

ESDesign can be used to constantly focus and re-focus the attention of stakeholders in an educational system on its core business because it is inspired by and structured around the design of environments for effective learning. Instructional objectives therefore drive the choice of technologies, whether for use in the classroom or for institution-wide integration. Decisions about design, implementation and management of technology thus keep the learners' and teachers' needs in mind, at the course, program, department or institutional level.

In this respect, the ESDesign model provides a useful framework for planning and monitoring change, because it focuses on improving learning, not technology per se; and because it outlines the roles of all the major stakeholders in the academic community, thus encouraging them to enter the dialogue that must underlie matching of their respective means and ends.

One advantage of having actively teaching participants in the ESDesign process is that they more likely to be familiar with the design of instruction than other stakeholders, and have the potential to apply that experience to macro and meta-level planning activities. As in teaching, there is a formative component to the ESDesign process. By explicitly addressing learning needs in all spheres of activity, a process of continuous improvement of learning is possible, at least theoretically, as a potential application of double loop learning. What instructors learn by supporting the learning of their students at a class or course level, they should be able to apply to program or curriculum design, either directly or by sharing with their collaborators. This should help the entire organization learn to monitor and improve its performance. By nature of its cost in time, money and other resources, technology requires us to move to a steeper, more persistent learning curve.

Conclusions

As the modeling literature shows, when the integration of technology into teaching enters the academic mission, the task becomes highly complex, far beyond the compass of individual instructors. Studies of actual integration efforts indicate that a large number of factors come into play, their role varying with the specific features of the educational institution. What can we draw from experience to try to prevent an organization that hopes to harness technology from re-inventing the wheel?

We have explored a number of different approaches that might set beginners off on the right path. They could start by applying one of the tools borrowed from engineering disciplines, Fault Tree Analysis and Capacity Maturity Modeling, to conduct an analysis of the particular needs of their institutions.

In this context, Fault Tree Analysis uses a brain-storming type methodology to try to determine ahead of time what sorts of barriers a technology implementation plan might encounter (Aaron, 2001). Our experience with an application of FTA on a large scale (over 550 potential participants) indicates it can provide information which reflects general findings in the literature, but adds detail on how members of the institution expect various factors to influence technology integration in their specific context.

In our field trial, three of the four active members of the FTA Team reported that the process met their expectations and that it did not consume an excessive amount of time. However, they remained "undecided" about using FTA again. Roughly half the 75 responding members of the focal group thought the process useful. Significantly, about two thirds of them agreed that the teaching technology goal set by the FTA Team was worth achieving, indicating that the process helped in establishing consensus. However, the great majority (83%) of the focal group judged their part in the analysis too complicated, and the remainder suggested some method other than a long survey be used. Indeed, even in our simpler FTA application (avoiding the use of AND and OR gates), verifying faults and the relationships among them posed a challenge for the average respondent, undoubtedly contributing to a low level of response.

Given the reservations expressed by our sample, use of FTA on a large scale might not be an optimal approach. Rather, the technique might be better applied in small but representative task forces, perhaps as a prelude to evaluation of specific issues by focus groups. FTA might thus be a useful way of prioritizing areas of concern before planning begins. Further, Aaron (2001) recommends that fault analysis be supplemented by examining the state of mind of the participants before and during the technology implementation. Aaron did enhance this application of FTA by obtaining feedback from the end users. She suggests the level of involvement could be extended by the application of tools like the Stages of Concern questionnaire (Hall, George and Rutherford 1977), the Technology Adoption and Diffusion Model (Sherry, Billig, Tavalin and Gibson, 2000) and the Technology Acceptance Model (Davis, 1989), to guide planning actions throughout the implementation.

Capacity Maturity Modeling involves intuitively simpler methods, basically interviews and observations to determine whether an organization meets an established set of criteria. In Montgomery's field test of the technique in six schools, a supporting survey indicated participants' concerns for the same group of factors reported in the literature. However, her TI-CMM method itself examined whether the schools possessed the characteristics - the "key processes" in Table 4 - required to deal with these factors. In short, the proposed TI-CMM tests the readiness of an organization to integrate teaching technologies, we might say from the opposite perspective of Fault Analysis.

Though developed with data from a small sample of schools, Montgomery's TI -CMM appears to be a useful and manageable tool for preparing a technology integration plan. For one thing, the method does not consume a lot of resources - neither time nor personnel. Secondly, the data it requires are neither difficult to comprehend nor to access. Finally, its validity appears to be borne out in this sample by the finding that the two schools ranking highest on the TI-CMM scale (those with laptop programs) scored higher on the attitudinal scales, integrated technology into classes more often, and rated themselves higher in terms of their stage of integration and proficiency level (Montgomery, 2003, p.45). Apart from some reticence regarding their technology plans and their pedagogical goals, the respondents expressed no difficulties with the research method. Accordingly, TI-CMM offers promise as a way of determining which organizational needs should be addressed in a technology integration plan.

Ives' examination of technology integration models reveals the range of activities that can be called "technology planning". Her analysis indicates, for example, that universities can spend an enormous amount of time on plans, but still omit some of the "key processes", like implementation protocols and evaluation activities (Ives, 2002 p.161). The modeling approach offers the great advantage of exposing the different types of goals that planners can attempt to address (for example, the focus on strategy, on instructional design, or on implementation in Ives' sample). This in turn indicates what sort of factors have to be considered in the plan, and which stakeholders have to be involved. Given the special nature of educational institutions, especially their typically slow rate of change, inexperience with collective action, and low level of technological applications at the workface, Ives' development of the ESDesign model provides a more comprehensive approach to the means and ends of education.

Aaron's work extends the application of Fault Tree Analysis to planning issues in education. Montgomery develops a version of a Capability Maturity model for integrating technology in teaching. Ives' critique of different approaches to planning for technology integration at the institutional level leads her to a more comprehensive Educational Systems Design model.

Taken together, our studies demonstrate that planning for the integration of teaching technologies can become more systematic through a variety of tools and techniques. As an intensely interdisciplinary activity, this sort of planning involves many players and processes acting simultaneously, interdependently. Any attempt to model it must envision a dynamic, even cyclical process of planning, implementation, evaluation and revision. This should not surprise us, because it mirrors education itself.

Acknowledgements

The authors sincerely thank the panel of anonymous reviewers who contributed to significant improvements in the original draft of this paper.

References

Aaron, M., (2001). An exploration of the applicability and utility of fault tree analysis to the diffusion of technological innovation in educational systems. Unpublished doctoral dissertation. Concordia University, Montreal.

Ainley, J., Banks, D., & Fleming, M. (2002). The influence of IT: perspectives from five Australian schools. Journal of Computer Assisted Learning, 18(4) 395-404.

Bates, A. W. (2000). Managing technological change. San Francisco: Jossey-Bass Inc.

Becker, H. J., & Ravitz, J. (1999). The influence of computer and internet use on teachers' pedagogical practices and perceptions. Journal of Research on Computing in Education, 31(4), 141-157.

Brown, D. G. (1999). Always in touch: A practical guide to ubiquitous computing. Winston-Salem, N.C.: Wake Forest University Press.

Brush, T. (1999). Technology planning and implementation in public schools: A five-state comparison. Computers in Schools, 15(2), 11-23.

California Department of Education., (2004). The California K-12 education technology master plan recommendations. Retrieved April 27, 2004 from http://www.cde.ca.gov/la/et/rd/ctlmain.asp

Carnegie Mellon Software Engineering Institute. (2002). Capability maturity models. Retrieved April 27, 2004 from http://www.sei.cmu.edu/cmm/cmms/cmms.html

Daniel, J. S. (1999). The mega-universities and knowledge media: Technology strategies for higher education (2nd ed.). London: Kogan Page.

Davis, F. D. (1989). Perceived usefulness, perceived ease of use, and user acceptance of information technology. MIS Quarterly,13, 319-340.

Dexter, S., Anderson, R., & Becker, H. (1999). Teachers' views of computers as catalysts for changes in their teaching practice. Journal of Research on Technology in Education, 31(3), 1-26.

Dicks, D. J. (2000, November). Changing how we teach. Plenary presentation, Educational Technology Conference. Concordia University.

Ertmer, P. A. (1999). Addressing first- and second-order barriers to change: strategies for technology integration. Educational Technology Research and Development, 47(4), 47-61.

Fuller, H. L. (2000). First teach the teachers: Technology support and computer use in academic subjects. Journal of Research on Computing in Education, 32(4), 511 _ 537.

Goodhue, D., Littlefield, R., & Straub, D. W. (1997). The measurement of the impacts of the iic on the end-users: the survey. Journal of the American Society for Information Science, 48(5), 454-465.

Hall, G. E. (1976). The study of individual teacher and professor concerns about innovations. Journal of Teacher Education, 27(1), 22-23.

Hall, G. E., George, A. A., & Rutherford, W. L. (1977). Measuring stages of concern about the innovation: a manual for the use of the SoC Questionnaire. (ERIC Document Reproduction Service Number ED 147 342).

Hasselbring, T., Barron, L., & Risko, V. (2000). Literature review: Technology to support teacher development. Washington, D.C.: National Partnership for Excellence and Accountability in Teaching.

Hooper, S., & Rieber, L. P. (1995). Teaching with technology. In A. C. Ornstein (Ed.), Teaching: Theory into practice. Needham Heights, MA: Allyn and Bacon.

Indiana Department of Education, Office of Learning Resources (2002) Projects. Retrieved April 27, 2004 from http://doe.state.in.us/olr/projects.html

Ives, C. (2002). Designing and developing an educational systems design model for technology integration in universities. Unpublished doctoral dissertation, Concordia University, Montreal.

Jonassen, D. H., Tessmer, M., & Hannum, W. H. (1999). Task analysis for instructional design. Mahwah, NJ: Lawrence Erlbaum Associates.

Laurillard, D. (2002). Rethinking university teaching: A Conversational framework for the effective use of learning technologies. London: Routledge Falmer.

Leggett, W. P., & Persichitte, K. A. (1998). Blood, sweat, and TEARS: 50 years of technology implementation obstacles. Techtrends, 43(3), 33-39.

Montgomery, B. (2003). Developing a technology integration capability maturity models for K-12 schools. Unpublished M.A, thesis. Concordia University, Montreal.

Paquette, G., Ricciardi-Rigault, C., de la Teja, I., & Paquin, C. (1997). Le Campus virtuel: un réseau d'acteurs et de resources. Journal of Distance Education, XII(1-2), 85-101.

Reigeluth, C. M., (1995). Educational systems development and its relationship to ISD. In G. J. Anglin (Ed.), Instructional technology: Past, present, and future (2nd ed., pp. 84-93). Englewood CO: Libraries Unlimited, Inc.

Rogers, E. M. (1995). Diffusion of innovations (4th ed.). New York: The Free Press.

Rogers, P. L. (2000). Barriers to adopting emerging technologies in education. Journal of Educational Computing Research, 22(4), 455-472.

Schrum, L. (1995). Educators and the Internet: A case study of professional development. Computers and Education, 24(3), 221-228.

Sherry, L., Billig, S., Tavalin, F., & Gibson, D. (2000). New insights on technology adoption in schools. T.H.E. Journal, 27(7), 42-46.

Stephens, K. G. (1972). A fault tree approach to analysis of educational systems as demonstrated in vocational education. Unpublished doctoral dissertation, University of Washington, Seattle.

Stephens, K. G., & Wood, R. K. (1977-78). Thou shalt not fail: An introduction to Fault Tree Analysis. International Journal of Instructional Media, 4(4), 297-307.

Surry, D. W ., & Ely, D.P. (1999). Adoption, diffusion and implementation and institutionalization of educational technology. Retrieved January 29, 2001, from http://www.coe.usouthal.edu/faculty/dsurry/papers/adoption/chap.htm

Tapscott, D., (1999). Growing up digital: The rise of the net generation. Toronto: McGraw Hill College Division.

Wood, R. K., Stephens, K. G., & Barker, B. O. (1979). Fault Tree Analysis: An emerging methodology for instructional science. Instructional Science, 8, 1-22.

Wozney, L., Venkathes, V., Abrami, P.C. (2001). Technology implementation questionnaire (TIQ): Report on results. Centre for the Study of Learning and Performance, Concordia University.


ISSN: 1499-6685



Copyright (c) 2004 Mandie Aaron, Dennis Dicks, Cindy Ives, Brenda Montgomery

Creative Commons License
This work is licensed under a Creative Commons Attribution 4.0 International License.