Educational Rationale Metadata for Learning Objects

Canadian Journal of Learning and Technology

Volume 28(3) Fall / automne, 2002

Educational Rationale Metadata for Learning Objects

Tom Carey

Jonathan Swallow

William Oldfield

About the Authors

Tom Carey is a Professor of Management Sciences and Associate Vice-President (Learning Resources and Innovation) at the University of Waterloo, where he also directs the Centre for Learning and Teaching Through Technology [LT3]. Tom Carey can be contacted at: tcarey@uwaterloo.ca

William Oldfield is a librarian at the University of Waterloo, seconded to the LT3 Centre to lead activities in learning object standards and repository development. William Oldfield can be contacted at: wroldfie@library.uwaterloo.ca

Jonathan Swallow develops learning objects and evaluates their impact as a Principal Instructional Designer in the LT3 Centre at the University of Waterloo and the Interactive Learning Centre at Trent University. Jonathan Swallow can be contacted at: jswallow@trentu.ca

Abstract

Instructors searching for learning objects in online repositories will be guided in their choices by the content of the object, the characteristics of the learners addressed, and the learning process embodied in the object. We report here on a feasibility study for metadata to record process-oriented information about instructional approaches for learning objects, though a set of Educational Rationale [ER] tags which would allow authors to describe the critical elements in their design intent. The prototype ER tags describe activities which have been demonstrated to be of value in learning, and authors select the activities whose support was critical in their design decisions.

The prototype ER tag set consists descriptors of the instructional approach used in the design, plus optional sub-elements for Comments, Importance and Features which implement the design intent. The tag set was tested by creators of four learning object modules, three intended for post-secondary learners and one for K-12 students and their families. In each case the creators reported that the ER tag set allowed them to express succinctly the key instructional approaches embedded in their designs.

These results confirmed the overall feasibility of the ER tag approach as a means of capturing design intent from creators of learning objects. Much work remains to be done before a usable ER tag set could be specified, including evaluating the impact of ER tags during design to improve instructional quality of learning objects.

Rsum

Les chargés de cours qui recherchent des objets d’apprentissage dans les logithèques en ligne seront guidés dans leur choix par le contenu d’objets, les caractéristiques des étudiants à qui elles s’adressent et le processus d’apprentissage incorporé dans l’objet. L’article rend compte d’une étudede faisabilité afin de déterminer si les métadonnées enregistrent des informations axées sur le processus sur les démarches pédagogiques en termes des objets d’apprentissage, par l’intermédiaire d’une série de marqueurs de raison d’être éducative [ER] qui permettraient aux auteurs de décrire les éléments essentiels de leur intention du concept. Les marqueurs ER prototypes décrivent les activités reconnues comme détenant une valeur pédagogique et les auteurs sélectionnent les activités dont le soutiena été essentiel à leurs décisions de conception.

La série prototype des marqueurs ER consiste en des descripteurs de l’approche pédagogique utilisée dans la conception, plus en des éléments auxiliaires optionnels pour les Commentaires, l’Importance et les Caractéristiques qui réalisent l’intention du concept. La série de marqueurs a été mise à l’essai par les créateurs de quatre modules des objets d’apprentissage, dont trois à l’intention d’étudiants du niveau postsecondaire et un à l’intention d’étudiants de la maternelle à la douzième année et de leur famille. Dans chacun des cas, la série de marqueurs ER leur a permis d’exprimer de façon succincte les approches pédagogiques clés incorporées dans leur conception.

Ces résultats ont confirmé la faisabilité générale de l’approche des marqueurs ER en tant que moyen de capturer l’intention du concept des créateurs des objets d’apprentissage. Beaucoup de travail reste à faire avant qu’une série de marqueurs ER utilisables puisse être déterminée, y compris l’évaluation de l’impact des marqueurs ER pendant la conception pour améliorer la qualité pédagogique des objets d’apprentissage.

Introduction

Interactive instructional software is increasingly being made available in the form of modular learning objects, as the various papers in this Special Issue of the Canadian Journal of Learning Technology demonstrate. The learning object format holds the promise of widespread availability of high quality interactive learning, through the exchange and re-use of learning objects - and therefore an increased return on the invested knowledge and resources of their creators.

This paper focuses on learning objects encompassing a learner activity, i.e. more than just a media asset or other digital resource. For the potential for re-use to be achieved at this level, the learning objects must have the following characteristics to be re-used in multiple contexts:

  • A modular implementation to provide technical interoperability in re-use environments.
  • A cohesive unit of study, reflecting learning needs and domain knowledge within a subject.
  • A coherent learning design, consistent with the educational approaches of potential re-users.

In addition, the socio-technical environment must support re-use, including a technology infrastructure to make learning object exchange accessible and a social context to make exchange and re-use culturally acceptable (even normative).

Much current attention is focused on the technology infrastructure to make learning objects accessible, to both learners and to instructors who currently structure much of the learning process. The development of metadata to describe learning objects is part of this effort.

Instructors searching for learning objects in online repositories will be guided in their choices by the following kinds of information:

  • Content to be addressed to meet the intended learning objectives.
  • Characteristics of the learners, especially age range and prerequisite knowledge, perhaps motivation level and other details which influence the choice of instructional approach.
  • Process aspects of learning supported by the object, such as learner tasks and the instructional approach or values embedded in the object.

This project investigated a new type of metadata: a set of Educational Rationale tags to allow creators of learning objects to describe the critical elements in their design intent. Our goal was to determine the feasibility of providing process-oriented information about instructional approach. The purpose of such metadata would be to guide potential users of learning objects, primarily instructors choosing objects to support learners and potentially learners themselves. Additional users of the metadata could include learning object developers selecting components for reuse in aggregate learning objects and authors selecting instructional strategies for application in new learning objects they are designing. As noted below, our investigation suggests that Educational Rationale metadata could have positive impacts on both the technical accessibility and the cultural acceptability of widespread learning object re-use. By capturing the creators’ design intent, the metadata could make it easier for other instructors to find appropriate learning objects consistent with their educational approach. The design and development of the metadata may also engage the instructional community in dialogue about the educational values and rationale underlying interactive learning experiences.

Prototype Educational Rationale Tags

Current authoring tools enable authors to create learning objects, on their own or with the help of technology savvy students (Carey, Harrigan, Palmer and Swallow, 1999). This process offers several links to instructional design theories and principles. For example, instructors who take part in learning object development have an opportunity to rethink curricula (Frayer, 1999). Students who help develop learning objects gain from engaging in authentic, challenging, and multidisciplinary tasks (Stites, Hopey and Ginsburg, 1998). Most importantly, the learning objects can be tailored to a specific set of instructional goals, and the learning needs of a specific set of students.

The scope of this project was to develop a set of metadata tags to make information about instructional strategies more explicit. In order to be successful, the descriptors must allow informed selection and reuse, and should also be general enough to reveal similarities between learning objects.

We have assumed throughout that the learning is active: the learners’ tasks involve some form of interactive engagement in which they manipulate content objects interactively. Passive reading of online information can be included in the context of a more active task.

The metadata tags we developed and tested are listed in Table 1. In the following sections we describe the evaluation conducted on these prototype tags and some of the other approaches we tried which were less successful in capturing the intended information.

Table 1. Prototype Educational Rationale Metadata Tags

The following additional levels of description proved useful in capturing design intent:

  • Comments on the educational rationale.
  • Relative importance, measured by implicit/explicit1 or amount of expected learner time or prioritization of ER tags by learning object creator.
  • Annotation by specific features, where this provides additional insight about the instructional approach, e.g., case study, war stories, hints, prompts, randomness, opposing opinions, guide, coach, pre-emptive feedback, scaffolding.

Research Studies and the Basis for Educational Rationale Metadata

The Educational Rationale metadata tags used in this study were developed based on a review of educational research literature. We have attempted to choose tags which are consistent with the principles of learner-centered design (Communications of the ACM, 1996). We came to the conclusion that while there are multiple ways to implement learner-centered design principles, in the end, what is important is what learners do. It is learners’ cognitive activity that determines what is learned. Therefore, we analyzed the literature to find cognitive activities that are generally accepted as being of value in learning. The prototype tags reported in this study represent our best attempt to describe a comprehensive set of cognitive activities which others have reported to be of benefit to learners.

Anchor New Knowledge in Authentic Contexts [Anchor]

An "anchor", such as a case-study or problem situation should provide the basis for designing learning activities (CTGV, 1990); Knowledge should be presented in the context of where and how it is actually used (Lave & Wenger, 1990). Learners can gain expertise by becoming involved in a community of practice (Lave & Wenger, 1990).

Set a Goal to Solve a Non-Trivial Case or Problem [Goals]

Instruction should be casebased (Spiro, Feltovich, Jacobson, & Coulson, 1992). Adults tend to organize learning around problems rather than content (Knowles, 1984); Learning is facilitated when learners must try to solve practical, social, personal or research problems (Rogers & Freiberg, 1994). It is important to identify appropriate goals when confronted with a problem (Anderson, Boyle, Farrell & Reiser, 1987).

Develop Motivation to Perform Tasks and Understand Knowledge [Motivate]

Student motivation can be achieved by attention-getting, relevance-producing, confidence-building, and satisfaction-generating strategies (Keller, 1987). An adult’s motivation to learn depends largely upon how relevant the material is to their job or personal life (Knowles, 1984). Students must be willing and able to learn as a result of the experiences and contexts provided by instruction (Bruner, 1966).

Apply Theory in Practice [Apply]

Learners first acquire knowledge as declarative information, and make inferences from this declarative knowledge to acquire procedural knowledge (Anderson et al., 1987). Practice enables learners to refine their skills, and tune their knowledge (Rumelhart & Norman, 1981). Students should be given control over the amount of practice they receive (Merrill, Li, & Jones, 1991).

Employ Multiple Styles of Learning [Styles]

Learners should be encouraged to apply their own learning strategies and styles (Sternberg, 1983). Learning activities should be flexible enough to motivate the use of multiple forms of intelligence (Gardner, 1993).

Customize the Learning Agenda [Customize]

It is important to involve adults in the planning of their instruction (Knowles, 1984); Learners should be encouraged to employ their own strategies and styles (Sternberg, 1983). Students should be given opportunities to participate in all aspects of the learning process, including control of its nature and direction (Rogers & Freiberg, 1994).

Monitor Comprehension and Adjust Learning Strategies [Monitor]

Learners need to be able to monitor their comprehension, and choose appropriate learning strategies, in order to use exploratory media effectively. Prompts and self-checks can help learners monitor learning and choose strategies (Park & Hannafin, 1993).

Adapt Task Difficulty to Match Needs and Capabilities [Adapt]

Instruction should be flexible so that students are willing and able to learn (Bruner, 1966). Adults should be encouraged to attempt more challenging problems, while given some control over the organization of their learning programs (Cross, 1981).

Engage in Expository or Teaching Activities [Teach]

Tutors of a variety of ages have been found to gain as much as or more from the tutoring process as their tutees (Cohen, Kulik & Kulik, 1982).

Use Trial and Error to Discover Something New [Discover]

Learning is facilitated when students are allowed to explore, and are given guidance when they ask for explanations of an unexpected event (Merrill, 1997); Skills can be developed by enabling learners to use successive approximation, with feedback provided immediately after errors (Anderson et al., 1987); Whenever possible, learners should be taught to discover processes, rather than receiving them already formulated (Landa, 1976). Experience (including mistakes) is the most effective basis for learning activities (Knowles, 1984).

Collaborate to Accomplish Part of the Learning Task [Collaborate]

Learners benefit from social interaction and collaboration (Lave & Wenger, 1990; Vygotsky 1978).

Engage in Self-Evaluation [Evaluate]

It is important to involve adults in the evaluation of their instruction (Knowles, 1984); Selfevaluation should be the principal method by which progress or success is assessed (Rogers & Freiberg, 1994).

Reflect on the Learning Process [Reflect]

Time and support for reflection provides learners with a opportunities to construct meaning, and a needed break from ongoing presentation (Park & Hannafin, 1993).

Confront and Resolve Misconceptions [Misconceptions]

Learners can compartmentalize experiences, allowing naive and ritual patterns of misunderstanding to persist after instruction (Perkins & Simmons, 1988). Sources of misconception include applying everyday meanings for technical terms and simplifying the internal structure of a concept (Laurillard, 1993).

Extrapolate Beyond the Information Provided [Extrapolate]

Learners actively construct knowledge by manipulating information, and by extrapolating beyond the information given (Bruner, 1966). Support should be provided, so that learners can build their own representations of knowledge (Spiro et al., 1992).

Relate New Knowledge to Prior Knowledge [Relate]

Learning activities should enable learners to make use of their prior experiences (Cross, 1981). Related prior knowledge is the most important factor in mediating the acquisition of knowledge (Park & Hannafin, 1993).

Examine New Knowledge From Different Perspectives [Perspectives]

Content should be represented in multiple ways, with many, diverse examples. (Spiro et al., 1992).

Differentiate Knowledge Types [Differentiate]

Much of the knowledge in complex domains is context-dependent (Spiro et al., 1992); Algorithms and heuristics are more valuable than prescriptions (Landa, 1976). Learners need to acquire context-dependent declarative and procedural knowledge, heuristics, and an understanding of how to use that knowledge, in order to become successful problem solvers (Schoenfeld, 1985).

Integrate New Knowledge [Integrate]

The nature of illstructured domains requires that learners interconnect knowledge. (Spiro et al., 1992). Learners need to acquire integrate declarative and procedural knowledge, heuristics, and control process, in order to become successful problem solvers (Schoenfeld, 1985). The more information is processed during learning, the more it will be retained and remembered (Craik & Lockhart, 1972).

Elaborate New Knowledge [Elaborate]

Elaboration activities are associated with increased learning, and can be prompted (Hamilton, 1990). It is important to structure content in order to make it more easily understood (Bruner, 1966). Learners should be provided with curriculum materials that allow exploration (e.g., interactive videodisc programs) (CTGV, 1990).

Think Critically About New Knowledge [Critique]

Learning activities avoid over-simplifying content, and learners must be given opportunities to develop their own representations of knowledge from multiple, varied perspectives. (Spiro et al., 1992). Learning improves as the amount of mental effort increases (Park & Hannafin, 1993).

Each of these clusters from the research literature represents a body of empirical data about positive impacts on learning outcomes. A more formal clustering technique could be developed to link research papers pursuing the same theme. Even with the informal clustering carried out for this experimental study, the definition of tags around clusters of research papers has two potential implications:

  1. The set of tags can be extended as new empirical data appear in the research literature. However, there is a built-in constraint on idiosyncratic extensions: a suggested new category of Educational Rationale would require the appearance of a cluster of papers in the literature which was differentiated form those already cited.
  2. The choice of Educational Rationale tags can imply the results expected by the designer if an empirical study of learning impacts were undertaken. Ideally, all learning objects would be subjected to rigourous empirical evaluation; in practice, this is not feasible. But having the designers indicate their expectation of where and how the learning object would demonstrate impacts can provide insight into their intentions for enhancing the learning process.

Example of a Metadata Description

The following example of ER metadata was developed as an example for the evaluation described in the next section.

Learning Object: FrogHeart Physiology Lab [reference from EdMedia]
Authors: Professor Norm Scott, Department of Biology, U. of Waterloo
Dr. Jonathan Swallow, LT3 Centre, U. of Waterloo
Overview: FrogHeart was developed for a senior Biology class investigating the effect of pharmaceuticals applied to the heart.
Educational Rationale prepared by: Dr. Jonathan Swallow

The design reflects the importance of having the learners engage in activities intended to engage learners in the following learning:

Anchor new knowledge in authentic contexts [Anchor]
Comments: The Frog Heart Physiology Lab is a laboratory simulation. It replaces a wet lab within a working multimedia model. Frog Heart is designed to provide learners with an authentic laboratory experience.
Importance: High priority. (implicit; 50 - 75% of total time with Frog Heart package)
Features: simulation

Think critically about new knowledge [Critique]
Extrapolate beyond the information provided [Extrapolate]
Develop motivation to perform tasks and understand knowledge [Motivate]

Comments: It was important that learners do more than just apply treatments and record the most obvious results. We wanted them to look more carefully at the data generated by the treatments they applied, to find less obvious patterns, and then to try to determine the reasons for what they observe. Some encouragement was provided by the laboratory assignment. Learners were instructed to look for as many changes as they could find, and also to note variables which did not change. Support was also provided by worksheets which learners used to make predictions about the results of different treatments. Learner predicted results by drawing pictures of the waves they expect to result from each treatment. By making these drawings, learners had to manipulate a fuller set of variables, and were motivated to match their predictions with the results generated by the simulation.

Importance: Highest priority. (explicit; 25 - 50% of total time with Frog Heart package)
Features: prediction worksheets, assignment
Customize the learning agenda [Customize]

Comments: It was important that learners discover patterns and ask their own questions. They were prompted to ask questions, and worksheets were provided on which they were to place their predictions. These predictions became the basis for a significant part of the learning agenda.
Importance: High priority. (implicit; 5 - 10% of total time with Frog Heart package)
Features: prompts, prediction worksheets

Confront and resolve misconceptions[Misconceptions]
Comments: Prior knowledge was elicited by having learners predict the results of treatment. Discrepancies between predicted and actual results forced learners to confront their misconceptions. They were assigned the task of explaining these discrepancies.
Importance: Highest priority. (implicit; 10 - 25% of total time with Frog Heart package)
Features: prediction worksheets, assignment

Evaluation of Prototype ER Metadata

Ease of Use of Prototype Metadata

Our target was to have a `walk up and use’ comfort level. The participating learning object creators were given the list of ER tags in Table 1, i.e. without the additional notes above referring to the instructional design research on which they were based. They were also given the worked example from FrogHeart that had been prepared as a beta case by one of the project staff.

The authors were asked to choose ER tags that described the educational approach used in the design of the learning object, in response to the following instructions:

Each of the following learner activities has been shown to be valuable in certain learning contexts. In designing the use of your learning object, you will have chosen to emphasize some of these activities and to omit others. Indicate which activities you regard as important in understanding the rationale - implicit or explicit - behind your design.

All of the authors reported that they were able to use the tags with ease.

Most of the authors were able to adopt the learner-centred perspective implicit in the metadata. There was some ambiguity about Setting Goals as a learner activity: one author confused the objective of encouraging the students to develop new goals around metaskills with their actions to actually set a goal for themselves [which was not part of the learning object activities]. A similar confusion occurred with another author who suggested additional metadata tags which addressed objectives rather than learner activities, as noted below. This suggests that the relationship of the following design aspects should be clarified for authors: a) instructional objectives, b) learner tasks as set in the learning object, and c) the educational rationale which derives the latter from the former.

Effectiveness of Prototype Metadata

All the authors reported that they were able to express the critical aspects of their designs using the metadata tags. Of the 21 prototype tags, 18 were used, with the following frequencies:

  • 5 tags were used in only one system.
  • 10 tags were used twice.
  • 3 tags were used three times.
  • 1 tag was used in all four systems [Anchor].

Three tags were unused: Teach, Relate and Differentiate. The four authors used between 6 and 12 separate tags to describe their designs.

Suggested Additional Tags

One of the authors suggested that additional tags were required to fully capture the educational rationale. The additional tags were:

  • Access multiple levels of information with ease.
  • Explore related links.
  • Develop an enduring interest in the subject.

The author’s explanation for the tags is given with the Rainforest description in Appendix A. The first two are described as "depending on interest and ability" and thus appear to be instantiations of the Adapt tag, whose description might be extended to "match needs, interests and capabilities". The third tag is described as "the overriding learner objective" and its introduction therefore appears to be an instance of the confusion of objectives, tasks and rationale as outlined above.

Insights During the Evolution of the ER Tags

Several insights were key to development of the prototype ER tags.

  1. Descriptions of instructional methods - receptive, directive, guided discovery, exploratory - were found to be too general to convey the desired information. Also, a single learning object might incorporate more than one such method, so more diversity and specificity was needed.
  2. We began by describing what the learning object did, but found it much more helpful to emphasize what the learners do. The ER tags thus describe activities that have been demonstrated to be of value in learning, and authors select the activities whose support was critical in design decisions. This encourages learner-centred design thinking, and illuminates the `latent curriculum’ in which the design of a learning object is situated and in which re-use might be most appropriate.
  3. Details of the design, which can only be understood in context, should not appear in the data, since a meaningful context will not be present, e.g., many learning objects provide task scaffolds to reduce user effort on a task. Knowing that there are scaffolds deployed is not as informative as knowing why it was important to include the scaffolds: to maintain motivation, to adjust task difficulty for various learner backgrounds, to encourage `trial and error’, etc.
  4. We knew that any attempt to select a subset of instructional theories or approaches would be contentious and therefore counterproductive. Accordingly, we intentionally included ER tags that could be redundant, i.e., provide more than one way to describe the same design rationale elements.
  5. We also intended for the ER tags to be extensible, so that our choice was not perceived as a statement about preferred instructional design approaches. This might have led to a profusion of idiosyncratic data that would counter the goal of promoting reusability. To limit such unproductive extensions, we suggest that any additional ER tags be documented from the research literature on instructional design. This allows for growth, but provides a criterion to anchor such extensions in empirical studies of learning impacts.

Conclusion and Future Actions

Overall, we were pleasantly surprised that the instructional design principles could be so concisely stated and so easily related to the authors’ own design intent. The variability amongst the educational rationales was a positive indicator that useful information was being captured. The learner-centred format of the metadata tags, combined with the degree of author selection as to what was important for their target learners, avoided the obstacle of requiring consensus on higher level constructs or patterns such as problem-based learning, situated instruction, cognitive apprenticeship, etc. The prototype ER tags represent the authors’ personal view rather than an objective judgment; they are therefore open to question as to how successful the intent was realized and to some ambiguity about how each author interpreted them. However, it is our assessment that this modest tradeoff is easily justified to achieve the ease of use demonstrated.

Much work remains to be done before a standard ER tag set can be specified. Next steps should include the following:

  • Test the ER metadata for re-users of learning objects.
  • Revise the prototype ER metadata through a larger consensus-building project.
  • Develop guidelines for use of Educational Rationale metadata.
  • Relate ER metadata to appropriate modularity for learning objects.
  • Develop additional metadata to include evaluation knowledge.

Test the ER Metadata for Re-Users of Learning Object

This project has demonstrated the feasibility of applying an ER tag set to capture critical design knowledge from authors of learning objects. The other essential participants will be users of learning objects, in particular instructors seeking to identify objects for re-use and adaptation. A further study will be required to demonstrate the feasibility of the prototype ER tags to provide understandable and applicable information to this group.

Revise the prototype ER metadata through a consensus-building project

The prototype ER tags were generated from the research literature cited in the references. We determined during the project that it was easier to find consensus language for these principles than for learning objectives or learner tasks, so they hold the most potential for a fixed vocabulary with standard meaning. The authors’ choose the instructional approaches, which most influenced their designs, so consensus is not required on the relative importance of the various educational philosophies [thankfully].

However, a wider consensus will be needed on what tags to include and possibly on how to group them in meaningful clusters. This could, for instance, be part of a larger program of protocol development within the IMS initiatives. As mentioned in the previous section, there was some confusion in distinguishing the educational rationale from other design aspects such as learner tasks and instructional objectives. For example, at this time the Learning Design working group seems likely to include metadata for these items in its protocol proposals, in line with the way these elements are handled in the various Educational Modeling Languages.

Relate ER Metadata to Appropriate Modularity for Learning Objects

One of the essential factors for re-use is determining the right size or `granularity’ for learning objects. At the moment, these vary from single media elements to packaged courses with dozens of hours of instruction. There are a number of initiatives underway to apply software engineering guidelines - such as those on cohesion and coherence - to learning object design.

The development of Educational Rationale metadata offers an additional rule-of-thumb: a learning object is too large if the descriptors of its educational rationale become too numerous. That is, a coherent design will present learners with a consistent instructional approach; a description with numerous metadata tags likely reflects a learning object, which is in effect several coherent learning objects which could be partitioned [or possibly a design of low quality]. Studies could be developed to assess this hypothesis, e.g., various empirical tests could be devised to compare ER tag structure with instructional coherence as judged by human evaluators.

Develop Additional Metadata to Include Evaluation Knowledge

Instructional design, like user interface design (Rubin, 1994), should be learner centred (Communications of the ACM, 1996). Learning objects need to be developed based on an understanding of the learning needs of students. Learning object development should be an iterative process, with frequent user testing of components as they are designed and implemented. Learning object usage patterns cannot be assured without studies of actual usage.

Including evaluation information in metadata is expected to have at least two benefits: i) it will provide a more objective measure of how the learning object is used by learners, and ii) it will encourage learning object developers to conduct more user testing. More user testing is expected to result in better quality learning objects. It is at least as important to know what learners do with a learning object when they use it, as to know what the learning object does for learners.

Another kind of evaluation knowledge that should be captured and described in metadata is third party evaluation of learning object quality. Information from an assessment of a learning object by its developer can be useful. However, an evaluation of a learning object by other teachers and developers who have attempted to use it could be even more valuable. Inclusion of these data would make the metadata a `living system’, which changes as the learning object it indexes is evaluated.

Acknowledgements

This work was funded in part by Industry Canada - Multimedia Learning Group. We thank Nadia Lombardi and Yuri Daschko for their insights and support, and the learning object authors who participated in our evaluations (Roger Suffling, Pascale Proulx, Mieke Delfgaauw, and Celia Clark). We also acknowledge the insightful comments of our colleagues in the University of Waterloo’s Centre for Learning and Teaching Through Technology.

References

Anderson, J., Boyle, C., Farrell, R. & Reiser, B. (1987). Cognitive principles in the design of computer tutors. In P. Morris (Ed.), Modeling Cognition . NY: John Wiley.

Bruffee, K. A. (1995, JanuaryFebruary.). Sharing our toys: Cooperative learning versus collaborative learning. Change, 27(1), 1218.

Bruner, J. (1966). Toward a Theory of Instruction. Cambridge, MA: Harvard University Press.

Carey, T.T., Harrigan K., Palmer A. & Swallow J. (1999). Scaling up a learning technology strategy: Supporting student/faculty teams in learner-centred design. Journal of the Association for Learning Technology, 7(2), 15-26.

Cognition & Technology Group at Vanderbilt (1990). Anchored instruction and its relationship to situated cognition. Educational Researcher , 19 (6), 210.

Cohen, P.A. Kulik, J. A. & Kulik, C C (1982). Educational outcomes of tutoring: A meta-analysis of findings. American Educational Research Journal, 19, 237-248.

Collins, A., & Stevens, A. L. (1983). A cognitive theory of inquiry teaching. In C. Reigeluth (Ed.), Instructional Design: Theories and Methods (pp. 65-119). NY: Academic Press.

Communications of the ACM (1996). Special Issue on Learner-Centered Education, 39(4), 25-49.

Craik, F. & Lockhart, R. (1972). Levels of processing: A framework for memory research. Journal of Verbal Learning & Verbal Behavior , 11, 671684.

Cross, K.P. (1981). Adults as Learners. San Francisco: JosseyBass.

Farance, F. & Tonkel, J. (1999). Learning Technology Systems Architecture, Draft 5. Retrieved March 1, 2000 from http://edutool.com/ltsa.

Frayer, D.A., (1999). Creating a campus culture to support a teaching and learning revolution. CAUSE/EFFECT, 22(2). Retrieved March 1, 2000 from http://www.educause.edu/ir/library/html/cem9923.html.

Gardner, H. (1993). Multiple Intelligences: The Theory in Practice . NY: Basic Books.

Hamilton, R. (1990). The effect of elaboration on the acquisition of conceptual problem-solving skills from prose. Journal of Experimental Education , 59, 5-17.

Keller, J. M. (1987). Strategies for stimulating the motivation to learn. Performance and Instruction, 26(8), 1-7.

Knowles, M. (1984). Andragogy in Action. San Francisco: JosseyBass.

Landa, L. (1976). Instructional regulation and control: Cybernetics, algorithmization, and heuristics in education. Englewood Cliffs, NJ: Educational Technology Publications.

Lave, J., & Wenger, E. (1990). Situated Learning: Legitimate Peripheral Participation. Cambridge, UK: Cambridge University Press.

Laurillard, D. (1993). Rethinking University Teaching. London UK: Routledge.

Merrill, M. D. 1997, November / December). Instructional Strategies that Teach. CBT Solutions, 111.

Merrill, M.D., Li, Z. & Jones, M. (1991). Instructional transaction theory: An introduction. Educational Technology, 31(6), 712.

Newell, A. & Simon, H. (1972). Human Problem Solving. Englewood Cliffs, NJ: PrenticeHall.

Park, I. & Hannafin, M. (1993). Empirically-Based Guidelines for the Design of Interactive Multimedia. Educational Technology Research and Development, 41(3), 63-85.

Perkins, D. N. & Simmons, R. (1988). Patterns of Misunderstanding: An Integrative Model for Science, Math, and Programming. Review of Educational Research, 58(3), 303-326.

Rogers, C.R. & Freiberg, H.J. (1994). Freedom to Learn (3rd Ed). Columbus, OH: Merrill/Macmillan.

Rubin, J. (1994). Handbook of Usability Testing. Toronto: John Wiley.

Rumelhart, D. & Norman, D. (1981). Analogical processes in learning. In J.R. Anderson (Ed.), Cognitive Skills and their Acquisition . Hillsdale, NJ: Erlbaum.

Schoenfeld, A. (1985). Mathematical Problem Solving. New York: Academic Press.

Spiro, R.J., Feltovich, P.J., Jacobson, M.J., & Coulson, R.L. (1992). Cognitive flexibility, constructivism and hypertext: Random access instruction for advanced knowledge acquisition in illstructured domains. In T. Duffy & D. Jonassen (Eds.), Constructivism and the Technology of Instruction. Hillsdale, NJ: Erlbaum.

Stites, R. Hopey C. E. & Ginsburg L., (1998). Assessing lifelong learning technology (ALL-TECH): A guide for choosing and using technology for adult learning. Philadelphia: National Center on Adult Literacy Report PG98-01. Retrieved October 1, 2002 from http://www.literacyonline.org/products/ncal/pdf/PG9801.pdf

Sternberg, R.J. (1983). Criteria for intellectual skills training. Educational Researcher, 12, 612.

Vygotsky, L.S. (1978). Mind in Society. Cambridge, MA: Harvard University Press.

Appendix A:

Educational Rationale for Digital Field Trip to the Rainforest

Authors: Digital Frog International, Puslinch ON [www.digitalfrog.com]

Overview of Digital Field Trip to the Rainforest:

"The Digital Field Trip to The Rainforest is a computer-based learning tool designed for K-12 students. While not all activities are suitable for the younger grades, the activities that appeal to younger students do not insult the intelligence of adults.

Our stated educational rationale is that the software should:

Lure the users in to the subject.
Encourage inquiry, exploration and discovery.
Actively involve students in meaningful learning.
Reinforce with positive feedback.
Nourish a lifelong love of the subject.

Educational Rationale prepared by: Celia Clarke
The design reflects the importance of having the learners…
Anchor new knowledge in authentic contexts
Comments: The field trip allows students to learn about the rainforest organisms etc. by "experiencing" the environment.
Importance: High
Features: Exploration and discovery is encouraged by 360 degree panoramas and the opportunity to find out about organisms and topics from within the field trip. Ambient rainforest sounds increase the illusion of being there.

Adapt task difficulty to match needs and abilities
Employ multiple styles of learning
Use trial and error to discover something new
Examine new knowledge from different perspectives
Comments: Because this software is designed for multiple grade levels, it is important that learners can perform activities based on their abilities and various "intelligences".
Importance: High
Features: Web game - can be played at standard or advanced levels, can be played at a very simple level by "reading" the silhouettes, links can be drawn based on trial and error, reasoned thinking based on prior knowledge or by conducting research by following the hyperlinks. Niches - can be played by non-readers - feedback provided by icons, or by reading the comments.

Suggested Additional Tags

Explore related links
Develop an enduring interest in the subject
Access multiple levels of information with ease.

Illustration of Additional Tags

Explore related links
Access multiple levels of information with ease.
Comments: Because this software is designed for multiple grade levels, it is important that learners can access the information on a very simple level and explore further depending on interest and ability.
Importance: High
Features: Exploration is encouraged by hyperlinks (blue text), anecdotal text on pictures (pointing finger) and definitions on almost every word (basic words are defined for ESL and younger readers; scientific words add more detail).
Develop an enduring interest in the subject.
Comments: This is probably the overriding learner objective. K-12 is the time when interest is stimulated and nourished, when decisions are made about post- econdary studies and when students form many opinions and learn to care about the world around them.
Importance: Highest
Features: Visual appeal, experiential field trip, videos, photographs, interactive games, interesting tidbits of information.

Endnotes

1. We also considered extending this to include "/supported/assessed".


ISSN: 1499-6685



Copyright (c) 2002 Tom Carey, Jonathan Swallow, William Oldfield

Creative Commons License
This work is licensed under a Creative Commons Attribution 4.0 International License.