Phoebe in the Context of the Design for Learning Programme

Liz Masterman, 23/09/07, based on an earlier draft by Marion Manton
(Updated 26/09/07)

This document outlines the ways in which the Phoebe project is working towards the overall aims of the JISC Design for Learning (D4L) Programme, and to fit into the evaluation framework devised by Glenaffric.

A. Fit to the Aims of the D4L Programme

1. Ensure the process of designing, planning and orchestrating learning activities (‘design for learning’) in UK post-16 and higher education is based on sound pedagogic principles, is evidence-based and learner-centred.

Phoebe has been designed with the explicit aim of supporting “effective practice” in design for learning in the following ways:

• The “ontology” of D4L, (i.e. the components of a learning design, the relationships among them and the general structure of the design) has been based on evidence gathered from the field (interviews with practitioner-informants, consultation activities with JISC Experts' Group, D4L programme participants and other experienced practitioners) and from previous research (the Learning Design Tools project).
• The implementation of this ontology within the Phoebe tool is flexible, in that the actual structure and components of any plan can be customised to suit the user's needs, as well as the terminology used.
• The support and advice component draws extensively on the fieldwork described above.
• Example learning designs and case studies will be drawn from sources that have either been peer-reviewed or are from acknowledged, authoritative sites.
• The evaluation plan has a strong user focus, with “events” intended to test the usability and usefulness of the tool, to which we will invite representatives of the various envisaged user groups to work with Phoebe to accomplish tasks that are as close to real life as possible.

2. Promote the development and implementation of tools and technical standards to support the process of design for learning

Phoebe is one of a number of pedagogic planning (PP) tools, both online and paper-based, that are currently in development (largely as part of research). The project team is actively involved in the PP development community in the following ways:

• Participation in the PP “round table” at the London Knowledge Lab (July 2007), to share progress, elicit common themes and concerns, and position the different tools within the overall field of design for learning (incl. Learning Design).
• Investigation of possible integration with other PP tools. So far these have included the London Planner and Compendium (as used by the Open University).
• Active support for, and participation in, D4L initiatives by other projects (e.g. EDIT4L) and institutions (e.g. University of Greenwich Staff Conference 2007).

The project team is also disseminating the concept of pedagogic planning within a D4L framework within the wider e-learning community through:

• Conferences: e.g. CAL’07, Alt-C 2007
• Publications: e.g. chapter in forthcoming Handbook on Learning Design and Learning Objects (ed. L. Lockyer et al.).

In terms of technical standards, the Phoebe project team is keen to contribute to discussions on the requirement among the practitioner population for, and the feasibility of implementing, such standards. However, it does not currently seek actively to design, extend or otherwise promote standards.

3. Promote the sharing of expertise in design for learning, for example through sharing and re-use of effective pedagogic learning designs, use models or exemplars

The guidance material within Phoebe will incorporate learning designs, examples, case studies and summaries of research findings. In addition, it will provide access (provisionally via the oxphoebe del.icio.us account) to well established, reputable repositories of learning designs and online journals with free subscriptions.

The customisation features of Phoebe (see below) will enable institutions to add guidance material and re-usable learning designs relevant to their situation.

Integral to the sharing of “effective” learning designs is some explicit statement of what constitutes “effectiveness” in this respect, and who should determine whether a particular design meets the criteria for inclusion in the tool. The Phoebe project team is aware of its responsibilities in populating the “core” tool with appropriate materials, and welcomes any opportunity for discussion regarding how this might best be achieved in a constructive and collaborative manner.

4. Support the establishment of communities, services and resources to promote and sustain effective practice in design for learning

Establishing communities of users, delineated by individual departments and/or entire institutions is central to the project’s mission and integral to our sustainability plan.

From the technical perspective, the tool will:

• Support individual user accounts and (eventually) groups?
• Enable users to determine whether they wish to make individual learning designs available to other Phoebe users for viewing and/or repurposing, together with the copyright terms under which each design may be used.
• Allow user-administrators to customise learning design templates (components, structure and terminology), provide alternative help pages and (eventually) edit the “default” help pages?

From the social perspective, the project team will work with interested individuals and organisations to explore the demand for, and logistics of, implementing custom versions of the Phoebe tool. The eventual goal is to develop a means for departments/institutions to host their own (local) versions of Phoebe. The project team would relinquish responsibility for determining the effectiveness (or otherwise) of customised materials.

B. Contributions to the D4L evaluation questions

1. What is effective design for learning?

Phoebe has been working with provisional definitions as follows:

Design for learning: “the process by which teachers – and others involved in the support of learning – arrive at a plan or structure or design for a learning situation” (Beetham & Sharpe, 2007, p. 7) that strikes “an appropriate balance between e-learning and other modes of delivery” (JISC, 2004, p. 11). (NB this is more restrictive than the definition in the Background document to the original D4L circular 01/06: viz., “designing, planning and orchestrating learning activities as part of a learning session or programme.”)

-> As a planning tool, Phoebe fully supports this conceptualisation of D4L, and although it offers extensive advice on the use of innovative digital technologies it leaves users free to incorporate activities that use more traditional (“non e-“) media.

-> Where possible, the Phoebe team has sought to discuss what is meant by “Design for Learning” with other projects in the programme, in order to arrive at a set of aligned understandings that may be different for different groups, but have an underlying common principle.

Effectiveness in relation to design for learning: This is assessed (qualitatively if not quantitatively) in relation to 4 groups of stakeholders:

• Acceptability to practitioners – Does the espousal of D4L fit practitioners’ existing approaches to learning and teaching, or develop them in positive ways, and make acceptable demands on practitioners’ time, resources and competences?
• Outcomes for learners – Do learning sessions designed in the spirit of D4L provide learners with a learning experience that is appropriate, or adaptive, to their learning preferences, needs and goals?
• Effectiveness for organisations – Is the espousal of the D4L approach efficient in terms of organisational costs and processes, human resource management, and systems interoperability?
• Capacity-building across organisations – can D4L-inspired models and resources be shared and re-used productively by other practitioners, with minimal input on their part? To what extent do the concept and practice of D4: contribute to the development of expertise across organisations and communities/networks?

(Adapted from the Background document to JISC Circular 01/06.)

-> Phoebe is working to support the first, second and fourth of these stakeholder groups (although students are supported only indirectly), through a) its technical implementation (flexibility), b) its content, c) active engagement with potential user groups and d) participation in discussions surrounding sharing and re-use, especially issues associated with the representation of learning designs emerging from the Learning Design Tools and MoD4L projects.

2. What efficiencies are afforded by design for learning approaches?

There is a widely held – and anecdotally supported – perception that engaging with e-learning increases teachers’ workload (at least in the first instance), as they find themselves obliged to go back and re-plan lessons that they have run successfully for years, as well as to spend time getting to grips with unfamiliar technologies. There may well be no way to circumvent this process, but the design of Phoebe is intended to make it as efficient (and painless) as possible by:

• Exposing all the details that a teacher needs to think about when designing a technology-mediated learning experience, and pointing out relationships and dependencies among those details.
• Providing short explanatory introductions to alternative pedagogies, techniques and tools, with suggestions for implementation in their teaching and links to more in-depth information for those users who wish to explore the ideas/tools presented further.
• Making available examples of different pedagogies, techniques and tools in order to inspire users to incorporate them in their own teaching. These examples can range from “light bulb” suggestions of 3-4 sentences to more fully elaborated case studies and actual learning designs.
• Facilitating the sharing of learning designs, either for inspiration or to make it easier for a teacher to hand on a learning design to a successor.

3. Do design for learning approaches work?

a) As implemented within the D4L programme: each project will report its own outcomes. However, without a pre-existing consensus on what D4L approaches are, the responses are likely to be varied not just in the degree of “success” achieved (however this is measured…) but in the interpretation of that success.

b) As implemented outside the D4L community: doubtless, a number of institutions and individuals are already practising D4L without consciously recognising it as such. Their experience also needs to be captured and aligned with D4L.

c) As supported by Phoebe: the following usability measures will address this question:

• The ease with which teachers are able to construct and maintain pedagogical plans within Phoebe.
• The ease with which they are able to realise those plans as a learning experience with their students (this also depends on the additional components of the learning design e.g. handouts, over which Phoebe can have no influence).
• The suggestions that users make to improve the functionality of the tool.
• How useful they perceive the tool will be in their own practice, and in that of their colleagues and/or students (where the latter are trainee teachers)
• How effectively the tool is embedded within different institutional contexts. This depends on: i) The acceptability of Phoebe’s “core” content and the ease of customising it: i.e. how much needs to be changed (as opposed to local examples added) and how easy is it to change (functionality)?

ii) CPD/ITT policies strategies within organisations and (outside Phoebe’s influence)

4. How pleasing are design for learning approaches in use?

a) In relation to the D4L approach: users need to be made aware of the distinction between D4L and their traditional practice (= a shift of perspective and/or the use of technology in their teaching and/or the use of specific tools such as LAMS and Reload (+Moodle?) to construct and orchestrate learning experiences?), and to be given the opportunity to compare D4L with that existing practice in terms of the 4 dimensions of effectiveness defined above.

b) In relation to the support for D4L provided by Phoebe: this will be captured in usability evaluations in terms of:

• The degree of enjoyment that teachers report in their use of Phoebe.
• The number of bugs reported and resolved
• The suggestions that users make to improve the functionality of the tool.

5. To what extent do ethical and legal issues affect design for learning?

Legal issues are primarily related to:

a) the sharing and re-use of learning designs created in, or linked to from, Phoebe
b) the content of Phoebe’s guidance system written by the project team
c) any illustrations (incl. screenshots) retrieved from other sources and included in Phoebe’s guidance system.

Ethical issues attach primarily to the manner in which Phoebe is presented to practitioners. There is a danger that, when experienced teachers are introduced to new tools and approaches, they will feel that their current practice is being criticised or found wanting. Although the terms in which Phoebe is presented in CPD and ITT contexts are dependent on the facilitators, the guidance material within Phoebe must be phrased manner that respects practitioners’ expertise while offering them new things to think about.

6. To what extent do the conceptual precepts of the D4L Programme support its implementation?

There is still no articulated consensus, or range of consensual understandings, about what exactly these conceptual precepts are. In order to answer this question, therefore, the JISC D4L community needs to:

i) Discuss and reach one agreed definition, or a cohesive set of definitions, of D4L.

ii) Map this definition to practice observed outside the community that appears to follow/implement the same precepts.

iii) Co-opt practitioners both within and outside the D4L community in order to disseminate the D4L “message.”