Assessing Emergence.

Building on the tentative ideas on assessing complexity, below, it might be useful to have a look at related ideas on assessing emergent learning: .

1. Best option: separate out emergence and assessment.
Simplest way to do this is to design and event for learning and emergence, and manage it on that basis. No assessment criteria in sight. Then ...
read more here ...

Assessing Complexity

Attending a conference on digital storytelling at Cheltenham, and a session on its application in Sport, particularly community sport, we were shown a great digital story on football. It was a personal account of a student's journey through yobbo (football) behaviour to higher education.

Asked how this (excellent) presentation was assessed, the presented said: "We though it was great, but didnt know what to do with it, so we just gave it (inaudible, but it sounded remarkably like 80%)".

So... how do we assess digital story telling? Well, clearly a lot of unexpected things emerged from the process, which were new, of obvious value (of some sort) and which communicated at an important level. Sounds like complexity to me.

Here's a stab at assessing complexity:

1. Let the process happen, with minimal interference: set it up with useful constraints (define what shouldn't happen), let the process and product emerge, and let the story get told.

This is an inherently messy process, and should remain so. It needs support: technical, advice on story, sounding board for ideas, etc, but not too much direction. This seems to be in line with the CeAL approach.

2. Let the digital story lie for two weeks.

3. Ask the student to revisit the story, and think about what value it has, and for whom. In other words get the story teller to explore, set, apply, and evaluate the usefulness of benchmarks. The student should determine how this gets done, but it could for instance include a series of questions, like:

3.1 What audience/s is this for ?

3.2 For what purpose/s?

3.3 What criteria would the audience apply to assess its value (go and ask a few real people to do this)?

3.4 Considerning all this, what would you, the student, do with it?:
3..4.1 Write it down to experience, and bin it?.
3.4.2 Redo it, as there is value in there someplace?
3.4.3 Load it onto Youtube, etc
3.4.4 Lock it away, it's actually far too personal - maybe revisit it in 12 months time?
3.4.5 Use it as a resource in your professional work: If so, specify how you would do so, try it in practice, and report back.

etc, etc ...

4. Issues for the Staff to consider:
4.1 Take this as sufficient material to assess the students' ability to reflect on the process, product and value in a real community that they have specified.
4.2 You might do something more formal, and ask the student to comment on the alignment, or lack of alignment, between:
  • Their own benchmarks for the value of the story, for the specified audience and context, and
  • The benchmarks for the story set by members of the specified audience.
4.2 Or you might do your own, quick, exercise and ask people in the professional field concerned for their benchmarks and their assessment of the digital story (for the purpose that the student specified); compare it to the student's assessment (see 3, above); discuss all of this with the student, and give them a mark for their contribution to the discussion (or a written report on the discussion if necessary).

The details of these proposed procedures are not important. What is important are the principles:
  • The story telling is complex: so set up constraints and limits, not prerequisites or prescriptions or even specific outcomes.
  • Let the story emerge.
  • Leave it to incubate for a few days/weeks
  • Involve the story teller in benchmarking the story as it has emerged, and welcome, and even seek out, the unexpected and unintended.
  • Involve the story teller in defining the audience for the story-that-has-emerged.
  • Involve members of this audience in benchmarking the story, against their own criteria, and/or the story teller's criteria.
  • Focus your assessment of the student,not on their product, or even on the process of the construction and creation of the story, but rather on their ability to manage a creative process, and create and apply benchmarks in communities of inquiry, and/or communities of practice.
  • If your time is limited, invent a short version (no more than 5%) of this process, and apply it instead, preferably with your own version of these principles, after all, you are the academic.
  • If you get the student to participate in generating benchmarks (e.g. self, peer, staff, client) and comment on them, you can assess what they have learned about a whole range of things, independent of the technical quality of the digital story that they produced.

[ cross posted in learning-affordances wiki.]
external image c.gif

external image bBL.gif
external image c.gif