Matthew Allen

 

Disclaimer: Live blogging

Growing Knowledge: what is the future of research?

(details)

A Times Higher Education debate hosted by the British Library, featuring Matthew Gamble, David Gauntlett, Alex Krotoski, Ben Hickey and chaired by Phil Baty.

---

Phil Baty starts the debate: it is fundamentally about the way that IT will profoundly change the nature of research. Introduces the speakers.

Hickey

(A-level student)

Has grown up surrounded by network technologies and assumes they will be crucial at his time at university. he ponders however whether the research collaboration between people and computers might lead more traditional people to question the validity of his work because the boundaries between him as researcher and technology are indeterminate. [Cyborg researcher?]. perhaps universities, because of their traditional outlook, may hinder learning and research. On the other hand maybe technology creates too narrow a vision and the voice of experience from earlier times can shed revealing light on a problem. Points to a problem – younger people with whom Hickey spoke are largely uninterested in universities and research, seeing it as irrelevant and distanced from the real-world problems they face.

What is revealing about Hickey’s contribution is the way in which someone who have grown up with technologies of networks, intelligent agents and so on construes the role of technology in research: as something that, in effect, stands OUTSIDE of the normal practices of researchers and potentially enables research and learning directly from / with computing code, without human (academic) intervention

Gamble

(PhD candidate)

Mismatch between the potential that technology provides (connectivity, immediacy and scale) and what is current normal practice in academic research. this potential is, however, what causes the problems as well. The web might become the “invisible college” which promotes the circulation of scholarly literature outside of the norms of academic journal publishing and, indeed, the formal structures of universities.

Provides example of crowd-sourcing data analysis within Galaxy Zoo project where large amounts of data was given to many individuals online for them to do mciro-analysis of data, out of interest in the subject. discovered things which the researchers were not even aware they should be looking for.

Gamble’s critique of traditional science is important: he reveals that lurking within the technologies of network collaboration is, in fact, a deeply ideological project towards openness and altruism. Open science, while often construed as made possible through the Internet and similar tools, is more about a reaction against the institutionalised narrow and profit-oriented sciences which have emerged over the past fifty years

Notes the resistance of scientists who resist open data (the so-called “selfish scientist”) and who are obsessed with publishing, not finding things out. “Altruism is quickly beaten out of young scientists”. So, there are tools for collaboration but are not used significantly.

Concludes by calling for a different mode of publishing: it’s not just open publishing, but also publishing of data, the methods, processes, the discussions about projects and so on.

Krotoski

Discussing Web 2.0 and scholarship. Ponders the reality of such technology in the real world, outside of the world of enthusiasts (such as myself I should admit). Recounts how she spoke with phd students as they commenced their studies – almost none of them had any kind of online presence, definitely not blogging and so on. Students told her that they were discouraged by their supervisors from being online and open. They certainly were not taught about how to do it. this was, from the traditional perspective, ‘wrong’.

So, she continues, what of the future? She emphasises the validity of blogs or similar: ideas can be trialled and discussed with peers, useful self-promotion (on the basis of quality, not spin), writing becomes a habit and reflection possible. Krotoski views scientific / technological research in the USA, where this use of social media and Web 2.0 is more prominent, as being influenced by industry, who are not interested in long-term peer review publishing but rapid and iterative publishing of ideas and their development.

I wonder if there needs to be greater discrimination between types of ‘web 2.0′ use [which I had discussed with Aleks before the event, so no criticism here]. This discrimination is, pretty much, about identifying the unkown, but useful tools of the web which, probably, critics of ‘web 2.0′ use but don’t realise these tools could, from another perspective, be seen as web 2.0

K. comes back to key point: how do we trust what is online; is it valid and reliable; how can we assess that? Normal position emphasised — it’s about training people to have that capacity to assess. Baty contributes a point: traditional publishing filters the content to give it more reliability.

Gauntlett

Online publishing and distribution of information is very useful, even required, for academics. Open publishing helps the world and is ethically required; it is great, too, for academics because it makes them self-reliant. moreover, the web and similar tools makes academics public intellectuals again, rather than closeted.

scholarly publishing — from a time when distribution was very limited, and filters needed because of low bandwidth. G. has a great view on the failures of the peer-review system because it assumes reviewers are entirely uninvested in the outcome except from a rational scientific perspective. Perhaps academics can do the filtering themselves by using what is good, from their view.

Gauntlett noted he first built a website in 1997; some of the most keen advocates for web 2.0 and knowledge networking are often longer-term Internet users who, perhaps, have understood the web more from a self-creative perspective?

debate now ensues

Something of a confusion emerges from the discussion between the academics about peer review – there’s a slight problem with comparing and contrasting peer review with complete ‘openness’ (eg Twitter). In fact, the discussion might more usefully concern the reshaping of peer review so that it is more productive, in improving and expanding work in a supportive manner. One example is the peer review process of Critical Studies in Peer Production.

Question from audience regarding new kinds of research methods which the Internet might produce. — too much data produces new methods; online behaviour produces new methods; nice contradiction between Gamble enthusing about the Semantic Web vs Krotoski worried about the missing human condition.

Gauntlett makes an interesting comment — it appears that crowd-sourcing can elevate people to being partners in science (as in the Galaxy Zoo), “citizen scientists”; this is like citizen journalists and so on. I read this as another example of the meme/trope of participation and democracy which is ideally or occasionally true but, in fact, is a general mythos within which hierarchies and elites persist.

In 2009 I ran a series of workshops as the first main component of my ALTC Fellowship to group brainstorm and analyse ideas about online learning and web 2.0 technologies.  During these workshops, so many good ideas were raised that I felt compelled to write up a report distilling the wisdom of more than 200 participants at 7 locations so that it might provide something of a guide for others.

At the same time, as I reflected on the workshops and what happened within them, I realised that they gave me an insight into the discourse of e-learning and Web 2.0 versions thereof in contemporary Australian higher education. Thus, I have also reported my responses to and analysis of those workshops. It’s one reason why the report has taken a while to produce and finalise.

Finally, then, here is the report Innovative Education Online:  Ideas for the future of learning & the Internet

My thanks again to everyone who attended and helped organise these events.

 

Listening to Michael Dyson, from Monash talking about portfolios in teacher education: great presentation.

Dyson says:

• Education of educators is first of all premised on turning them into people who practice self-development. gives example of very first unit. [So, care of the self is central, and making students include themselves as subjects in the learning process - nice!]
• Learning is change dramatically – globalisation, computing, and so on. [But, perhaps, there is an important qualification on some of the more optimistic claims for 'new' learning: learning is embedded within society in ways that shape those possibilities in ways that are not entirely concerned with 'better' learning. At the very least, the definition of better is contested: is it cheaper? is it more orderly and commodifiable? is to linked to national norms and needs?]
• The creating mind is the goal. [Interesting - not creative, but more positive and active - creating. Good difference]
• Reflection is essential to achieving the kind of succcesses in self-developmental learning; using Dewey (2003), emphasises “active persistent and careful consideration”; reflection is not taking “things for granted…[leading to] ethical judgment and strategic actions” (Groundwater-Smith, 2003).  [ Further work needed, perhaps, to understand reflection for this new generation, if one takes as given the significant changes in knowledge: is reflection as developed in 20th c the right kind of reflection?]
• ALACT model – action, looking back, awareness of the essential aspects, create alternatives, trial.

[This is really helpful - I like the added 5th step, compared to the normal action research 4-step model]

• “the artefacts placed in their portfolio showcase who they are and their current onling learning”; these artefacts are attached to the standards which define what it is to be an educated teacher according to outcomes required. [So portfolios are a clear negotiation of the student's understanding of those requirements and standards?]
• Exploration of the actual portfolios that students have created, using a paid-for service iwebfolio (was subsidised). Variety of successes and failures, all the material goes into a digital, not paper portfolio. Notes the fact that the metadata on when and how material uploaded is available, unlike other means of generating a portfolio. [I emphasise: the portfolio is a genuine, real requirement for teaching employment. It is authentic learning]
• Use of standards / outcomes as information architecture to drive cognition in inputting information (adding artefacts, commenting etc [So, the portfolio is 'scaffolding' into which a building goes, with a clear design brief. It might be a hghly structured knowledge engine]

I am wondering if the students genuinely are doing this work for themselves or if they imagine an audience of ‘judges’ – their teachers who grade the portfolio or the employers who might use it? Managing multiple audiences is tricky, even with technology that allows it – because if you can shape the portfolio for several audiences…. then does the self audience survive?

Then again, maybe the whole point is that the students are not yet capable of being their own audience.

Some other portfolio software (and look how it is more than just a portfolio…)

http://www.pebblepad.com

See also other posts including the first one, on Web Communications 101, which explains more of the context.

Web Publishing 206

(basic unit description)

Students doing the BA (Internet Communications) learn, in WEB101, to create a web presence that acts as the primary locus of their online identity, with links to other services and applications. In Web Publishing 206, the focus moves much more directly to writing effectively for the web (where writing can also including other media, but emphasises the written word).

The authenticity of the assessments in Web Publishing 206 are principally mobilised by requiring students to write regularly, on their blog, exploring different aspects and techniques of good online writing. The blog is assessed in its own terms, and also as the basis for students’ reflective essays which ensure that students are thinking about (as well as doing) this crucial online communication task.

Some examples of students’ blogs are:

• Eighteen Songs – WEB206 Weekly Blog
• Percussive Sweet Spot
• On the Internet, No-one Knows I’m A Blog
• Damien’s Web Publishing 206 Weblog | The Worlds of MMO’s

Notably, most students make virtually no reference to the ‘study’ component of these blogs: these are genuine blogs addressing audiences outside universities. Use of the tag Web206 however enables academic staff to look into them to find relevant content! And one student cleverly ‘colonised’ the name WEB206 : WEB206 | a Curtin University of Technology unit

While in WEB101 there was a strong sense that other students were the audience (along with the teacher), in WEB206 students are developing a much greater awareness of real audiences. In this respect, if no other, the assessment task is significantly advantaged by making it public knowledge networking.

As before, the blogging linked with other services and tools, pricipally delicious, as in these examples:

• Annoted Bibliography Feed (on the blog)

Once again, we see the value of the tag – the tag Web206 enables just the relevant links to be pulled from delicious into the blog, enabling a student to also use delicious for many other purposes. In this way, knowledge networking drives the nature of the assessment completion.

More findings from Web206 (which has only just run for the first time in late 2010)  will emerge over time. Thanks to Dr Helen Merrick, chief wrangler of publishing.

—————————————————-

The first of several posts, each relating to a different unit of study at Curtin

Introduction

Over the past two years, students in Internet Studies, Curtin University studying the BA (Internet Communications) and related courses have been doing a lot of authentic assessment involving online activities. These assignments are  authentic in that they are ‘true’ to the content of their studies (that is, aligned with the outcomes), ‘ real’ within the likely fields of employment for graduates, and ‘natural’  for the the emerging dominance of knowledge networking in society. More on these three variations on authenticity in a moment.

Not all assessments fit this pattern (nor should they), but we have seen significant improvements in the motivation of students to complete and exceed the requirements of assignments, as well as a greater degree of creativity and expression suggesting deeper engagement with learning. It has also, we think, improved students’ attention to more scholarly traditional assignments (such as essays) because of the variety we engendered across all assignment tasks. (And, it should be noted: essays are authentic – to the lifeworld of academic which also remains important as well as work and elsewhere).

Much of what makes these assessment approaches authentic is that they are public. Here, then, are some examples which suggest some of the value of embracing public knowledge networking as the basis for assessment, at least in courses that involve digital media and communications but, most likely, in any course where students need to work with, communicate and reflect on knowledge and, in doing so, become producers, not just receivers.

Web Communications 101 (WEB101)

A major component of the assessment in this unit is a ‘web presence’. More than a website and blog, a web presence interlinks a central node with linked  services and nodes to expand the digital footprint of a user and established their online identity. The negotiation and communication of identity is central to this unit: it’s not just ‘how to blog’.

A very small number of examples of these web presences are:

• Michelle’s Curtin Uni Learning Portfolio
• Caja Design – Web 101 – Blog
• Smartworks Education
• Outback Gold – A Website For Those With Gold Fever
• Reflection (and Stuff )

Over 400 students have taken the unit: sorry, can’t show them all. In particular, look at how some students have made their web presence almost entirely ‘real’, with bare hints of what it connects to (their study); others have not. Some students, as evidenced by these presences, are now using them as part of other units of study too.

Note that students happily created their own informal, computer-mediated network spaces such as Web101 – Curtin University | Facebook; and staff teaching also use the web as it was intended – free and rapid information exchange – to support this unit:  Web101 Assignments FAQ.

A big part of the unit also involves the use of twitter: see the most recent  Twitter search; delicious is also used.

Please look at “I Tweet Therefore I am?” by Dr Tama Leaver, chief architect of the WEB101 learning experience.

———————————-

As I have argued elsewhere: the authenticity of these assessments is not a simple ‘flip’ from artificial academic work into ‘real’ web work. They are a negotiation and a compromise in which equally valid requirements from both knowledge networking and education are brought into a creative and productive tension. In the next instalment, I will provide some examples of what happens for students in the followup unit to WEB101.

On Thursday 2 December, I am presenting at the National Curriculum Innovation and Quality Forum on the subject, “Risks and opportunities in authentic learning via the Internet”.

The basic brief for this keynote presentation is to:

• summarise approaches to authentic learning in the BA (Internet Communications) at Curtin University;
• identify the key benefits in using a public knowledge networking approach to authentic learning; and
• highlight risks and strategies for managing those approaches in the pursuit of authentic learning online.

While I hope to do that, with a particular emphasis on giving some examples from the great work that students in the BA (Internet Communications) have done, I also have found that in preparing my talk I have had to develop a more coherent argument about the nature of authenticity in learning and the relationship between education and learning.

The talk can be found here: https://netcrit.net/content/nciqf2010.pdf

This paper draws also on some specific work I have done on the authentic assessment in our online conference unit, Internet Communities and Social Networks 204 and more generally on social media and authentic assessment (presentation in the UK, May 2010)

Some of the examples I refer to will be listed on my blog within the week.

I recently answered an email from a colleague asking for advice about wikis, especially in the face of his university’s (inevitable) suspicion about anything that is not authorised, locked-down, served from the campus and generally (IMO) unusable for agile teaching and learning. I thought I would share an edited version of my views, since it neatly captures some of what I’ve been thinking about as part of my ALTC project on Web 2.0 and online learning.

Agile teaching: responding to needs and concerns in the learning design of students’ experiences, activities and tasks which takes account of current events, new technologies in ways that institutionalised curriculum design and enterprise technology practices can’t cope with because they are too structured, clumsy and slow-moving. Agile teaching implies agility of mind as well as design and technology – it’s being playful, picking up and putting down, making limited and short-term commitments to particular ways of teaching and content, on the basis that it’s more fun, more engaging and ultimately more realistic as an educator and thinker to be moving forward, not circling the bureaucratic wagons

 

---

Yes, we use wikis in our teaching, in two ways. First, some of the students naturally set up wikis to work on collaborative projects or similar without us telling them to – we leave that up to them! They also use other technologies, such as Ning and similar networking sites/services and, of course, simpler forms of communication and collaboration. Wikis perhaps are suited only to some kinds of people for this task?

Second, we run a unit of study in which students are required to author their first assignment in a wiki – but not collaboratively (they look at others, but don’t edit). Then, their second main assignment – a group report – must be online and while we provided an alternative publishing space (openzine.com) they didn’t like it and so all, I think, ended up using a wiki. This year (semester 2) we will require them to use a wiki. Elaine Tay and I have written a paper about this unit and how we use wikis – currently about ready for submission for publication. Briefly, we concluded, it’s not the wiki itself, but the social affordances of groupwork that are the most important learning design consideration: wikis are too open a technology to really constrain or require a form of collaboration unless you create the social context for it. Note that an unexpected outcome was that students benefited from seeing each other’s individual work and, as far as we can tell, there was no great worry about it. Some ideas about this were presented to the Teaching Learning Forum in Perth this year.

I should add – we also used to use a wiki as the place where students doing a whole-of-group collaborative project on virtual communities would create and publish their material. We used a wiki like this in the mid-2000s and it has been a great success though, recently, as numbers have grown (that unit now has 100+ students online), it has not worked as well and we’ve changed our thinking (see below re the online conference). What we loved about this wiki environment is the way it challenged the students to imagine the form and structure of the collaborative presentation, not just the content. It enabled task division – some people did more design work, others more content work. It also promoted a sense of student ownership – what made this task authentic was the students’ own taking of responsibility in the snowfield of the blank wiki!

So, in short, you can and in some cases should use wikis in teaching. I think the most important affordances of wikis are as follows.

• The fact that the process of creating and editing wiki pages is relatively simple, and yet produces a shared resource, makes this software a very powerful tool for managing knowledge work within a group whose abilities and knowledge of the content of the site varies as much as their technical skill.
• Wikis permit (and even promote) collaborative individualism. Traditional cooperative publishing activities tend to require a lot of discussion of what to change and how to do it, before you actually make changes, create content etc. Wikis allow individuals to jump in and work relatively safely and the collaboration – the forming of the group happens in the process of editing and developing the content.
• Plus, at a very simple level, a wiki rapidly allows us to get material online, shared, reading and writing and thinking about audiences – whether all the other students or (preferably) a real audience of web users

If you look at http://www.wikimatrix.org/ you will see a listing of 100s of wiki engines and sites. Some would need to be installed and run from a server (think mediawiki for example), others are hosted (like pbwiki). I’ve not looked at a lot of them in detail recently but have been using http://wikidot.com – this is free, but with ads and some restrictions. It’s cheap to buy a license however. Personally I like it, but my colleagues think I am nuts! They tend to go with pbwiki, now available via http://pbworks.com/ but the cost is an issue. Just had a quick play with wikimatrix and I suspect you will find better free solutions. The key reason to pay money is to get finegrain control over access / publicness etc and/or lose the adverts. I doubt size is an issue that would make you pay money.

A recent interesting development which might work for a small group (since I am unsure how stable it is) would be http://www.springnote.com. The beauty of springnote is that it wraps up a wiki approach in a metaphor (the notebook) and some visual clues. It’s primarily aimed at private (eg not publicly accessible) work by individuals OR groups Or both in the same space. But it can be exported to a public site. I’d caution against using it with more than 10-20 students at first, and you should download / backup regularly. I discussed Springnote recently at the University of NSW:

Part of the problem with wikis is that they are highly unstructured and can be challenging to manage in terms of access / revision etc. It was a 3-week online conference, students submitting 2000 word papers in 1 of 4 streams (after major assistance with improving the papers). Students then read papers, commented and replied to comments on their own papers. We used WordPress (a blogging engine but which actually is more like a content management system these days) installed on a server I pay for in the US. It worked absolutely fine – 85-90% of students have said they learned more this way than from normal study mode. Most students were external, but not all. The challenge, though, is to design the learning experience – this approach worked because it was a social event, culturally encoded and built into the assessment, teaching and feedback structure of the unit.

So, in short: wikis work. Choosing a wiki is a personal matter and thank goodness for wikimatrix! Making it work, however, remains an exercise in teaching – that often-forgotten aspect of ‘learning technologies’, ‘the student experience’ and so on. Teaching with a wiki involves careful assessment design, strong encouragement and endorsement of its use and, in the end, a realistic and believable purpose for it. Otherwise students will simply dismiss it as ‘technocrap’.

Following my presentations on the ‘top 10′ web 2.0 applications for learning which exist in the real world of the Internet, I have started to develop a model of knowledge networking which allows us to understand how the contemporary web (a better description now that Web 2.0?) serves to enable knowledge networks, and how those networks might exist within the complex digital ecology of the Internet. These ideas are still under development.

In 2009, I argued that learning was a special case of knowledge work and, that as all knowledge work becomes, or tends towards, being knowledge networking, so too learning changes in its character (regardless of whether students and teachers actively engage with such a change). In doing so, I claimed that knowledge work is best understood in quite simple terms (at least initially), as involving the classic input – process – output model which has dominated information sciences and systems research for many years. I don’t doubt that knowledge is far more complex than this model at first suggests (see Allen and Long, 2009). But, the complexity cannot be ‘explained’ by making the model appear more sophisticated, for the complexity comes from the social conditions within which knowledge work occurs. Therefore, superficially, we might as well continue to think in terms of knowledge work being done within the input-process-output circuit, and just remember that every output is an input (and vice versa) and processing is also continuous, influencing how information even comes to be considered as inputs and outputs.

To gain more understanding, however, and to emphasise the way that the Internet promotes distributed and collaborative knowledge work (why it is networking, not just working), I want to now model knowledge networking in the contemporary Internet in slightly different terms, though the relationship to input-process-output should be obvious. I identify four crucial elements which, collectively and interactively, generate the system of knowledge work conducted through and for the web; in doing so, I hope to provide a better way of thinking about the purpose and possible application to elearning of the entire system.

First, we can identify online behaviours and web services that work as information pumps: these pumps draw on the apparently infiinte (though actually limited) reservoirs of information within and around the web and then circulate the information with various degrees of filtering, flavouring, and transformation through many different channels and pipes. Note that information pumping involves both humans and computers. Some examples of the web services that might naturally appear as information pumps can be found at Newsmap, or at Evri; but do not think that it is the ‘sites’ alone that are the pumps. Delicious also serves as an information pump, distributing the work between its users and the systems maintaining the lists of tagged links. Pumps can either be sites that are visited (such as the innovative new instance of Cuil, Cpedia ; or they can be feeds (primarily RSS) from sites, gathered and analysed in various ways and presented to users (see RSS Voyage for an elegant version; and many different variations thereof.

Second, we can observe that many web services entice and require uses to engage in the manipulation, creation, re-expression or other forms of cognition using the service as a partner in these activities. Thus, the second element of knowledge networking online is the existence and use of cognition engines: these engines – fuelled in part by the information pumps – work with users to ‘do’ the knowledge work. There is an incredible variety of cognition engines, from complex and highly structured (Cohere for example), to deceptively open and simple (wikis would serve as these kinds of engines – a current personal favourite is Springnote, with its elegant notebook metaphor). Engines can involve innovative creative activity (making simple movies from text at xtranormal; the delights and frustrations of Prezi) or some traditional cognitive forms – the visualisations that can be made at Manyeyes for example, or mindmapping as at Bubbl.us. Cognition engines can promote reflection, too – like a personal current favourite Betterme

Cognition engines often contain significant affordances for collaboration: yet we can also see that many web services are specifically designed for the kinds of collaborative endeavour, which generates the third key element in the knowledge networking system: social environments. Ranging from loose social networking utilities (Ning), to detailed groupwork systems (Wiggio), but also including real-time interaction channels, such as the simple Tinychat conference room system or more sophisticated systems like Elluminate, these environments establish an array of spaces, mediated by technologies, within which people can act socially in knowledge work.

The fourth and final element discernable within the contemporary web as a knowledge work system is the publication outlet.

In conclusion, then, we can say that the knowledge networking dynamics of the web involve distributed, conjoint action by humans and computers through web services which serve as information pumps, cognition engines, social environments and publication outlets. But, to be clear, it is not the case that each web site we encounter online serves for just one of these dynamic elements. Indeed, most web services include a combination of features which means they serve as all four elements at once, whether closely coupled (as for example in facebook), or more loosely. That said, individual users, as they form and participate in networks of knowledge, traverse several sites, use many services, to carve out from the available opportunities their own particular kinds of knowledge networks. Let me finish by providing two examples: one that is contained largely within a single service; and one that spans several.

Diigo, a bookmarking, collaborative research and web annotation service, is a clear example of how one single website can host services and permit user behaviours that constitute an entire knowledge network. Diigo pumps information (both from the web and within its own system), with a significant degree of filtering and enrichment by users; in the work of organising, analysing and reframing information, it serves as a cognitive engine; yet, since many people are engaged in that task – often in well-defined and purposive groups – it is a social environment as well; finally, the results of this knowledge work can become public, so Diigo serves also as an outlet, with an audience, for publication.

Yet, knowledge work systems can span several services and sites as well. RSS feeds found and managed through Feedmil can push information into a cognition engine involving Listphile: the cognitive work here is to array and manage individual items within a pretermined list form; and, while listphile is itself a social environment, the collaborators using it choose Wiggio as the locus of many of their active collaborative endeavours. Finally, the list – while available in Listphile for public consumption – is pushed to the world as embedded code within a blogging-based website linked to Twitter, using Tumblr. Note that, in this example, the Tumblr site is itself also serving as a cognition engine at times; the listphile service involves collaboration, social action and a degree of publication, but that the specific knowledge network formed emphasises specific uses for these services within the model I am outlining.

Ultimately, the contemporary web demonstrates the fluidity and agility of the so-called Web 2.0 approach – data and human endeavour is no longer necessarily concentrated at specific places and times, in forms that are unique or limited in their re-usability. Within such a web, many forms and examples of knowledge networking, using countless varieties of applications, will occur. But, in general terms, I would argue that all knowledge networking involves the collective activation of the four distinct elements I outline – information pumps, cognition engines, social environments, and publication outlets. Higher education must learn to imagine and build its own knowledge networks that draw on this model, and on the many excellent services for knowledge work available on the web.

Assessing in the disciplines: focus is on self and peer assessment
Three papers, across three disciplines (nursing, media, education), each providing an example of how these assessment forms are working.

Using peer and self-assessment with academic moderation… (Warland)
This approach was based on literature that asserts peer and self assessment improves quality of learning; generates reflection on learning; gives increased confidence and independence and responsibility. Literature also provides some negatives – lack of comfort and confidence from students in judging; worries about doing it accurately and correctly for grading. Thus Warland only used the peer assessment for formative (not grading) assessment. The context is a real-world ward setting for student nurses to learn practical skills (time management, people skills etc); two-day workshop, with simulation day (playing roles of nurse, patient, relatives, doctors, etc) and activities on second day, more independent, less closely monitored by the teacher (increasing chance of peer feedback). Use of assessment rubric – students judged themselves and a peer – also provided open-ended comment. Rubric based on nursing profession / university graduate attributes (scale 1-10). Surey of students about their experience indicated high levels of agreement with the value of the approach and the effectiveness of the way it was conducted. Little worry reported about the process. Concludes with useful comment re ‘imperfect’ feedback – rapid, and located within the students own world; quicker and closer to the students and their activity. From academic perspective, the rubrics speeded up the marking – teacher used the rubrics to guide her assessment and do it more quickly (essentially moderating the feedback and giving a grade on it). Notably, by taking the pressure off ‘the grade’, students happy to give feedback – what perhaps this conclusions implies is that lecturers may themselves do better in feedback if not grading.

Transitioning media students for self and peer directed assessment (Wilson)
A key idea: the need for programs (courses) to develop consistent narratives which sustain approaches to student assessment and learning. The idea of a program narrative is vital, but also should be understood as an iterative process based on changes from year to year. There need to be consistent transitional processes, moving students from a presumed lack of knowledge about self assessment through to a more mature, adult and independent mode of assessment (based on Kift’s work on the first-year experience).

Context is the media program, and the assessment change was part of a broader review of what the outcomes should be, the focus of the curriculum and so on. Clear that the program renewal involved debates and arguments among staff about the extent of self and peer assessment: for example, the initial expectation was that by 3rd year peer assessment would be at 75%. This goal proved hard to achieve. Claims that CEQ and other surveys prove the benefit of the approach; recognition from ALTC.

Heart of the change and the commitment to peer assessment (in the face of opposition from staff and students) is the shared vision of what the professional media workers needs to be. To sustain the commitment to peer assessment, also, cites the work of Perry (in Nilson, 2003) about the way students and people move from a position of imagining there are right and wrong answers, judged by authorities, through various stages of relativism, to an end point of commitment – of making judgments of what is right, according to self perception of broader contexts. One of the critical links between this assessment approach and the graduate attributes is the idea that students are forming a community of peers and that peer judgments of their self assessment practices are essential. The students learn the ethics, practicalities and similar process issues involved in assessing oneself. Concludes by emphasising the consistency of the narrative across the program, and the need for legitimate rationales that are sensible in terms of graduate outcomes.

Using self and peer assessment to enhance students’ future learning in higher education (Thomas et al.)
Context: it is part of an ALTC distributed leadership project around assessment; nice quote “some assessment even seems to encourage learning”, but as Thomas asks “does our assessment actually encourage learning that is important in the future (post-graduation)?” To achieve this, assessment needs to require students to judge their own learning: “part of becoming an accomplished and effective professional”. Notes the way these approaches involve academic socialisation.

One critical issue: teacher-driven self assessment might socialise students into being ‘like the teacher’; in terms of looking beyond graduation, perhaps the way students will assess needs to be not at all like the teacher might do. [Yet this raises the inevitable conflict between education for practice, and education for certification].

Notes another barrier to acceptance by students – can be seen as laziness by lecturer (Cassidy, 2007 cited); need to actively promote a shift in responsibility and yet also power. [ I would, on reflection, add that making the powerless (students) responsible makes the situation worse; empowering students such that they expect the responsibility leads to success]. As with other speakers, resistance from other staff was encountered, especially around the question of reliability: [what generates this obsession with reliability? Is it fear of negative comments from outside, or a desire to remain empowered to judge the worth of others and thus sustain one’s own self-construct as important?]

A key point: while electronic peer assessment tools appear to offer advantages, they all come with in-built assumptions about the process which, for Thomas, did not sit well; he therefore used manual processes but judged that the time taken was no more than other methods.

Evidence from the research into effectiveness of process: yes, many students enjoy it or accept it; others though point to a major hurdle – the lack of quality in peers’ work – if they read work which indicates a peer is not good at the subject, they lack the confidence in their judgment.

Must clarify the intentions and purposes: students do NOT understand why we do innovative and different approaches and need to assisted to learn;

Authentic Assessment of Authentic Tasks
ATN Assessment Conference Keynote; Jan Herrington

Opens with the maxim “We assess what we value and we value what we assess”. Uses it to show how assessing time online, numbers of posts to forums, doing MCQs values lower-order knowledge repetition, the time spent online, and quantity of participation.

Cites Angelo “educative assessment tasks” – that should be the focus of our attention. Anything which is ‘to do’ – the task – that matters most. Tasks and assessment are inseparable.

Reprises the classic ‘from this to that’ movement for online learning – eg from instructivist to constructivist; individual to collaborative. Suggests that there are now further moves towards connectivism, Web 2.0, and so on.

Herrington outlines 9 elements of authentic learning:

1. Authentic Context, reflecting the way something will be done in the real world; embracing the complexity of real world; provides purpose and motivation of learning;
2. Authentic tasks, which have real-world relevance; may take the whole semester; complex and ill-defined (because time has to be taken to learn what the problem is);
3. Expert performance, which Herrington linked to Web 2.0 – the crowd has the expertise; the expert knows more than you and can mentor – not necessarily an acknowledged expert; Herrington justifies the lecture when it is expert performance, not because it is transmission;
4. Multiple perspectives
5. Collaboration, including joint problem-solving and social support;
6. Articulation, where online learning can be silent, it becomes authentic as a learning environment where people speak to their learning, present publicly and defend their positions;
7. reflection, that gives opportunity to consider post-facto the choices made by students (reflection in action and on action differ);
8. scaffolding and coaching role, which to be authentic can include others;
9. authentic assessment, which through the Internet, provides public audience and thus motivates people to work harder at preparing for that greater exposure – the product at the end of the assessment has to be authentic beyond being just something for assessment.

Herrington gives an example of a virtual environment involving interviews, artefacts which mimic the real world, a screen design that cues people to the ‘reality’ in a manner not unlike a game. Emphasises that authenticity is not linked necessarily to high-resource, intensively produced environments.

More on authentic assessment: list of either ors which does beg the question of whether authentic is serving as a euphemism for ‘better’.

1. Context factors: fidelity / transferability to world beyond classroom

2. Student factors: production of knowledge; problem solving; collaboration with others; conversation; performance of knowledge
3. Task factors: ill-structured challenges; wide range of responses; assessment integrated with the activity

Some problems with authentic assessment:

• minimum number of assessments challenges authenticity
• restrictions on group work or amount of grades for group work
• invigilation requirements

Herrington emphasises the alignment of assessment to the task – e.g. don’t use exam to assess authentic task. Instead, assess work from the perspective of the ‘assessor’ in the real world – role play the role from the real-world context. Also emphasises the need for scenarios to be consistent with what students expect and understand, to prevent the cognitive challenge being to ‘accept’ the authenticity. Cites Savery and Duffy: problem must be real – but challenges it to argue that students can manage with moderately real, or believably real.

Herrington concludes with a substantial example (Virtual Records) which dates from some year ago but shows many of the key elements – combining relatively artificial representations of reality, but with astute – quite dramatic / produced – cues which turn them into reality for students once inside the system. A key point which emerges to guide us: Alesio’s (1998) concept of cognitive realism.

A critical commentary
It is clear from Herrington’s presentation that the term authenticity is growing in its reach to serve as a synonym for multi-dimensional, high-quality learning experiences. Within her 9 elements, it is true that all are necessary and that, indeed, all can be linked to the world outside of the artificial confines of ‘study’. Yet, to some extent, this extension lowers the precision of application of authenticity by essentially saying all study is just like not-study; in fact it is not…authenticity at moments, even quite extended moments, is vital – but the artificiality is also important.

At times the assumptions – that authenticity involves, say, collaboration – really expose the partial definition of authenticity. What, for example, if the task is (in the real world) individual and not collaborative? Would it not be authentic to limit collaboration? Developing this point to consider Web 2.0, one of the challenges for the emphasis on collaboration is that, to be honest, Web 2.0 is also about networked individualism and thus, to be authentic, different forms of collaboration would be required.

One of the examples Herrington gives is a ‘virtual environment’ which is authentic in her judgment. The primary markers of authenticity here are visual display (imagines of labs in this case), interactive navigability (as if they were inside the images) and so on. While clearly different to ‘read this about…’, its authenticity might also be seen as quite limited; that said, what makes it authentic is the assessment task which is very much ‘real world’. Thus authenticity is a complex – made up of more and less real components whose totality allows students to believe themselves to be ‘in the real world’ even when they are not. Indeed, belief, not actuality, matters – else the mistakes they migh make would not be possible and it is through mistakes that we learn,

Herrington’s examples – for example from educational research units / courses – show that some kinds of content / learning outcomes do suit better the authenticity approach, either because of the cohort or similar factors, or because making a viable realistic scenario or similar works easily; perhaps, because there is alignment between what the academics ‘know how to do’ (eg research methods) and what is being taught. Therefore they can model their practice. In some other areas, perhaps, the learning concerns broader and more shallow material – not ‘inhabited’ by the teacher. Of course, authenticity can still be generated, but perhaps takes more work, or perhaps requires a focus on that particular element of being at university which makes it most especially ‘artificial’ and avoid that aspect, so as to reduce the ‘unreality’ overload.