Thursday, July 26, 2007

George Siemens discusses peer review

George Siemens has a post on his Connectivism Blog about peer review. So much of it echoes my own views and experience. He starts off his post by citing a situation in which a Masters' student in Lisbon related to him the skepticism of her assessors of the theory of connectivism and of Knowing Knowledge because of the lack of peer review. Expect for the location, that student might well have been me. As I stated in my Ardent Student blog in February:

"I struggled to get approval to submit a paper on Connectivism recently, because there is little or no peer reviewed writing on it in any of the recognised journals. I’m obviously not built for this world, because my immediate response is “So what?” I don’t understand why someone else needs to have written about something before I can write about it!"
It is worth noting, however, that I finally did gain approval for the assignment and I was delighted by the feedback that my lecturer's curiosity had been aroused by my submission and that she was set to explore the matter further out of personal interest. Score one for the rebels!

As a Masters' degree student, I find that there is this requirement always to reference peer reviewed work in my assignments. One of my profs tells me that it can take two years from the time an article is accepted by a publication to the time it appears in print. Two years?! By that time, things will have moved on, the thing you're writing about will be passe and something else will be flavour of the month. As a consequence, I find myself making tenuous links to material that is peer reviewed. So, if I'm writing about connectivism, I will dig out peer reviewed material about behaviourism, constructivism and cognitivism. I might even include a reference to the gestalt theory, which shares with connectivism an emphasis on pattern recognition. But I still feel as if I'm window dressing.

It's bad enough for the shorter 3000 word assignments, but it is really going to be a problem for my dissertation, which must be 20000 words long and must include a hefty amount of literature review. The project will dominate my life for a period, so it must be something of consuming interest to me, but as I noted, "How can I possibly write about something that matters to me if I’m restricted to a topic with an adequacy of been-there-done-that from recognised luminaries?" Ron Lubensky commented that he was experiencing the same frustrations.

Like George, I kick against the air of elitism that seems to go with the concept of peer review in academia. As he says: "Peer review is based on the assumption that a handful of experts are capable of validating ideas." There is a clear line between professional journals and peer reviewed journals, and it seems to me that the attitude of academia towards the former is one of faint distaste, in much the same way that broadsheet readers feel about tabloids. I can't help wondering whether the term hasn't been hijacked, like so many other learning terms, to serve one agenda. George touches on this in relation to the ongoing fragmented online conversation about connectivism. I tend to agree with him, after all - should "peer reviewed" not mean reviewed by one's peers? I would imagine that a few peers, handpicked by a publication as being tuned in to that publication's ethos, are less objective and therefore less reliable than a plethora of self-appointed peers who happen upon the material in the pursuit of their own informal , lifelong, lifewide learning journey. Coincidentally, in a recent paper, I touched on this point in respect of George's book Knowing Knowledge, saying:
"On the flip side, as mentioned before, not much of the material I read is peer reviewed in the academic sense of the phrase – most of it has, however, passed the acid test of the online community which is ever willing to make its criticisms known.

An excellent example of this is the book Knowing Knowledge by George Siemens..."
There is so much more I could say on this subject, but most of it would simply be a repeat of what George has said in his post. I will say that it's nice to find that I am not alone in my wilderness, but this doesn't change the fact that I am going to have to toe the line, at least until October next year if I evere want to write the letters MA after my name. And, shallow as it may seem to confess it, after all the work I'm putting in, I am determined that I will come away with at least that much to show for it!


rlubensky said...

Hi Karyn,

Yes I'm in Masters Degree dissertation writing mode. The issue of standing on the shoulders of respected academics begs the question of who is doing the respecting. In my case, I am a degree candidate in an educational faculty, but my work about participatory methods reaches into the political and communications literature. There is material written about learning outside of the educational academic sphere, including well-cited research handbooks, but I take a certain risk in basing my work on it because it is unrecognised by the school of thought to which my candidature is subscribed. The attitude of my supervisor toward some of that material has been dismissive.

Further, my supervisor is the Editor of a highly regarded, long-running traditional educational research journal. He described the rise of open access, online journals as a "proliferation". I don't think any are on the graduate faculty's list of recommended journals.

On a different slant, I've been a peer reviewer for an international academic educational conference, and it was an honour. I can put this on my CV. Federal funding into research programs increasingly depends on a demonstration of impact and influence (esp. UK & Australia) by researchers, so being able to list peer review and publication into peer-reviewed journals is actually becoming more crucial.

George Siemens said...

Hi Karyn - appreciate your thoughts here. While I highly esteem the value of peer review and subjecting new ideas to rigorous analysis, I share your "so what" attitude toward ideas I encounter that may not have expert peer review status.

A part of me thinks that we are at a point where the notion of peer review is enlarged to include "the many" (a task I'm experimenting with our Journal of Emerging Learning Technologies (tentative name) which should be available within the next few months). Expert peer review is important...but I think having true peer review enlarges the depth and quality of review (rather than reducing the quality of idea/knowledge validation).

...and, I can completely see your point of toeing the line - if a system is laid out a certain way, we can lament its failings, but in most cases, we still have to honor its structure. We only need to martyr ourselves a few times in our lives...and I imagine gaining the MA initials does not quite qualify :).