Tuesday 14 February 2012

Reviewing ReView

I had a chance to have a look at the workings of ReView while I was at UNSW. It is an impressive feedback tool with some affordances that aren’t yet available in competing tools but also some limitations which may be deal breakers for most institutions.
Pros
1 Mapping
ReView is really well suited to the Australian Tertiary Education sector which has made significant investments in the identification of Graduate Attributes (GAs). Institutions are now being (or will soon be) required to map learning outcomes and student achievement against those GAs. ReView’s key design feature is its ability to do this as a natural part of the marking process. Academics identify assessment criteria and link them directly to GAs so when they come to generate the feedback, these automatically map onto them. This constitutes a huge time saving in what could otherwise have become a very onerous process. In the HE sector in the UK we are not yet required to do this – but I don’t think it is far off. Designing this kind of mapping in where we can to pre-empt such a requirement and finding the right tools to help us do it may be worth considering.
2 Transparency
As tutors mark using ReView, they generate a grade based on student achievement against defined Assessment Criteria. This is, I believe, a great way of improving the transparency of marking for students (making it clearer to them how their final grade was arrived at). While I acknowledge the critical scholarship on the use of scored or calculated rubrics and assessment criteria in this way (particularly Royce Sadler’s work) I feel that the benefits it affords students outweigh the potential or actual drawbacks in terms of integrity. The tutors using this tool determine student attainment using ‘sliders’ which I do take issue with (below) but which work in much the same way as the rubric calculator in Grademark.
3 Analytics
Spitting out the back of ReView is a really interesting ‘dashboard’ which shows rich and valuable data on the student achievement harvested from the marking. I didn’t get to see this in action because (as is always the case with these tools) unless there is live student activity in it, it’s difficult to ‘mock up’ demos of things like this. But I saw enough to get the gist of it and it looks more advanced than simply the raw data which is generated by Grademark.
4 Mobility
This tool is designed to work on mobile devices – particularly tablets and especially iPads. This makes them portable. It means that this is a tool which is very well suited to the marking of studio-based work (such as design, textiles, fine art) and it’s also fantastic for marking hand written exams because it doesn’t need assessment to be submitted to it in order for it to be marked and returned.
5 Self-evaluation
Unlike Grademark, this tool includes a student self-evaluation tool. Students can indicate what they think their work deserves. To achieve the same thing in Grademark requires a workaround and lots of data entry. The student self-evaluation is clear to the tutor as they mark and this may influence their judgement in unhelpful ways. I feel that if there is going to be a student self-evaluation function, it must be ‘blind’ to the tutors as they mark.
Cons
1 Integration
Currently ReView is not well integrated or integrable in that it is a stand alone tool (it’s not yet a building block for any of the major VLEs) and in that it is only a feedback tool not a marking tool. In other words – students can’t submit their work to it and tutors can’t comment directly on their work with it. The danger of this is that it will generate false economy. So even if it saves tutors time in the marking of student work, it may cost them or their institutions more time in terms of mark entry, handling  submissions, returning student work etc. Tutors may find themselves moving between two or even three different systems to received, read, annotate, plagiarism check, return and enter the marks for a piece of student  work. Additionally, the transparency it achieves through the rubric ‘sliders’ may be counteracted by the lack of clarity as to precisely where the strengths and problems in the work are located if they can’t be marked on the work itself. For instance – a comment saying that some sentences are poorly constructed is useless to students unless they are clear which ones are poor and which ones aren’t. The integration within VLEs will no doubt come with time, but it’s looking unlikely that the marking tool is going to emerge. As such, while it does some lovely analytics, its not the ‘granular’ level that GradeMark achieves.
2 Clarity
I have concerns about the ‘sliders’ themselves. If we are using rubrics and assessment criteria to improve transparency, we need to take great care not to then obfuscate what we are doing. The ‘sliders’ allow tutors to decide whether a piece of work is in the high or low range within a classification (i.e. that against a single criteria it can be a ‘high 2.1' or a ‘low 2.1'). This to me is one step forward and two steps back. When you have five or more criteria (averaging  20% or less per criteria) the difference between one classification and another is going to be  2% or less of the total. To be making judgements within that classification (within 2%)  is marking to a level of accuracy which is simply not reliable or helpful to the students. It is for this reason that I think the ‘radio button’ approach of the Grademark rubric calculator is more transparent. In other words, it doesn’t leave them wondering what makes their achievement a ‘high’ rather than ‘low’ 2.1 (for instance) against a particular criteria. It does allow tutors to tweak a final grade away from a borderline (eg 69%) but my hunch is that when rubric calculators are used, students are less inclined to complain about a mark that ‘comes out in the wash’ to that number than one which is arrived at holistically by the tutor. As a result – I don’t think we should shy away from awarding borderline marks if that’s what the rubric (which has been clearly communicated to the student before hand) calculates. Anything else is duplicitous.
3 Cost
This tool looks like it’s going to be quite expensive in comparison to its competitors. Given that this is likely to be a tool that would need to be used in conjunction with other marking and submission tools and that we can probably achieve many of the affordances it offers with some workarounds within Grademark, it’s going to prove a hard sell to many cash-strapped institutions at the moment.
Final Evaluation
This looks like a fine tool that will almost certainly be the right tool for many marking jobs. I think it will be especially attractive to colleagues marking physical objects (like artworks, models etc) and performances (music, drama, presentations etc). I suspect many will find it useful for marking exams, especially if feedback is required on them. It won’t replace marking tools like Grademark and it may be hard to justify the investment if we can find workarounds which achieve similar things within the tools for which we already hold sight licenses.

10 comments:

  1. How had you been capable to go to this summary? Your own dissertation-writing-help.org post echoes which you does sufficient research nevertheless it might not harm to add more modern developments inside the area. That will have made this page far more thought-provoking.

    ReplyDelete
  2. Any moderately examine schedule disolve in respect to twelve months, permits you tremendously cut back your current payment monthly. It's getable to makes minimum amount regular monthly costs, or maybe you may be able to ton any entire from your minimum amount examine once you in addition as. anytime somebody ton else, somebody decrease what size this kind of home mortgage furthermore avoid having captivation sale price. the particular acknowledge disposition cash advance elgin product is likewise quick captivation disposition product. In your case, because of this you may be able to settle your house mortgage at first furthermore save captivation around the home mortgage.

    ReplyDelete
  3. As to the speed of interest of automotive title loans, rates ar supported native, state and federal laws. Local, State and Federal governments have the authority to manage these rates additionally. because of this, it's a necessity to note that automotive title disposition firms ar in strict adherence to those laws and rules. they are well-aware of these circumstances. It is, therefore, necessary for people to make your mind payday loans corona up on the foremost reputable disposition firm which could offer them the best and sincere facilitate.

    ReplyDelete
  4. A financial gain money loan may be a money loan, sometimes to an payday loanorganization, supposed to fulfill daily cash wants throughout times once financial gain is inconsistent. These loans area unit short in nature; borrowers sometimes should repay them in thirty to a hundred and eighty days.

    ReplyDelete
  5. I value the post.Really looking forward to read more. Much obliged.
    The Solluna Resorts
    Ashoka's Tiger Trail Resort

    ReplyDelete
  6. Arizona has become the best contempo accompaniment to booty allegation adjoin the payday accommodation industry. With anniversary allotment ante on loans extensive over 400%, abounding acquainted the allegation to footfall in. The accuracy is, the bulk makes it assisting to the lender and affordable to the consumer.
    Payday Loans Chicago

    ReplyDelete
  7. At one time I had 7 loans out at altered payday accommodation companies advance out amid Idaho and Washington. Well over bisected my paycheck would go to pay off these loans. I apparently had over $3,000 attributable at one time with an absorption amount of 400%! They didn't alarm anniversary added and so it appeared like I was alone alive with one company.
    cash advance corona

    ReplyDelete
  8. Spitting out the rear of Review could be a very attention-grabbing ‘dashboard’ that shows wealthy and valuable knowledge on the scholar accomplishment harvested from the marking. I didn’t get to visualize this in action as a result of as is usually the case with these tools unless there's live student activity in it, it’s troublesome to ‘mock up’ demos of things like this capstone project proposal.

    ReplyDelete
  9. Academics determine assessment criteria and link them on to GAs therefore after they return to get the feedback, these mechanically map onto them. This constitutes a large time saving in what may otherwise became a awfully burdensome method assignment service.

    ReplyDelete