Thursday, 8 December 2016

Assessment Philosophy: NCEA vs University

I remember back in year eleven, not long after I'd had my first NCEA external exam (or perhaps the first few) and we were at a petrol station we don't normally go to (so this was likely in Pukekohe)  I was talking to my mother about the exams. I'm not exactly sure what I said but I know I commented on the plastic bags. Indeed, looking back, NCEA runs some extremely professional exams. Nasty quibbles about subconscious bias in marking are avoided (because your name isn't on the booklet, just your NSN and some other numbers I don't know the meaning of) and because people have set seats, it is not really possible to angle yourself so maybe, if you squinted, you could read someone else's paper. And when there are resource booklets (e.g. unfamiliar texts, resource interpretation) they are proper booklets that don't fold weirdly. None of these things are true of university exams. Which is the point. Now that everyone is finished their exams, I think I'll compare and contrast how assessment works at university and in NCEA (sorry IB and CIE people, but I think university's philosophy is the same as CIE's anyway).

The first thing to note about uni and NCEA is that NCEA is an assessment system. That is, while NCEA appears to have a syllabus or a curriculum that is studied, it is not a whole programme... it just so happens to be used for one particular curriculum. There is no reason why NZQA couldn't take a curriculum from somewhere else in the world and adjust NCEA to fit that curriculum, and then sell/lease the modified version of NCEA to that education system. This isn't necessarily apparent when you're an NCEA candidate in NZ because the same sorts of thing happen in history and calculus and they're all branded as NCEA assessments. This nature is then further clouded by the extensive online resources that NZQA has on its website (more on this later). The resultant coherency between different subjects is not replicated at university except insofar as they all share the same percentage based philosophy.

I'm not sure how it is at other universities, but I imagine pretty much all of them work in the same way as Auckland. That is, assessment is pretty much the responsibility of the department and/or faculty. In fact, lecturers could well have a lot of control over the means (e.g. to have tests or not, how many etc.) of assessment in addition to being the people in charge of writing all the assessments (and, I believe, the learning outcomes). In some cases this means that courses will be very personal (and, in fact, some history courses at Auckland are departmental and others 'belong' to the lecturers) while others (especially large stage one papers) are team-efforts. In other words, I can't tell you what kinds of assessment bundles you'll face because I've done courses which have online quizzes + assignments + test + exam, courses which consist of an essay + a different kind of essay + third kind of essay and courses that are literally test + exam or just the exam... and I know for a fact that things called 'lab reports' exist but I've never done one myself. But in all strands of these bundles that one philosophy is still there. Percentages are, in fact, the only definitely consistent thing about university assessment. And it's not something compatible with NCEA's philosophy.

Even with Grade Score Marking (GSM), or perhaps because of it, I think most ex-NCEA students will find the transition to uni a bit of a conceptual shift. As you've probably guessed from the description of the above bundles, the workload isn't all that different. You know how all your internals would be due at once towards the end of terms two and three? Well, that still happens, but the meaning of both the assessment and how well you did on it changes between NCEA and uni, even if completing assessments feels pretty much the same. Thinking about it now, it may help to imagine that each course in uni is a different standard in your major. So, if you did history at school, there were the three internals (research, writing, perspectives) and the three externals (resource interpretation, causes and consequences, the other one/trend). And from this you might have E, E, E, M, N and M or whatever. In a history major you might take 103, 106, 219, 217 and some stage III papers and you'll get grades like A-, A, B+, A+ and whatever in your stage III papers. The trouble is that for each of these courses you'll have multiple different assessments, so a course is both equivalent to a standard (from the perspective of a major) and equivalent to a subject but with one overall mark (in that it has several different assessments within it). I don't know if that explanation helps. Say something about it, please.

But I was talking about percentages. In terms of having one single mark per course, each assessment, say, assignment one and test one has a different weight. That weight is expressed as a percentage. Some pretty common weights for assignments are 4% (e.g. 5 assignment stats papers), 5% (e.g. 4 assignment stats papers) and 10% (e.g. assignment 3 in Stats 10x) whereas I think most tests are between 20 and 35% (but I've had 40% tests on several occasions so maybe I should extend that to 40%). But you also get marked in terms of a percentage. (The letter grades don't really get used, except on SSO at Auckland.) We'll use my Stats 10x results as a learning aide here because it's the largest course at the university so you might well end up taking it yourself (or have already done so). For the three assignments (weights of 5, 5 and 10 percent) I got 90%, 100% and 90% (our marks were rounded to the nearest ten), which meant that the contribution of the assignments to my final mark (out of 100) was 18.5 out of a possible 20 (4.5 + 5 + 9). I imagine this wasn't too much more than the average because the individual average marks were 87.4%, 92.3% and 84.4% (i.e. in Semester Two 2014). You might wonder how these percentage grades come to exist (especially when dealing with essays) but the marking schedules so familiar from NCEA generally don't exist... and even when they do, what says that Question 2(b) should be worth 4 points and Question 2(c) 3? With NCEA, an Excellence level question requires different things to a Merit question. That is, there is no way to convert a percentage system to NCEA because E questions aren't just harder, they're different.

Thinking back to my experiences as a first year, you might think it natural to make credit counts and weights equivalents. As an example, you could treat a 20% test as being roughly the same as 5 credits (24 credits in a course, 20% of 24 = 2.4*2 = 4.8 = 5).  I would advise against this. I can remember trying this just the one time (for what I think was my first uni test), but I feel it's useful to explain why such translation isn't helpful. If we take my year eleven English results, I got 7A, 14M and 7E credits altogether (each 7 was a 3 and 4 credit standard, all E's were internal), I was pretty happy with that return (partly because I hated the subject by this point in time) and you might've been too (I don't know). Now, say I get the 7 A credits pretty early on. That's okay, that doesn't stop me from getting the remaining 21 credits because everything is pretty insular in NCEA but university has just the one mark, so assessment isn't independent. If we tried to understand 85-100% as Excellence, 70-85% as Merit and 50-70% as Achieved (in terms of the amount of effort required), and 7/28 credits as a weighting of 25%, if I then got 69% of 25% my weighted return is 17.25 marks. Consequently my maximum possible mark for the course thus becomes 100-25+17.25 = 92.25 because I've dropped 7.75 weighted marks.

Dropped marks? How do you interpret that? Well, if I did get 69% on a 25% test, I'd be thinking I'm lucky. I'd be disappointed but I'm lucky because I can still get an A+, although I now need to get a minimum of (75-2.25)/75 = 97% from all the remaining assessments to do so (which is, in my view, very difficult to do). In other words, it is now harder to get a the "equivalent" of an E* (before I just needed 90% on all assessments), whereas in NCEA to get an M or an E for the remaining credits my level of performance is unchanged by the 7 Achieveds. Also, imagine (even though I've just shown it doesn't really work like this) that I can translate the aforementioned English result into a rank score and then divide it by the maximum possible rank score to reach an approximation of my percentage return from Y11 English. This turns out to be 84/112 = 0.75, whereas I am only satisfied with greater than or equal to 80 marks at uni (a priori... see Maths 150 and 250 where I was happy to pass). Why? Because 20 percentage points is a lot of marks to drop. Indeed, my advice is that the way to think about uni is through dropped marks and the maximum possible final result. Don't get too obsessed about this but this, I feel, is the appropriate conceptual framework to use.

Okay, so that's how to understand assessment. In practice, the way to think about this is to choose a final grade as a target and then try and do better than that level in every single assessment ("shoot for the moon"). If the course has an exam, hopefully you end up with a bit of leeway. For example, I've been trying to raise my GPA so I've been trying to get to A+ in every single one of my courses (I managed 4 from 11 courses this year). In one course, I managed to get 91.93% of the coursework marks (assignments and tests), which just so happened to represent 50% of the final mark (this doesn't always happen, for one of my other courses it was 30% and some courses have no exams so it's 100%). This left the following calculation:

90 (minimum level for A+) - .9193*50 = 44.035
44.035/Exam Weighting = Percentage Mark in Exam Required For A+ = 44.035/50 =  88.07%

As it happened, I didn't get anywhere near that (it was 73%; the top score was 96%, minimum 19% and mean/average 62.9%), but you can see that by doing better than your target mark in the coursework you reduce the burden you must satisfy in the exam. This is good for several reasons. Firstly, it makes the exam less "high stakes" than it could be (which is good for motivation when studying). Secondly, while this can make your semester more demanding, it does mean that you get a fairer idea of your abilities. With Accounting 101's second assignment I was prepared to not do so well in that in order that I could do better in some courses I cared more about. This was a mistake because while I recognised that I did worse than I expected to even slacking off, I didn't really know whether I didn't know the content as well as I thought or if I just hadn't tried hard enough. Thirdly, it may make your semester less demanding because you're motivated to start things as soon as possible. That matters because it's due dates, not release dates, that pile up. In any case, make sure you plot out when things are due in the first week. This will pick out any test clashes that you may have (these happen) and tell you which weeks you can expect to be busier in. This will be a big help because some courses give you a little bit of choice about when things are due (maybe you can pick Week 7 for that presentation or possibly the course has multiple essay questions due at different times: due date flexibility is a rarity). Maybe knowing that Week 6 is going to have lots of things due means you know not to pick up extra work hours because week one was chill or says to you that you can't go to this event but the similar one in Week 7 is fine.

There is one other aspect of assessment at university that doesn't necessarily appear related (indeed, it took me more than a week after making this post "live" to realise this oversight): Grade Point Averages. GPAs are known to, I feel, to most NCEA candidates in that they're a staple of American school-based media, but I certainly wasn't familiar with the GPA prior to university. At first glance, it doesn't have much bearing for assessment purposes to note that your final, holistically derived (from the weighted marks) grades are averaged across all your courses. But the truth is, you really should be listening when the university tells you that GPAs live with you. It's not an inherent part of the assessment philosophy, but it does matter that the assessments are intended to have fairly permanent impacts on a measure which can be used to determine entry to courses, to scholarships (and staying in them) or to new programmes (e.g. switching undergraduate programmes or applying for postgrad stuff). And sure, if you've heard that GPAs are pretty crude, you're right. But they still exist. And what it means for your or me when we're a first year sitting down for a first test is that the results of this test will be living with us in two, three years or whatever. As we say in sports (or I do, anyway), it's not what you do, it's the recovery... if your test performance was terrible and the weighting high enough, you could end up really restricting your GPA because your maximum possible mark goes way down. Don't stress about this too much, though. Firstly, because it's really just an extra thought to attach. Secondly, because if you think too much about Big Things, you risk daunting yourself and making things psychologically tougher than they need to be... focus on the task at hand, only remembering that you these long term concerns. Thirdly, because GPAs are recoverable to some extent. You want a GPA of 7, fine. You get a 6 on one course... that's fine, you need an 8 from another to compensate. But the higher your GPA is, the harder it is to compensate for cock ups... but the bright side is, that's because your GPA is high.

I feel like NCEA and uni feel pretty similar in terms of how the assessment works. After all, coursework and internals are pretty similar... sometimes you've got weeks, sometimes you have days and sometimes you have tests... and exams are exams. But they're fundamentally different philosophically. In fact, I hope I've shown you that you do need to appreciate that one thing should feel different: at university, assessment is holistic, not isolated and modular. In fact, GPAs mean that performance across courses/subjects is more holistic too. I don't know if knowing this makes university better, but it certainly doesn't make it harder... and I wish I'd appreciated many of this post's ideas more when I was a first year. But there's also a snappy takeaway here "think about your maximum possible mark," and that I think definitely helps.

*Confusingly, I am treating an A+ as equivalent to E here but just before it was A and A+. But you can see the calculation for the A inclusive equivalent works out the same way: (desired mark - already gained weighted marks)/(maximum possible mark- already gained weighted marks) = (85-17.25)/(92.25-17.25) = 90.33%, which is more than 85%... which is the lesson being illustrated here.

No comments:

Post a Comment