Pages

Thursday, 16 January 2014

Blind Faith

While searching for this blog on a different computer I used the search terms "Dale Carnegie" and "NCEA", reasoning that this should find it (even with a number of pages to scroll through). While I grossly over-estimated We Are Here's exposure what I did find was Unclothed Emperor, a blog post on Theology Geek NZ. It appears to be a website with some of the same sentiments of this blog in that it seeks to establish an online New Zealand-centric discussion around "what it means to be in the Church in New Zealand". There are some glaring differences. Firstly, this blog is written by an atheist. Secondly, they like Carnegie's comments. Also, they had no comments so, again, I turned to here. That first point, largely, is why the title "Blind Faith" is the way it is. From the looks of it, their source blog appears to be much the same, so the title stands. However, they do allow comments so I think I'll post a link here as I've already started.

To introduce the substance of the post rather than the origins of my reply... Contra Celsum's comments relate to the Listener report referred to earlier. This is why Carnegie pops up. I will approach my discussion in the same manner that I did with Carnegie's contribution to the Herald article.

Poor Qualifications Exposed 
The education establishment, which is made up of educrats, principals, many teachers, teacher unions and politicians, have long celebrated the National Certificate of Educational Achievement (NCEA) testing system in New Zealand secondary schools.  Recently, however, tertiary institutions provided anecdotal and experiential testimony that our official education system is failing to prepare students for study at universities, colleges, and other tertiary institutions.  The report was prepared by the Tertiary Education Commission--a government quango.
 Right from the start we can see an anti-NCEA bent. That's where most of NCEA's criticisms seem to come from... animosity to the idea itself rather than any basis in the reality of the system. For one thing, this starter is blatantly wrong. NCEA was criticised at the time by those in the establishment. NCEA is criticised now (famously by John Morris, formerly of Auckland Grammar, with the switch to CIE a few years ago). Wherever you go NCEA attracts criticism. Often it lacks any grounding and, typically, is the result of misunderstanding what NCEA is. At its heart, standard based education is a good idea. Certainly, it's better than the old best 50% gets to pass everyone else fails nonsense. That says nothing. If you're in a clever cohort, tough potatoes (some American unis have a similar issue with their admissions scheme). Just cross your heart and pray that your peers are dummies. With NCEA, though,  there are standards. Everyone can pass if they can meet the requirements of the standards. This is a good idea. But, I think, I've already covered this.
According to the NZ Listener:
A confidential Tertiary Education Commission report reveals profound and widespread concerns about the way NCEA prepares students for further study. It paints a picture of substandard mathematics and science education, NCEA students coming unstuck in their first year at university and tertiary providers scrambling to come up with their own diagnostic tests and remedial courses. The document is a summary of formal reports from 15 tertiary institutions – universities and polytechnics – that offer engineering courses.
 Now, I didn't really address the Report in my previous comments. I'll do that at some later date. Here they'll detract from my main point: this Contra Celsum blog-post is wrong. Briefly, the Report is, in large part, about people not taking calculus and physics. It's not the science education, it's the students taking the wrong subjects and the universities blaming this on NCEA (somehow, and I strongly suspect "universities" is, more or less, mostly Victoria).
How does the problem manifest?
  According to the report there are three aspects to the failures and inadequacies of NCEA at tertiary level (at least as far as engineering goes):
The problems the report flags with NCEA fall into three main categories:
• students getting confused or being given poor advice on subject choice;
• those who do the right subjects still being unprepared for tertiary level study; and
• the system not creating a good work ethic.
This is straight back to Contra Celsum's own words. The first bullet point is ridiculous. How is an assessment system responsible for students "getting confused or being given poor advice"? The last exam I did didn't have on the cover (or at the back, or anywhere for that matter) a section on "where this subject can take you". You heard it here first, exams need appendices. Or, at the very least, a url that students can take down and follow to see where to go next. No-one expects this of CIE, why would anyone in their right mind expect it of NCEA? To be honest, it's not something that NZQA should necessarily being mucking around with either. This is a failing of teachers, parents and universities. My big concern was whether or not English was necessary for what I intended on doing. What did I do? When the unis and other providers etc. came to our school I asked. When I wanted to see what was needed for Auckland Uni's engineering programme, what did I do? I went and found it. ACT would love this report... "personal responsibility" and it's lacking. On the unis' part (Victoria's website, for reference, is very unclear on the subjects), on the teachers' parts, on the school's and on the students' parts. What it can not be, should not be, and is not is NCEA's fault.

This second bullet point can be, at least, a valid criticism. However, I'm going to refer to one of the comments on the Listener's webbie. To my mind, it says everything that needs to be said when dealing with the broad statement. After the near-rant of the previous paragraph this should be a welcome change.
It would be nice if tertiary institutions would finally provide details after 30 years of complaining. My recollection is that 30 years ago school certificate and UE had no relationship at all with university courses and it didn't affect the people who are executives today.
Wise words from "Rogermanofthepeople". From memory (I read this article last when it was published online), there is the further complication that the concerns are about an inability to go from the way NCEA wants things done to the way the unis want them done. Roger is putting things into perspective. Although it should get a paragraph, I feel I dealt with the third bullet point sufficiently in Victoria or Victorian? so we'll move on.
Let's address the latter issue first--failure to create a good work ethic.  We can understand how the NCEA system can be blamed for this particular failing.  It works on a system of achievement credits.  If a student achieves a certain number of achievement credits in a subject, they are deemed to be able to matriculate at a university.  But students focus upon achieving credits (often during the school year, via internal assessments) not on mastering subjects.  Maybe some would say the distinction is artificial: to achieve credits in a subject necessarily requires an adequate level of subject mastery.  It does not.  More to the point, once credits are achieved, secondary students are tempted to cruise--hence, the system does not create a good work ethic. 
Let's give credit where credit's due. Celsum has thought of something I didn't in my discussion of work ethic... a means through which NCEA could possibly be blamed. To stoop to pettiness, though, I wouldn't use "latter" in a three point reality. The thing with credits is that they are ongoing throughout the year. There is a tendency for internal assessments to pile up towards the end of term and I know in year thirteen many felt under intense pressure at the end of term three. I didn't but that's because, with my subjects, term two was more concerning. It's for this reason that statements to the effect of, "I've got these credits, time to sit back and relax" don't apply. There is no let up. It's work throughout every term, in every subject for the entire year... with end of year externals looming as well to consider. Students and teachers (and the extra workload of internal marking/moderation and teaching is why a lot of teachers have issues with NCEA) are always facing assessments that matter. Internals aren't mocks, they count. You can't just cruise and then gear up for a week's study (or night's cramming) when the end of year exam or mocks approach. However, he has a point. Stuff done for internals does go out the window. Except, the same happens with end of year exams.
We can illustrate this point via contrast.  Not so long ago in UK universities a student studying for a BA qualification faced one set of exams--after three years study.  What this forced upon students was a work ethic of hard application over a lengthy period of time.  Not only were students confronted with the subject matter of the month, but they could not allow themselves to forget what they had learned two years ago.  The NCEA is the opposite: by issuing credits, students can put that aspect of achievement aside; the more credits they "earn", the more they can cruise.  We have witnessed students sitting NCEA exams who leave after the first (obligatory) 45 minutes, having not written more than the odd sentence in the answer book.  When asked why, the response usually given is that they did not need to sit the exam; they had already achieved sufficient credits to move on.  They were in cruise mode. 
This is, possibly, a flaw inherent in any system where internals/coursework apply. However, when you're faced with a world where not all skills can be assessed in an exam situation (I believe I used the example of serious research, such as in history internals 1.1, 2.1, and 3.1 earlier) internals/coursework are necessary evils (or not because they do mean a greater variety of skills can be assessed). In the past, more topics were covered but they were covered in less depth or with a different focus. What happened then was appropriate for yesterday's world. Today things are different. We know more things. We have more things to know. We must reflect these changing realities in our education systems. But what I really want to share here is the perspective of NCEA candidates. And, like I said in a different post, the place to see their views is on the various NCEA meme pages (to be found on Facebook).


Source: NCEA Memes 2013, credited to Lewis Furmenger

Ignoring that the meme appears to use an electrician (which is a very respectable occupation) the meaning is clear. As far as students are concerned those who leave after 45 minutes are not going to uni. What does this mean in terms of Celsum? Well, simply it's that it's the students not the system. The only reason why they'd stay longer is if they're not allowed to leave. Being forced to stay the entire three hours is liable to result in no additional work either. I've known people to sit, for three hours, and write nothing. NCEA may create a situation where this type of behaviour is enabled. But, I think, in all likelihood that is not the case. Rather they do not care enough to bother. They are satisfied with merely passing and would do nothing more than what gets that, whatever the system. With NCEA that means that by the externals they have what they want. In a pure exam situation they'll do the bare minimum in the exam/s.

I think consideration should be made for the nature of the comparison. While it is true that Celsum probably would prefer everything to function how it was then for then, at uni, these days, it's a system similar to NCEA. There's a fundamental difference in that at uni it's a single mark that is an average across the exams and other work that students do. NCEA is separate. For a subject like English there is no pass that gets a mark out of NAME. Instead it's a series of separate results. In level one, for example, I had 7A, 14M and 7E. In a uni system that would be something like M for the subject (rather course). This is probably why Carnegie can make his observation (remembering that he seems fixated on Achieved students) but others make a counter observation that NCEA serves students well. Who to believe? Probably only foreigners. Everyone in New Zealand basically has a side and sticks to it regardless of changes made or realities that never suited the argument. I will point out that the LSE accepts NCEA and if you're unaware of what LSE means in this context, read pure facts before trying any opinion pieces like this.
Dale Carnegie, a professor at Victoria University’s School of Engineering and Computer Science, prepared his university’s submission . . . in conjunction with the Wellington Institute of Technology. He tells the Listener: “I would absolutely stand up in front of anyone and say that for the bulk of the students that Engineering sees, the study habits that they have developed and been permitted to use for NCEA have adversely affected their ability to survive university engineering study.”
Carnegie’s analysis shows that students coming into Victoria with NCEA Merit or Excellence grades are coping well. But most of their engineering students get into university based on “Achieved” grades. Anecdotal evidence is that some of those students “don’t sufficiently understand the subject and we feel that, historically, they would have been awarded a fail grade”.  Carnegie’s submission warns that students tend to “game play” NCEA at school, then run into problems when they realise this is not possible at university.
But it’s not just work habits.  Students in engineering schools also lack basic numeracy, implying the academic standards in schools are far too low.
This is mostly the Listener's comments so I'll address those. Carnegie is basing his remarks on "achieved" students. Auckland Uni, as I've mentioned before, requires at least 70M and 10E credits to meet the rank score to get into their engineering programme. That's right up there with programmes leading to medicine and surpasses the requirements for most conjoint options (as two degrees in four years a higher academic standard is expected). Basically I'm saying, "Bad Victoria, your lower standards are causing problems because they don't reflect the realities of what you offer." This is false hope for many students and it's clearly part of the problem. Victoria must fix its admissions and information schemes.

Gaming the system is something that pops up a lot in discussions around NCEA. Teachers and pupils are aware that with the advent of GSM attempting everything is the only rational choice. Results of M5, A3 and N2 in a three question exam would likely yield a pass with a score of 10... particularly if the cohort doesn't meet NZQA's estimates (this is something that is worth criticising) and, instead, performs more poorly. Other methods offered are inserting appropriate key words but I struggle to see how that doesn't apply with any test. There's a difference between knowing a concept and being able to relate it to a situation... keywords are the main way of separating this. Rote learned essays are a problem that plagues NCEA and, I imagine, any essay based exam. It's not something that I ever really tried (apart from my English mocks in 2012, didn't pay off) and NZQA takes measures against it by using questions that are different to those routinely used as exemplar questions. From their 2012 assessment report for level two English, "Questions were framed so as to discourage candidates from delivering an essay prepared by themselves or others before the examination." I recommend that anyone wishing to discuss NCEA read the reports like these and, if they're not a current pupil or teacher, consult a explanation of NCEA before attempting to find a soapbox. This was also done in the level three Classical Studies exam in standard 91394, where the Religion and Ideology question related more to the "Ideology" rather than the "Religion" part of the topic. This was enough to make me change my topic as my preparation had been focussed on the "Religion" aspect. I managed a merit in the end. There are other methods but, the reality is, that people will always try and find ways to reduce the effort required. Carnegie is wrong. What they realise is that other methods must be used.

Numeracy education in New Zealand, like English in a more grammatical (or, frankly practical) sense, is troubled. There are things taught pointlessly early and others taught later than they could be. But, again, Carnegie is the source here and we've got to realise that Victoria, for some reason, lets people who have done irrelevant maths (statistics as opposed to calculus) to do engineering. That must be considered. Although NCEA is an assessment system it can be considered at fault here. During the realignment a lot of things were dropped from calculus (such as first principles) leaving teachers and resources at something of a loss to figure out what was still around and what wasn't. The two calculus internals that I sat were also piss easy excellences (arguably the easiest seven E's I ever obtained) and I think that would be the case whatever happened. In fact, I think there's a very real argument to make that we could have tried to fit a third internal in but this is school specific so only leaves a moral story: what is the case in one place isn't necessarily the case elsewhere. It's a point that complicates this discussion but it's not "lol, internals". Instead it speaks of the differing compositions of courses like "calculus" depending on the school. Some schools are probably less willing to allow one of the more useful English standards to be optional than mine was. Speeches, as a result, tended to have little more than half a class participating at best. This may have continued in year thirteen but, of course, I'd stopped taking it after year twelve.
On the frontline, the report shows providers are telling the TEC that even when students pass the right NCEA subjects at school, “the majority of students are still unprepared for tertiary-level mathematics and physics”. Poor grounding in mathematics “was highlighted as a major issue for almost all providers … Some prospective engineering students are not equipped with basic numeracy skills.”
Looking at the paragraph in question it appears that the irrelevant maths problem has been avoided. Instead, I think a legitimate complaint and the achieved issue can explain it. But, back to Contra Celsum.
We do not mean to imply that the NCEA system is an abject failure in everything.  But it must get the basics right.  To certify students as qualified to undertake tertiary study in any subject, when in fact they cannot cope, lacking the basic fundamentals of further study in that subject has to represent a fundamental failure.  The success of education is not to be measured by the qualifications a student achieves (which lamentably is the chief focus of  NCEA--and of the Government, for that matter), but what they can go on to learn and master next.   
Meanwhile, a warning siren is sounding for parents who want their children to go on to tertiary study: do not assume that NCEA achievements will be a currency of value when it comes to university or polytechnic study.  It would pay to be duly sceptical of the bullish pronouncements of the government schools, the educrats, and the teacher unions and the New Zealand Qualifications Authority (which administers NCEA).  They are all talking their own book. 
I think I've shown that there is no reasonable evidence presented by Contra Celsum or the Report to justify this assessment. It pays to actually be aware of NCEA. And, that "warning" is plain wrong. NCEA, as long as it exists, will provide something that universities etc. have to recognise. But, parents and students alike be aware that for the year thirteens of 2014 University Entrance will require NCEA Level Three. That means about 18 more credits and they still have to come from a specific combination of subjects. Individual providers should (unlike what Victoria appears to do) have additional requirements that have to be met. Whether that means subjects or greater academic attainment (for example, Auckland's rank scores). This is a warning worth reading, rather than ill-informed scare-mongering.

No comments:

Post a Comment