Thursday, 12 February 2015

Fall in Scholarship Exam Candidates 2014

At the start of last year I wrote a post called "Scholarship Fee Changes". I briefly described the change (from three free ones to paying $30 per standard attempted) and talked about how this was an inequitable system. I also described how it incorporates yet another example of the unreality of New Zealand's free education system (i.e. that there is an NCEA charge of, currently, $76.70). At any rate, despite the fee changes only coming into effect for this year (i.e. 2015), is reporting a fall in the number or Scholarship candidates.

While we cannot be certain that this is due to the fees, I believe that it probably is. Developing a mechanism for this isn't so easy, however.. In my experience at a mid decile state school (as opposed to a high decile or private school where a factory like approach to Scholarship is taken -- this is partly why so many of the top awards come out of the same schools year in, year out), those who do Scholarship are those who put their hands up to do it. If the fee changes were to have an effect, it would rely on smart people misunderstanding the information available and assuming that the fee changes were already effective. However, there is some level encouragement coming from the school and/or teachers (this is potentially subject dependent). If, say, a school had traditionally seen many of its Schol candidates choose to sit the exams after having been tapped on the shoulder (as it were) by its teachers, then we could expect that teachers, adapting for when this would be necessary, tapping fewer pupils on the shoulder. Teachers don't want to set their pupils up for failure and would be particularly incentivised to be more selective if the pupil stood to lose something from not succeeding.

Naturally, this is a problem that would affect schools in the lower and middle parts of the decile system more. These are schools which have fewer resources due to flaws in the decile system and, due to a whole range of reasons, have to be far more focussed on passing. That's as opposed to building on success. Two big reasons are internal and external pressures to get good pass percentages. One of the flaws with school choice is that parents won't send children to schools when they think their children won't get a decent chance to succeed. New Zealand's approach school choice has been a half in, half out thing. Many lower decile schools don't have zones or have very large ones because they have very few pupils. This is largely because parents have some ability to get their children into other schools (whether through private options, out of zone applications, lying about where they live or moving house). As a consequence of this, and tables published by the likes of Metro, there is a fair amount of pressure on schools to get good results in things like NCEA that is external. Naturally, within a school you're going to be generating pressure to help pupils pass (without resorting to unethical actions) anyway. This is particularly true if many of the pupils are marginal or behind... as is quite likely when you're in a school system as inequitable as New Zealand's. This adds up to environment where the majority of resources are focussed at trying to bring people up to where they should be, rather than trying to bring them up to where the pupils (hopefully) would like to be.

There is some limited evidence to support the idea that the fee changes, working through something like the above mechanism, is responsible for the fall in candidates. That is, of course, the sizeable jump to nearly 30% of candidates passing scholarship exams from 2013's 21.8%. This is exactly what we'd expect to see if schools were being more selective in the pupils that they suggested try Scholarship.

Now, there are naturally other options. The 2010-14 cohort may have felt, in general terms, that it was fairly thick and didn't have a hope and, therefore, fewer pupils decided to try it of its own accord due to low self esteem. That's probably unreasonable. Maybe the 2011-15 cohort is considered to be thick by teachers and fewer high achievers were encouraged to try Scholarship in Year Twelve than previously. Again, probably unreasonable. It's possible that there was a decline in the pool from which Scholarship candidates rise. That is, perhaps the decline is not seen if you look at the proportion of NCEA Level Three Candidates who are also Scholarship Candidates between 2013 and 2014. That's obviously a proxy as there are a lot of CIE pupils who do scholarship (and schools which have issues with NCEA strangely, but not really, have no issues with Schol). Maybe the reputation of Schol fell due to the rise in the UE standard and fewer CIE schools encourage pupils to have a go at Schol (but the numbers are pretty big so I doubt it's this one). It doesn't even have to be due to one of these or even some of these... the fee thing I described above could explain some of the fall and factors like these more. Frankly, I'm not sure we'll ever know. But we can make some inferences on the likelihood of each one.

NZQA Annual Report on NCEA and New Zealand Scholarship Data and Statistics (2013), pg. 71

What does this say? Well, retention rates have been increasing for some time now. It is, therefore, unlikely that there was a noticeable drop in the number of candidates around. In other words, we should expect that there is a difference in the aforementioned proportions. Incidentally, these trends hold true for decile, gender and ethnicity breakdowns of the data. I won't include any images of them because they need to be quite big to be of any use. However, the argument would still hold true if there was an overall population dip. Assuming that everything else was pretty much unchanged, a population dip could well explain there being 2558 fewer Scholarship Candidates. And, that is indeed, what is the case:

NZQA Annual Report on NCEA and New Zealand Scholarship Data and Statistics (2013), pg. 75

So, that's a reduction of 2087 (assuming the 60,44 figure is meant to be 60,440). In other words, the population dip is a very convincing explanation. It also means that my proposed mechanism is likely wrong. There could be some effect, which would help explain the increase in the proportion of passes. However, that increasing proportion is likely better explained by the 2013 Schol candidates being guinea pigs. That is to say, the 2013 candidates faced newly realigned Scholarship standards that may or may not have been close to the versions used for the 2012 examinations. The 2014 lot of candidates could, additionally, be smarter anyway.

So, I will have to wait another whole year to see if my predictions for the effects of the fee changes are correct. Because, when you look at the evidence, there isn't anything that suggests that I'm right. Even my one hint, is better explained by another factor. In some sense, then, this blog post was an utter waste of time or an exercise in showing off my rationality (i.e. I can change my beliefs based on facts). It wasn't meant to be like this. It's more an example of the folly in coming to a conclusion and then doing the research.

p.s. in that post last year I mentioned that I was unlikely to have passed my economics Scholarship. That turned out to be true, by a single mark. If I had got that mark I would've been in for a sizeable reward due to having passed history (easily, but not really close to an outsanding pass) and classical studies (barely, just scraped through). All up, that meant $1000, which I've only now mostly all spent.

No comments:

Post a Comment