In his recent working paper Dave Allen highlights the substantial differences in the way that university degree calculations can be made.
The algorithms UK universities use to calculate a student’s final degree outcome can be complex and sometimes counter-intuitive; some commentators suggest that they have contributed to ‘classification inflation’ across the UK higher education sector. A less well understood concern is that the variety in algorithms potentially means the same set of marks can be awarded a different classification depending on what university the student attended.
The 17 questions and answers below aim to clarify the issue.
1. Is grade inflation happening?
The recent HESA data on degree qualifications confirms a continued increase (or inflation) in the proportion of ‘good honours’ (1st and 2:1s) being awarded – from 68% of all graduates in 2012/13 to 75% in 2016/17 Likewise, the proportion of first has increased from 18% to 26%. While the numbers cannot be disputed, the cause is.
2. Why is it occurring?
According to the Cambridge pro-vice chancellor for education, Professor Graham Virgo, grade inflation is not “a cause for concern” and is “down to tuition fees because students are more motivated and are working harder”. Alternatively, others like Nick Hillman, director of the Higher Education Policy Institute claim, “Universities are essentially massaging the figures, they are changing the algorithms and putting borderline candidates north of the border,” (The Telegraph)
3. Have there been changes to university degree Algorithms?
There have been significant changes to degree algorithms in the last 10 years. The recent report by UUK-GuildHE (Understanding Degree Algorithms, Oct 2017) found that many HEIs had changed their algorithms – primarily to ensure internal consistency between departments and faculties, but also to achieve “competitor or sector alignment” (page 18). (see also Higher Education Academy’s 2015)
4. What is a degree algorithm?
Degree algorithms describe the process universities use to translate module outcomes into a final degree classification (1st, U2, L2, 3rd). The algorithm software calculates the weighted average of the ‘counting’ modules, this average mark then determines the classification.
5. What is a weighted mark?
Degrees are made up of a number of modules and can have different credits e.g. 10, 15, 30, 20, 40 etc., students typically study 120 credits a year or 360 credits in total. The weighted average takes account of different module sizes (credit weightings). To calculate a weighted average the module marks are first multiplied by their credits, these weighted values are then added together; finally, this total is divided by the total value of credits.
6. Are all algorithms the same?
While all UK universities adopt the same classifications, how universities arrive at these classifications is a very different matter. The variation comes in how the average of each ‘counting’ year is weighted and whether some module marks are ‘discounted’ or removed from the calculation.
7. What is a ‘counting’ year?
Simply those years of study included in the degree calculation, it is notable that most UK universities do not include year 1 marks in their algorithms – the focus is on year 2 and 3.
8. Why is there a greater weight on year 3 studies?
The higher weighting given to year 3 marks captures the notion of the student’s ‘exit velocity’ or the standard that the student is performing at as they graduate from university. Alternatively, the higher weightings on year 3 might reflect a university’s requirement that programmes must become more challenging as students progress through them.
9. Is each counting year weighted equally?
There is wide variation in the weightings applied to year 2 and 3 marks. This can range from 50/50 [Oxford Brookes] to 20/80 [Derby].
10. How does the year weighting affect the degree mark?
This is best illustrated using an example. Assume the year 2 and 3 average marks are 64.38% and 69.00% respectively. If weighted equally [50/50] the combined average would be 66.69%, if weighted 20/80, this combined mark increases to 68.08 – an increase of 1.39% – all because a greater weight has been placed on the year 3 average mark. It follows that had the year 2 and 3 average marks been switch around, the increase in the overall average using a 20/80 weighting would be smaller [i.e. 65.30%].
11. How does discounting a module affect the weighted average?
Discounting or, removing the lowest marks can only improve the overall degree average. It follows also that “If only the worst, outlying marks are omitted, it is possible that this would lead to grade inflation” (UUK-GuildHE p.37).
Again, we can use a worked example to show the impact. From the previous example, if we exclude the lowest marks for 30 credits (in each year) the year 2 marks become 69.17% (up from 64.38%), the year 3 marks become 70.83% (up from 69.00%). Applying the same weightings the degree mark increases to 70.0% (50/50) and 70.5% (20/80) – the 2:1 is now a 1st. It follows also that the differences between those algorithms that discount and those that do not, will become greater as the discounted module marks get lower.
12. How common is discounting?
Without some central ‘register of practice’, it is hard to say exactly. The UUK-GuildHE survey suggests that up to a third of those universities contacted use discounting. It follows also that a large proportion of those universities that discount also apply differential weightings. The gradual shift in the use of discounting has probably been the significant driver behind grade/classification inflation.
13. What is a borderline candidate?
Most algorithms take the degree ‘average’ to either one or two decimal points e.g. 69.5% or 69.45%. This results in borderline marks where the exam board is a called upon to determine what classification is awarded. There are various methods, one includes using a simple rule whereby marks equal to or less than 0.5% below a classification boundary are awarded the higher classification ‘automatically’ and confirmed by the exam board (thus a 1st does not start at 70%, it starts at 69.5%). Alternatively, marks within a given band (e.g. 68.5% – 69.49%) might be granted an ‘uplift’ in classification (e.g. from a U2 to a 1st) using the preponderance principle: a 1st could be awarded if the student has 60 credits in the higher boundary in their final year. Not surprisingly, these borderline adjustments can have a significant impact on individual student’s classification and the overall profile for a given programme.
14. How do the different weightings and discounting effect a
university’s overall results?
The distribution of the degree classifications can vary significantly depending on the algorithm used. Figure 1 shows a simulation using the same set of marks for a number of students (211 in total) where 6 different algorithms are applied. The first four algorithms (UNI to UNI) have different weights for each counting year (Y2 and Y3 only), these weights range from 50/50 to 25/75, the fifth algorithm (UNI) ‘discounts’ 20 credits from each year and uses a 25/75 weighting. For comparison, the sixth algorithm (UNI) uses all years of study, equally weighted (which would be the outcome if the Grade Point Average (GPA) was applied – see below).
The impact is quite dramatic. In terms of the different weightings alone (UNI to UNI) the proportion of 1st ranges between 16% to 23%. The difference increases significantly once discounting is applied (UNI), form 16% up to 32%. In Figure 1 the proportion of students achieving a 2:2 (awarded where the average mark falls between 50-59%) also declines significantly from 28.9% (UNI ) to 18% (UNI). This simulation suggests a student’s post university ‘life chances’ may be significantly dependant on how their chosen university determines their classification (all other things being equal).
15. Do most students understand the degree algorithm that applies to them?
A good question. The simpler the algorithm the more likely the students will understand its implication, if not use it to set personal academic targets. However, the truth is that many algorithms are very complex, and many use more than one rule to determine the degree classification. Here the interested reader might like to see a YouTube video posted by Sheffield University, in particular the comments below this video. It is also very likely that students do not take into consideration the degree algorithm when choosing a university.
16. What is the bigger problem Grade inflation or Equity?
While the national data shows significant increases in the proportion of 1st and U2 being awarded, we cannot definitively say there has been ‘grade inflation’ – to determine this we would need the actual module marks. The increase in 1st and U2 is likely to be a combination of students working harder and gradual changes in degree algorithms. What we can say – with some certainty: is it is a concern that under the current system the same set of marks can result in such a wide range of degree outcomes. If equity and rigor are to be the hallmarks of UK higher education provision, these differences cannot be ignored or defended.
17. What can be done about it?
If valid comparisons between students’ achievements are to be made, it is follows that all universities should adopt the same algorithm when classifying degree outcomes. In this context the consistent use of the USA GPA classification system (or similar) has clear benefits. Jonathan Wolff (professor of philosophy at University College London) accepts that adopting the GPA is a “move in the right direction” but also takes the view that “we should simply issue students with transcripts to record their study, and leave it at that (Guardian). This is a laudable idea but one which students (and employers) might find difficult to accommodate.