Please log in or register. Registered visitors get fewer ads.
Forum index | Previous Thread | Next thread
A-Level results - algorithm testing 14:47 - Aug 15 with 3264 viewsPlums

Has anyone seen any reference to whether this algorithm was tested on last year’s full data set? I haven’t, yet surely this would be done.
It wouldn’t be a difficult task to compare last year’s predicted grades V the actual results against what the formula produced for the same set of data. As far as I’m aware, it hasn’t been used as a defence by the government so one can only assume it wasn’t done which is negligent at best.
[Post edited 15 Aug 2020 14:58]

It's 106 miles to Portman Road, we've got a full tank of gas, half a round of Port Salut, it's dark... and we're wearing blue tinted sunglasses.
Poll: Which recent triallist should we have signed?

0
A-Level results - algorithm testing on 14:54 - Aug 15 with 3243 viewsKeno

it was based on the Lambert Team Selection Programme 2019/20

Poll: What the penalty given last night because
Blog: [Blog] My World Cup Reflections

2
A-Level results - algorithm testing on 18:32 - Aug 15 with 3132 viewsMullet

I am trying not to flood the board on all things education, but I saw this from the DfE today. For a "debunking" it doesn't half set up some straw man arguments and undermine itself.

"96% of grades were the same or lowered by one grade" that means that 4% of grades were lowered by at least 2 if not more and most students missed uni places by just one grade etc.

https://dfemedia.blog.gov.uk/2020/08/14/misleading-a-level-claims-debunked/

There is one tweet here which if genuine does suggest that the soundbite that Eton and Harrow had no reductions was not evidenced either



In answer to your question I found this tweeted an hour ago. It appears to show the flaws fairly well, albeit it is a complicated read. The takeaway seems to be that they used the actual rankings

https://tech.newstatesman.com/public-sector/how-the-a-level-results-algorithm-wa

Poll: If Cook had the full season where would we have finished?
Blog: When the Fanzine Comes Around

2
A-Level results - algorithm testing on 20:09 - Aug 15 with 3089 viewsPlums

A-Level results - algorithm testing on 18:32 - Aug 15 by Mullet

I am trying not to flood the board on all things education, but I saw this from the DfE today. For a "debunking" it doesn't half set up some straw man arguments and undermine itself.

"96% of grades were the same or lowered by one grade" that means that 4% of grades were lowered by at least 2 if not more and most students missed uni places by just one grade etc.

https://dfemedia.blog.gov.uk/2020/08/14/misleading-a-level-claims-debunked/

There is one tweet here which if genuine does suggest that the soundbite that Eton and Harrow had no reductions was not evidenced either



In answer to your question I found this tweeted an hour ago. It appears to show the flaws fairly well, albeit it is a complicated read. The takeaway seems to be that they used the actual rankings

https://tech.newstatesman.com/public-sector/how-the-a-level-results-algorithm-wa


Thank you, that’s an interesting read and the Foxglove court case sounds as though it has a sound basis to proceed. As suspected, the testing of the algorithm seems to have deviated from the practice which really isn’t a great idea.
Looking forward to my son’s GCSE results on Thursday 🙄

It's 106 miles to Portman Road, we've got a full tank of gas, half a round of Port Salut, it's dark... and we're wearing blue tinted sunglasses.
Poll: Which recent triallist should we have signed?

0
A-Level results - algorithm testing on 00:16 - Aug 16 with 3020 viewsRobTheMonk

The school Madge works at and that I used to work at have had lots of grades downgraded. The cohort this year are very strong and arguably considerably stronger than last year. Some of the students have been properly screwed over and many teachers and students are rightfully feeling pissed off.
0
A-Level results - algorithm testing on 00:29 - Aug 16 with 3007 viewsStokieBlue

If it was actually done that could actually be part of the issue. It's fairly easy to create an algorithm which can fit historical data almost perfectly, however when you use that algorithm to predict future outcomes it gets it completely wrong.

It's know as Overfitting ( https://en.wikipedia.org/wiki/Overfitting) and is a big problem in machine learning if the user doesn't understand certain pitfalls (a large number of papers have been recalled due to this).

I am not saying this is what happened, I have no idea what the government have done and incompetence seems the most likely factor. Just thought it might be interesting as an aside.

SB

Avatar - IC410 - Tadpoles Nebula

1
A-Level results - algorithm testing on 07:14 - Aug 16 with 2971 viewsElderGrizzly

A-Level results - algorithm testing on 18:32 - Aug 15 by Mullet

I am trying not to flood the board on all things education, but I saw this from the DfE today. For a "debunking" it doesn't half set up some straw man arguments and undermine itself.

"96% of grades were the same or lowered by one grade" that means that 4% of grades were lowered by at least 2 if not more and most students missed uni places by just one grade etc.

https://dfemedia.blog.gov.uk/2020/08/14/misleading-a-level-claims-debunked/

There is one tweet here which if genuine does suggest that the soundbite that Eton and Harrow had no reductions was not evidenced either



In answer to your question I found this tweeted an hour ago. It appears to show the flaws fairly well, albeit it is a complicated read. The takeaway seems to be that they used the actual rankings

https://tech.newstatesman.com/public-sector/how-the-a-level-results-algorithm-wa


Mrs Grizz confirms they did use the actual rankings provided by teachers as part of the Algorithm.

Exam boards (She works for Cambridge Assessment) had to build this all into their systems to publish results.
0
A-Level results - algorithm testing on 07:14 - Aug 16 with 2971 viewsSteve_M

A-Level results - algorithm testing on 00:29 - Aug 16 by StokieBlue

If it was actually done that could actually be part of the issue. It's fairly easy to create an algorithm which can fit historical data almost perfectly, however when you use that algorithm to predict future outcomes it gets it completely wrong.

It's know as Overfitting ( https://en.wikipedia.org/wiki/Overfitting) and is a big problem in machine learning if the user doesn't understand certain pitfalls (a large number of papers have been recalled due to this).

I am not saying this is what happened, I have no idea what the government have done and incompetence seems the most likely factor. Just thought it might be interesting as an aside.

SB


It was done, it matches fairly poorly to last year’s results. Will see if I can find what I read on it later.

Poll: When are the squad numbers out?
Blog: Cycle of Hurt

0
A-Level results - algorithm testing on 07:20 - Aug 16 with 2964 viewsElderGrizzly

A-Level results - algorithm testing on 18:32 - Aug 15 by Mullet

I am trying not to flood the board on all things education, but I saw this from the DfE today. For a "debunking" it doesn't half set up some straw man arguments and undermine itself.

"96% of grades were the same or lowered by one grade" that means that 4% of grades were lowered by at least 2 if not more and most students missed uni places by just one grade etc.

https://dfemedia.blog.gov.uk/2020/08/14/misleading-a-level-claims-debunked/

There is one tweet here which if genuine does suggest that the soundbite that Eton and Harrow had no reductions was not evidenced either



In answer to your question I found this tweeted an hour ago. It appears to show the flaws fairly well, albeit it is a complicated read. The takeaway seems to be that they used the actual rankings

https://tech.newstatesman.com/public-sector/how-the-a-level-results-algorithm-wa


His follow up tweet shows why there were fewer deflated grades though in ‘top’ schools

Ranked 66 and still got an A, because most students are getting A* and As

0
Login to get fewer ads

A-Level results - algorithm testing on 08:36 - Aug 16 with 2926 viewslowhouseblue

A-Level results - algorithm testing on 18:32 - Aug 15 by Mullet

I am trying not to flood the board on all things education, but I saw this from the DfE today. For a "debunking" it doesn't half set up some straw man arguments and undermine itself.

"96% of grades were the same or lowered by one grade" that means that 4% of grades were lowered by at least 2 if not more and most students missed uni places by just one grade etc.

https://dfemedia.blog.gov.uk/2020/08/14/misleading-a-level-claims-debunked/

There is one tweet here which if genuine does suggest that the soundbite that Eton and Harrow had no reductions was not evidenced either



In answer to your question I found this tweeted an hour ago. It appears to show the flaws fairly well, albeit it is a complicated read. The takeaway seems to be that they used the actual rankings

https://tech.newstatesman.com/public-sector/how-the-a-level-results-algorithm-wa


am i reading that right - that the subject in which he claims to have had a grade 'deflated' was further maths?

And so as the loose-bowelled pigeon of time swoops low over the unsuspecting tourist of destiny, and the flatulent skunk of fate wanders into the air-conditioning system of eternity, I notice it's the end of the show

0
A-Level results - algorithm testing on 08:57 - Aug 16 with 2893 viewsStokieBlue

A-Level results - algorithm testing on 07:14 - Aug 16 by Steve_M

It was done, it matches fairly poorly to last year’s results. Will see if I can find what I read on it later.


That's pretty rubbish then, unless they didn't even try and fit the algorithm to a previous data set which is just stupid.

SB

Avatar - IC410 - Tadpoles Nebula

0
A-Level results - algorithm testing on 09:32 - Aug 16 with 2854 viewslowhouseblue

A-Level results - algorithm testing on 08:36 - Aug 16 by lowhouseblue

am i reading that right - that the subject in which he claims to have had a grade 'deflated' was further maths?


ok assuming that i've read it right and his concern is further maths. (i) being ranked 44th would have got him at best a C on the basis of the 2019 data he has given - so why did the school predict an A?; (ii) further maths is the one subject where the exams were still held in 2020 so his B would have come from an actual exam not an algorithm. seems odd to me.

And so as the loose-bowelled pigeon of time swoops low over the unsuspecting tourist of destiny, and the flatulent skunk of fate wanders into the air-conditioning system of eternity, I notice it's the end of the show

0
A-Level results - algorithm testing on 10:38 - Aug 16 with 2809 viewsMullet

A-Level results - algorithm testing on 07:14 - Aug 16 by ElderGrizzly

Mrs Grizz confirms they did use the actual rankings provided by teachers as part of the Algorithm.

Exam boards (She works for Cambridge Assessment) had to build this all into their systems to publish results.


I don't think anyone has ever disputed that CAGs were not used, they certainly were. What we do know is that they have been held up as cheating across the board and the government have "fixed this".

The opacity as to how and to what extent has lead to all sorts of rumours and misunderstandings. It is clear that CAGs seem to be have used where they fit with the outcomes the government desired or felt were tolerable.

That is scandalous and one of the reasons why people are so upset. This sense that they waited for months, saw what happened in Scotland and have somewhere in between blurred everything is typical of this government.

Poll: If Cook had the full season where would we have finished?
Blog: When the Fanzine Comes Around

0
A-Level results - algorithm testing on 10:46 - Aug 16 with 2801 viewsMullet

A-Level results - algorithm testing on 09:32 - Aug 16 by lowhouseblue

ok assuming that i've read it right and his concern is further maths. (i) being ranked 44th would have got him at best a C on the basis of the 2019 data he has given - so why did the school predict an A?; (ii) further maths is the one subject where the exams were still held in 2020 so his B would have come from an actual exam not an algorithm. seems odd to me.


Only 41 students got a C or above looking at the 2019 Further Maths result, so had the cohort been the same size he would have got a D. The inflation claims will be easier to deflect from Eton et al. because they have such a natural advantage as it is, but it does appear even on mild scrutiny they have been given a lighter touch because of this modelling rather than a complete reprieve.

The fact that each year is different, especially in "normal" schools makes it a very crude measure as is. For example, my cohort is bigger this year in one subject with more students who are top and bottom end. How this will come out on Thursday for me will be fascinating.

Poll: If Cook had the full season where would we have finished?
Blog: When the Fanzine Comes Around

0
A-Level results - algorithm testing on 12:32 - Aug 16 with 2748 viewsSouthBucksBlue

A-Level results - algorithm testing on 18:32 - Aug 15 by Mullet

I am trying not to flood the board on all things education, but I saw this from the DfE today. For a "debunking" it doesn't half set up some straw man arguments and undermine itself.

"96% of grades were the same or lowered by one grade" that means that 4% of grades were lowered by at least 2 if not more and most students missed uni places by just one grade etc.

https://dfemedia.blog.gov.uk/2020/08/14/misleading-a-level-claims-debunked/

There is one tweet here which if genuine does suggest that the soundbite that Eton and Harrow had no reductions was not evidenced either



In answer to your question I found this tweeted an hour ago. It appears to show the flaws fairly well, albeit it is a complicated read. The takeaway seems to be that they used the actual rankings

https://tech.newstatesman.com/public-sector/how-the-a-level-results-algorithm-wa


Hi Mullet (TWTD Education Correspondant extrodinaire)
What’s your understanding of the ‘tripple lock’?
The latest BBC News article says the higher of Mock, Predicted grade or Retake but I’d previously read it was Mock, Calculated grade or Retake?
Also some good news in that my daughter’s been awarded a uni place on the basis of her UCAS predictions and I’m hearing that many others have too so it seems that at least some unis are doing something positive about this fiasco.
Thanks
0
A-Level results - algorithm testing on 13:30 - Aug 16 with 2717 viewsSouthBucksBlue

A-Level results - algorithm testing on 12:32 - Aug 16 by SouthBucksBlue

Hi Mullet (TWTD Education Correspondant extrodinaire)
What’s your understanding of the ‘tripple lock’?
The latest BBC News article says the higher of Mock, Predicted grade or Retake but I’d previously read it was Mock, Calculated grade or Retake?
Also some good news in that my daughter’s been awarded a uni place on the basis of her UCAS predictions and I’m hearing that many others have too so it seems that at least some unis are doing something positive about this fiasco.
Thanks


It appears the BBC article has been amended from predicted to calculated grade.
0
A-Level results - algorithm testing on 17:59 - Aug 16 with 2644 viewsMullet

A-Level results - algorithm testing on 12:32 - Aug 16 by SouthBucksBlue

Hi Mullet (TWTD Education Correspondant extrodinaire)
What’s your understanding of the ‘tripple lock’?
The latest BBC News article says the higher of Mock, Predicted grade or Retake but I’d previously read it was Mock, Calculated grade or Retake?
Also some good news in that my daughter’s been awarded a uni place on the basis of her UCAS predictions and I’m hearing that many others have too so it seems that at least some unis are doing something positive about this fiasco.
Thanks


First off, congratulations to your daughter. There is a but of cynicism that universities are awarding places to ensure they don't miss out on funds, but I also saw there are caps placed on places by the government. Either way, it's telling and commendable that many unis are honouring offers and doing extra work to help students not be left in limbo.

Hopefully the public see and appreciate that and students in your daughter's year aren't penalised in any way when it comes to the job market etc.

Apologies I've been out all day so you may have seen this article now, assuming it isn't the one you referred to https://www.bbc.co.uk/news/education-53795831?at_custom1=%5Bpost+type%5D&at_camp

It's got some excellent points in regarding the general discussion, and shows just what a mess things are. The quotes from the Tory MP are commendable and may see him in Dom's crosshairs Monday morning too.

As per other threads and posts, I'm not entirely sure as it now seems to be completely cast into doubt in terms of what it means not just what they did. The fact that taxpayers are funding the appeals process is really telling too. Despite the government telling us it's free.

I suspect that CAGs were simply left where they fitted the model the government already had in mind and then they cherry picked mock data etc to suit. But I could be being so cynical now and obviously way off, no one knows as information is leaked, deleted, misquoted in a way that's all too familiar.

It sounds as if the biggest effect of the triple lock means there is now an unofficial pecking order, which means students such as your daughter are already sorted because the Universities have been willing to make offers that go over the decisions made. There will be those who have mock results (as the article shows) which are variable and dubious, but will allow some people to get into courses quickly.

Those hardest hit are likely to be left last. I suspect a few thousand kids who were in precarious positions when it came to further education have decided to turn their back on it. That decision making process might mean they add to a bumper clearing chase for places.

Poll: If Cook had the full season where would we have finished?
Blog: When the Fanzine Comes Around

0
A-Level results - algorithm testing on 21:21 - Aug 16 with 2583 viewsSwansea_Blue

Yep it was done, per subject. Best subjects it predicted 2 in 3, worse were less that 1 in 3.

Poll: Do you think Pert is key to all of this?

0
A-Level results - algorithm testing on 22:15 - Aug 16 with 2534 viewsSouthBucksBlue

A-Level results - algorithm testing on 17:59 - Aug 16 by Mullet

First off, congratulations to your daughter. There is a but of cynicism that universities are awarding places to ensure they don't miss out on funds, but I also saw there are caps placed on places by the government. Either way, it's telling and commendable that many unis are honouring offers and doing extra work to help students not be left in limbo.

Hopefully the public see and appreciate that and students in your daughter's year aren't penalised in any way when it comes to the job market etc.

Apologies I've been out all day so you may have seen this article now, assuming it isn't the one you referred to https://www.bbc.co.uk/news/education-53795831?at_custom1=%5Bpost+type%5D&at_camp

It's got some excellent points in regarding the general discussion, and shows just what a mess things are. The quotes from the Tory MP are commendable and may see him in Dom's crosshairs Monday morning too.

As per other threads and posts, I'm not entirely sure as it now seems to be completely cast into doubt in terms of what it means not just what they did. The fact that taxpayers are funding the appeals process is really telling too. Despite the government telling us it's free.

I suspect that CAGs were simply left where they fitted the model the government already had in mind and then they cherry picked mock data etc to suit. But I could be being so cynical now and obviously way off, no one knows as information is leaked, deleted, misquoted in a way that's all too familiar.

It sounds as if the biggest effect of the triple lock means there is now an unofficial pecking order, which means students such as your daughter are already sorted because the Universities have been willing to make offers that go over the decisions made. There will be those who have mock results (as the article shows) which are variable and dubious, but will allow some people to get into courses quickly.

Those hardest hit are likely to be left last. I suspect a few thousand kids who were in precarious positions when it came to further education have decided to turn their back on it. That decision making process might mean they add to a bumper clearing chase for places.


Thanks and thanks for your helpful posts.
Yes that was the article which was subsequently corrected.
My daughters subjects grades seem to have been affected more than most but the main thing is that she’s got her uni place. I guess in years to come the COVID year results won’t matter so much.
I’ve no vested interest thankfully but I hope for all concerned that the GCSE results are more consistent with expectations.
1
A-Level results - algorithm testing on 16:23 - Aug 18 with 2379 viewsSteve_M

A-Level results - algorithm testing on 08:57 - Aug 16 by StokieBlue

That's pretty rubbish then, unless they didn't even try and fit the algorithm to a previous data set which is just stupid.

SB


It wasn't what I had originally read but there's a bit in here which looks at applying the algorithm to last year's results:

N:ADPYamalADP CY20 https://unherd.com/2020/08/how-ofqual-failed-the-algorithm-test/

"Even by including some of the data they were trying to predict, OfQual found their accuracy in predicting exact grades ranged from two thirds for History to one in four for Italian. For most non-language subjects, over nine in 10 students would be within one grade of the true result, but 3% of Maths students (for example) missing a fair result by two grades or more adds up to a lot of teenagers. Over 10% of Further Maths students, ironically the only ones who can understand the tortuous workings of the algorithm that betrayed them, would be over a grade away from a fair result."

Poll: When are the squad numbers out?
Blog: Cycle of Hurt

0
About Us Contact Us Terms & Conditions Privacy Cookies Advertising
© TWTD 1995-2024