Towards A new Curriculum and Assessment Agency in Scotland Part Two

 

© Copyright David Hawgood and licensed for reuse under this Creative Commons Licence.

I did promise a follow up to my first post upon hearing the news that SQA and Education Scotland were to be re-engineered. Having taken part in some of the initial consultations.  It was good to hear that the reform will be phased and planned. But I am no clearer on where the destination will be. I am not sure publication of OECD senior phase report this week makes the destination any clearer.

There will be the same impasse around both what the content is in any national qualifications and how it should be assessed. I'll leave that to the end. 

I am going to jump to other side of challenge. There are some parts of the Scottish system that need fixed and these suggested changes would support any future system. 

If we stay fixed on the contentious parts of the challenge,  as I covered in my last post the real and present danger that by making the reforms all about schools then lots of other useful parts of the system could be lost  and we may not make the headway required. 

So here is a list of things that simply need fixed for any system to operate more effectively for all.

• Data : As a system, the Scottish system small as it is , is very poor at capturing and sharing data on what is actually happening across the learning system. To future proof the system we need to acknowledge that everything is now data and we need to set up a new awarding system that is more effective at providing learners , teachers , centres, employers and the broader community access to reliable information.
• It should start by making everything digital by default. Start by designing a system that is future proof.
• This could be as simple as making proper use of the Scottish Candidate Number (SCN),  it has been used  for decades but it is not used  across education and/or consistently by Higher Education Institutions. If you really wish to track attainment gaps being closed and the impact of FE, HE and work based learning this needs to be addressed. It is there already don't invent something new,  just ensure no one gets any public money unless they use it to report on learners' progress.    
• More ambitiously and much more productively would be to publish any outcomes or eventual curriculum in a machine readable way. Yes , other countries do this already ! . Then if a PDF document is the  output you need, you can have it , but by creating data in this way,  the assets can be easily reused across the system. No more collective keying of unit descriptors , experiences and outcomes etc into lots of spreadsheets and databases.
• What is a Digital Curriculum 
• Digitising National Curriculum Standards

• Certification :There is a quick easy win to make all certification digital and online. SQA were almost there, but lacked political support to push this across the line.  A new agency should start by making sure no learner ever needs to worry about a lost certificate again. The system should be set up to allow learner to share a secure view of their certification on any job application etc. Smoothing recruitment processes for all. It would also be cost effective way to deliver richer information to learners. This probably is about data again but needs a heading of its own. 

• Subject Communities Who owns and decides what is in the assessable certifiable bits of learning in the Scottish system ?.  It should be transparent and clear to all learners , parents , teachers. For teachers and learners there should be clear ways for them to suggest and shape the content of awards. There have always been subject panels - you do still need experts - but make the process more open. Qualifications could be maintained by an iterative yearly online process to keep them current. This with clear stakeholder engagement. Solves relevancy issues with computing and some sciences subjects. It needs to be clear that what arrives in a qualification is actually informed by national occupational standards when this is relevant. 

• Learner Communities for learners sitting national assessments the national system should have figured out a way by now to give learners some safe secure spaces to allow learners to access to peer support. If the system is not brave enough to tackle this, it should be brave enough with caveats to highlight services like The Students Room. that have been around for years - but the system should be able to do better. We don't encourage our learners to see that there is a national education system. We should be encouraging national and international engagement. 

• Courses and Assessment  Direct to Learners If most learners now have laptops. The new agency  should work towards having a clearer offer direct to learners. In partnership with relevant agencies Education Scotland, SDS and College Development Network and others. It can be piloted, it does not have to be a big bang. Any learner should have access to any national subject anywhere in Scotland and the opportunity to be assessed and certificated in that subject. This is something that any new agency should be able to coordinate - Colleges ,local authorities, Scholar and other partners can deliver. Perhaps precipitated by more of a focus on open learning. 

• Open Learning Materials  If you follow my blog you will see a lot about this. If the learning content is created by lecturer , teacher and or funded by public money whether through an institution , agency or local authority . The learning material should be made open and available under an appropriate open licence Non Commercial Share Alike to allow teachers and learners to remix and use.  Simply aligned to UNESCO global standards in this area. This does not replace a teacher or trainer but gives learners and teachers access to better learning resources. Initially this does not need massive investment in any national system it simply needs positive policy messaging and support so that all actors in the system adopt the UNESCO guidelines. 

• Staff Development  The new agency should be seen to be lowering the administrative burden on teachers and College staff while not diminishing their responsibility to understand any national standards – There should be pilots around roll on and off secure assessment ( Solar mkt 2) The system  can collectively maintain standards while lessening the assessment burden on teachers and learners.
• Validation process In Colleges and work based learning, the centres actually have to have teaching staff and resources in place to deliver new courses . This includes ensuring that staff have adequate training to deliver new courses.  This may be bridge too far but in many subject areas staff do need annual development. Perhaps having some better validation processes in school sector would raise standards. 
• Verification processes:  it still has not really been picked up but teaching staff do feel insecure on their decision making. Make sure that there is robust internal, regional and national mechanisms to support teacher decision making. Make sure everyone knows that standards they are working to. In an interconnected world this should not be a logistical challenge. Learners need this exemplification too. Imagine you are learning in a school or centre that can't show you what an "A" looks like. In learning it is not mythical - teachers and learners should be clear about competencies and levels they are working towards. 

• Digital Portfolio We should aim to give every learner a digital profile a portfolio of their learning.  They build it and they can decide who and or which components of this they wish to share. This more than an online CV and could include link to their digital certification from a range of sources. There are lots of ways this could be achieved freely and within GDPR regulations. 

The list above I hope is non contentious and looks beyond the battle around what should or should not be included in school programmes ( this is actually not about assessment at all) 

If the reform is because the current system is no longer fit for purpose. Then I really would expect to see the end of paper based examinations - I promised not to be contentious. 

I would go further and re-look at the subject silos - perhaps looking again at the experiences and outcomes and stretching these to end of formal schooling. But that is probably a bridge too far.  The suggestions above will support a new landscape whether we are assessing latin , maths, english, astrophysics, languages , welding , music , digital literacies,  meta-skills  or tap-dancing. 

Towards A new Curriculum and Assessment Agency in Scotland

Less than a year ago I watched political machinations around national awarding in Scotland and made some observations based on the extended experience of both working for SQA  and around the vocational system in Scotland , the rest of the UK and internationally. In the end, events pretty much turned out as I predicted, even to point of there being local academics engaged in the review.  

Then, this June, 2021,  in the middle of awarding season, the government announced that the SQA is to disappear and be replaced by a new Curriculum and Assessment Agency.  This in response to a critical OECD Report. The implications of which are far wider than national assessment. 

So what next, I don't expect any of the provisional grades for candidates to change between now and August and I fully anticipate another record breaking set of candidate results. 

It will be the same picture across the rest of the UK, with the exception that England , Wales and Northern Ireland haven't decided to dismantle their national awarding and accreditation infrastructure in the midst of a pandemic. So, at least in Scotland, we have a ready made scapegoat come August. 

It is time to reflect on : what we need from a national awarding body, what we expect of support agencies, funding bodies, audit and inspection organisations.

They are all, in one way or another, part of the mix.

What is missing is any real form of national discussion around what the future of schools assessment and certification should be. The Scottish government response to the OECD report is big on headlines but thin on detail. The press, including the educational press, tend to stick to the shallow end of any debate on national assessment systems.

BBC Radio Four ran an excellent series simply around thinking aloud about Re-Thinking Education with one programme dedicated to looking at alternatives to the current school exam system in England. I am not a big Lord Kenneth Baker fan but his opinions on ending any form of national assessment for 16 year olds is worth exploring. his thoughts on an academic vocational split at age 14 are abhorrent. 

I am concerned on two fronts. The populist decision to abolish the SQA does seem to ignore the fact that one way or another a new agency(s) will simply rise from the ashes. The timing  is appalling given the stresses and strains on the system, but it is perhaps that simple political expediency that heralds many education reforms, give them the big news just before schools and parliament go into summer recess. 

Lost in this and the only bit that has ever been in anyway exportable is the vocational education system founded in early 1980s by SCOTVEC and buried in the merger with the Scottish Examination Board on the creation of SQA.

In the worst scenario, asset strippers in the form of private sector awarding or the 'not for profit' awarding bodies in England will gladly hoover up SQA's commercial and overseas business and the receipts they brought to support national awarding in Scotland. I wouldn't even be surprised to see elements of Universities Scotland hovering around this area. 

Watch too as the Scottish government struggles with regulatory and other awarding requirements, when potentially the only alternative is the open market.

If the focus is just on schools then a new agency or vehicle needs established to look after vocational awarding in Scotland. It needs to be new and separate from agencies like SDS or SFC who fund the qualifications. Perhaps this is the new quality assurance agency for Universities and Colleges mooted in the SFC review papers and lauded here by Ewart Keep at least SFC talks about the tertiary education system and working with SQA and successor bodies in their Coherence and Sustainability Review.

Clarity here is needed quickly, vocational awards are at the heart of any economic recovery strategy. 

The loss of the SQA brand will damage awarding and accreditation business across the UK and the rest of the world. There will be live business and business in the pipeline at SQA that will all be contractually under threat currently due to uncertainties here. I am guessing not much consideration will have been given to this. 

While this uncertainty remains, I expect to see an exodus of specialist staff and general system stasis. A new national curriculum and assessment agency is not a quick build even if built from the building blocks that become available from a reconstituted Education Scotland and components of the SQA. 

But what about school assessment and certification ? Will removing SQA, rather than reforming the assessment system, solve the issues ?

In my view incessant power struggles over school curriculum and assessment held the whole Scottish system back. They certainly held SQA back from 2003 - 2015 while I was there and they were delaying things like HN reforms when I worked in the College sector immediately prior to that. I am guessing now there will be further delays around the HN Next Gen work which the College sector badly needs. We need reforms too around qualifications in the work based learning sector.

If the new curriculum and assessment body has a sole focus on schools, you then lose a lot of the economies of scale around things like on-line assessment and digital certification and data management and the technologies that are needed at the heart of reforms and organisations of this kind. 

I'll post separately on where the opportunities lie for a new sort of awarding agency for the public good.

Where will consensus on school assessment and certification come from ?

The school assessment system with its narrow subject and exam focus was not really about SQA but an image of what teachers and governments wanted.  To have a more flexible system it starts with the teachers and schools knowing and applying national standards, no matter what subject area, this could be in any domain or discipline, you can then have flexible assessment policies , it could all naturally flow through from the experiences and outcomes.

The awards that currently exist are shaped by subject panels from across the Scottish schools system to design rules approved by the CfE Management Board.

Having worked through the development of Curriculum for Excellence . The OECD Report is spot on in highlighting the disconnect between experiences and outcomes and assessment system.

 I think there was a confidence failure across the system in tackling this - there were some deeper challenges too in the creation and management of the experiences and outcomes and how learners transitioned from broad general education into the senior phase. Heads should have been together on this in Education Scotland , SQA and the sponsoring Education Department. I would put this outcome down to the failure of the CfE Board of Management over an extended number of years. 

The default position for school teachers does appear to be a written national exam at end of year. 

The tech and systems have been in place for years to support more innovative forms of assessment and certification but the school system has fought , resisted  and won battles to keep the exam system and year long courses, while maintaining glacial speed too on any curriculum changes. 

While the civil service and local authorities continued to embrace systems that only recognised achievement in exam based subjects. Universities like this too. I hope there is now a confidence in the system to tackle this.  It needs to be tackled system wide.  There will be a real drive in many quarters to ignore any learning from the last two years and get back to business as usual. 

I fear further insularism - the awarding system in schools needs to place learners more at centre , subject choice should not be determined by which school and where it is located , assessment should take a variety of forms and be ready when the candidate is ready - not annually and teachers should as matter of course be able to make accurate estimates on grading, and certification should be digital by default. 

It is where we have been heading for last two years but without properly having a system in place. Some of the disconnect between costs , expectation and delivery are long standing and need to be addressed, if any reform is to be a sustainable one. 

"The SQA levy to local authorities and entry fees charged to independent schools and colleges for the certification of national qualifications have remained unchanged since 2012-13. These fees contribute to the cost of awarding. The costs of awarding National Qualifications are greater than the contribution made by local authorities, independent schools and colleges."

"Scottish government funding in the financial year 2019-20 to the SQA was £41.4 million but, in 2020-21, that figure almost halved to £21 million"

Source https://www.tes.com/news/exam-cancellations-save-millions-second-year

As they shuffle the deckchairs, I think the focus will be lost on the national vocational system. I hope, given we have now had two years without national exams, the school system can now flex. 

By the end of the review process we should no longer have an exam hall, paper based, national diet of examinations for S4-S6 learners,  we owe them more and we should not have  lost our vocational awarding system. It is a challenging balance.   

Watch out too for the purveyors of state-wide assessment systems , GCSE Awards, T-Levels , A-Levels , International Baccalaureates of one form or another,  door stepping , carpet bagging,  the review , the new agency , local authorities, schools and probably Colleges too.  There are lots of commercial interests and political interests that swirl around this area globally. I expect some Scottish schools will drift towards A Levels as the saga unfolds. 

I wonder too if any of the OECD Report writers can highlight a country that has achieved the vision within their report. It is a strong and achievable vision if the system can finally all pull together. 

Ken Muir has a challenging year(s) ahead.  My thoughts are with my former colleagues continuing to deliver for Scotland's learners within the SQA. 

Further Reading Background:

Scottish Parliamentary Briefing on Scottish School Education

An Experienced Reflection on National Awarding

I worked for the SQA for most of the current century - and every year congratulations are due to all learners and this year, in that respect, it is no different. 

I'm glad I am on a beach in the Outer Hebrides this week.  Working for the SQA is a thankless task and colleagues will be working hard as ever to deliver. 

We need to step back from the hysteria.

These are exceptional times. This is the first time since the 1880's that the national exam diet has been cancelled. This has put a strain on everyone; learners , teachers , administrators and politicians. In times like this tough decisions need to be made and justified.  A decision was made today to uphold the estimates made by teachers. 

This I think exposes some deep fault lines in our system. Teachers have systematically over estimated pupils eventual results for years. The external exams being the method of arbitration. Without the exam this issue is thrown into sharp focus. 

I think the appeals system that was ready to go into operation would have supported the deserving cases. But we will never know. The noise about education being a postcode lottery isn’t just noise - but the appeals system would have adjusted these. 

What is the issue .

Many teachers are not particularly good at designing prelims , and or are unsure about standards. The evidence is pretty well known to those who have worked  in and around the system.  There is often a big gap between learners actual grades and those predicted by their teachers.  Appeals are often made based on invalid evidence, commonly cobbled together prelims based on items from past papers - when this was admissible evidence.  

The system has not done enough over a lot of years to make sure that teachers can make better estimates. Perhaps given the parental and institutional pressure that teachers are under, along with different learner performance between prelim and final - it is too hard a task. SQA has done its bit around Understanding Standards.

SQA has data on the reliability of estimates at school level and I am certain that SQA's initial response was based on sound evidence. In this, and ultimately the change of tack, they follow the instructions from the government.  SQA staff and the 15,000 appointees  ( who are mainly serving teachers ) do their utmost to make the system fair for all. 

Some observations - 

A decision has been made that will be very popular with learners and teachers , it looks as though it has almost cross party support. The government was never going to have an easy decision on this. 

Should the system now accept that teachers make these national assessment decisions ? ( I think  the view is that this is perhaps a one off ( I'd like to see more robust decision making moved here )  . No one, least of all the learners were prepared for what has been an incredible year, an upward drift of 14% across the board, does create a credibility problem , but who knows perhaps lots of learning was happening in lock down and schools and local authorities had to put in long hours creating their orders of merit.  Neatly too it creates a cohort of learners for Higher Education when overseas numbers are down.  Perhaps it just highlights that exams  are not really about quality control just quantity management for the tertiary sector.  It probably mirrors what is happening in the Higher Education sector - where grade inflation is much more of a reality - and when the  dust settles it will have ballooned this year. Please press, don't roll out the usual elite moaners and fixers from Higher Education about the school and college system - they have zero credibility on standards. 

What is clear and to restore learners faith in the system is that learners need a better means of evaluating their performance against national standards. The wide variances between school estimates and the original awards need tackled. It is an opportunity to revisit the whole exam system - roll on ,  roll off digital assessment for all is within reach along with digital portfolios of evidence.  Local Authorities , Education Scotland and the GTCS should take a much closer interest in teacher decision making. It is only recently that the GTCS started recognising teacher engagement in national assessment work as a critical part of CPD and often it is the schools with fewer SQA appointees that have the most divergent estimates.  Perhaps a starting point could be a comparison of previous years performance based on exams and this year's based on estimates. What was this year’s secret sauce ? 

The credibility of the national assessment system, whatever its future shape, is everyone's responsibility and it is ill served by political slagging matches and press hysteria.  I think we are still  not in a place where we can say the academic year ahead is without more uncertainties. 

In the meantime , I wonder how the more market driven education system of England will cope with a similar crisis, A stars all round I wouldn't wonder.