What a difference a day makes... 24 little hours... (of an exam!)

Seems like a timely point to do a blog about exams - though maybe too late to help anyone as exam season hits and papers are already written, moderated, and approved. Still, let's dive in!

One of the larger changes in the paradigm shift that is "going digital" in so many aspects of the university experience (for students and staff) has been that of a huge number of universities across the country changing from campus based exams to online exams. And not only the shift in environment, but also something of a panic-stricken move to doing anything possible to mitigate for anything and everything, even when doing so might compromise the point of having the assessment in the first place - that of 24hrs, open book, and all the other bells and whistles we as a sector have thought of.

Now before anyone "comes for me", I totally support some of the changes; was using a few of them myself well before the world turned on its head. To my mind, "traditional" exams are dinosaurs (except, you know, dinosaurs are still cool) and serve little purpose now we're in the digital age. In a world where we can access so much information almost instantaneously and wherever we are, what you know is now not quite as important as what you do with what you know. There are always going to be exceptions of course - medicine being a good example where sometimes you need to know in the moment, and you need to know a lot and be really sure. We'll always have that. There will always be times in any job where you need to know something right off the bat - there'd be no such thing as expertise anymore if that weren't true. But it's not exactly standard anymore for any job to put an employee in a position where they are effectively sent to a windowless basement with no access to the outside world, given only parchment and a quill and told "write down everything you know off the top of your head!" What would that be assessing exactly? What real world, authentic example are we looking to simulate there? How to work under pressure? How to break people? Plus, handwritten exams invariably mean poor formatting, editing, structure, those annoying little arrows in the margin to indicate where they would have preferred that paragraph to do had they not had 15 minutes left and therefore not enough time to re-write the entire thing to put it all in the order they know works better. Even if we simplify that down to a more short-answer or MCQ style exam and apply its value to the rapid knowledge example of sometimes needing to know right off the bat, you don't often need to showcase all the facts crammed into your head, and certainly not for three straight hours. No, the "reel off facts" or quick decisions thing is a bit more of a 30 second thing, and much more focussed.

Access to information is important (and thus confirming something or getting evidence via the internet is important). Crafting your knowledge and understanding into something meaningful is important (and thus being able to edit and reorganise and format digitally is important). Those are both skills that are genuinely used in almost every field of work. So what would an exam situation be looking to achieve now? Where in what's becoming a favourite adage of higher education, "assessment for learning, not assessment of learning" do exams fit in? Is it to have students "prove" what they know now that they can, on any given day outside the exam hall, look up even a prompt to get them to what they know? If anything, we've always known that this is not the case, considering cram revision is a centuries old practice. Ask any student any question off an exam a week after they sat it and you'll be met with a vacant stare as if they think you operate only on sense of movement like a T-Rex and if they don't move you might forget you asked the question and move away (see, brought the dinosaurs back there). Same with any of us with most things - it's a reality of life. We leave things until the last minute, and we cram knowledge into our brains for its immediate need, then push it out as soon as the need has passed.

So a change was definitely needed, and one thankfully prompted by an otherwise awful situation the world found itself in. For my part, I had been running my final year exam as an online and open book one for several years beforehand. I took a standard paper-based, handwritten, three essays out of six over three hours with each essay being a different topic, and turned it into a single, unseen, deliberately open style question done as an open book exam on PCs. It was still invigilated because I wanted them to use peer reviewed sources as evidence of any declarative statements, not wiki, and thus the limit of where they were supposed to look for that evidence was watched over. But I kept it to the original three hours to allow for the fact that they would be looking for information, and encouraged the following practice:

• spend an hour writing it off the top of your head (because you prepared, didn't you)

• spend the second hour looking for evidence to back up what you've said (and reference it)

• spend the third hour editing and formatting and reorganising

To me it felt more authentic, but still (thankfully as the first cohort experienced the thing and the marks came through) academically robust and assessing all the skills I expected of them, while giving them far more room to move by the single open question. It even assured they didn't cherry pick which topics to revise because now they had no idea how the question, the brief, may be framed and whether it would force them to draw on multiple specific elements of the taught content (and it always, however I wrote it, required multiple examples from multiple areas). How they wrote things became the proof of the pudding as to who had learned, and who was relying heavily on having the internet at their fingertips.

But then covid hit, and everything had to go online. The first year of this in the summer of 2020 was a sudden shift to having to convert what we had to digital. Not so much to redesign it for digital, but to make it available digitally. And with this came the recognition that we were (okay, not me as that's how I'd rolled for some time by that point anyway) having to implement something analogue and fenced off in a world of digital and distance. It meant risks, for students (connectivity, disruptions, isolation) and for staff (the potential for cheating in all its forms and the nature of the questions being weakened by information no longer being the prime driver). And with so few people being digitally agile, digitally prepared, and with enough headspace to work the problem as well as we might have liked, we had to open the whole thing up. We (and when I say we, I mean the majority of the sector) went with what could be best summed up as, "a paper exam done online with the world at your fingertips, and because we appreciate doing it this way is difficult as people have different home situations and connectivity, we're going to give you a whole day to do each of these - we'd really like it if you could only spend the time we ask you to, normally 2-3 hours, but we understand we can't police that so we won't hold you to it."

Sounds good, right? Fair for everyone, less time pressures, more room to rest and decompress and put your best work in; to take a big break after opening the paper and the ability to go away and brainstorm the thing before starting to write. But there are two problems with this, and one is something we should have seen coming, some did see coming, but we by and large only learned it the hard way.

The first goes back to my issue with old school exams - what are they simulating? Where is the authenticity? They've always had their issues, but traditional exams did at least put students in a position early on that they would undoubtedly face in almost any career they pursued - the need to bring your "A game" under a time pressure. The way I've come to see it is that coursework and exams perform two distinct and important roles (as much as many scholars and universities claim we should do away with exams entirely because they have had their day, this is short-sighted thinking stemming from not seeing a way around the problem or of enhancing the model to solve the problem and create a new opportunity, and so why not ditch that which one may not be able to fix). There are (broadly - let's not get bogged down in semantics) two ways in which performance, skill, and output is required in work: one is a multiple month project that one (or a team) work towards completion for some goal, objective, strategic aim and the end result is an output of some form that will be judged in some way by someone (a board, a client, a manager, whatever). The other is a short term and unexpected need that one had only a few days to complete (the old "I need that report on my desk by Friday"). Could be an update, a diagnosis or diagnostic test, a client brief, a report, doesn't matter. Coursework simulates the former but in a shorter space of time to reflect the time students spend on modules at any level of study - so instead of 6 months for a big project, you have 8-12 weeks. Exams simulate that more urgent need where maybe you don't have the time to compile reams of data or evidence, or to go through multiple drafts. Only they do a few days in a few hours. Both serve a realistic and authentic purpose. So where does taking what academically we feel should take two hours of focussed work and turning it into two hours worth of work stretched over a 24hr period come under that purpose? It doesn't. It makes it a mini-coursework, and so it is no wonder with that now widely used approach people ae starting to wonder what the point of an exam is if that's what it's become. It's killing itself through us not thinking creatively, laterally, logistically, whatever you want to call it.

The second issue is one far more on the radar of higher education in general - mental health. By taking 2-3hrs of work and telling students they have 24hrs in which to do it, who wouldn't use that time? You've moved the deadline is really all you've done, so of course the average person is going to think "I have more time, I shall use that time." Some may decide to only spend the time asked, but start the thing later in the day, well into that 24hr window. Some may start as soon as it's available and spend only the time asked. Some may spread that time out over many hours by having multiple short breaks (doesn't mean it isn't on their mind in that break time, even if they aren't putting font to screen). But the majority will maximise the opportunity that has fallen in their lap. And that's the problem; they use excessive time. And that does two things and is done for several reasons. First it creates disparity among submissions by now having no clue who submitted excellent work that took them the 2hrs you told them to spend, and who submitted excellent work that took them 18hrs of effort. That makes feedback somewhat meaningless because you now have no idea if they could reproduce that quality under true time constraints. And equally for those who do not do so well you have no idea if they could implement any feedback given them under any time pressure if they spent excessive time doing it a bit wrong in the first place. So secondly you're setting them up for almost a false sense of accomplishment or ability by their grade and any feedback not being truly reflective of their abilities because those abilities may not have been applied within the rules of engagement you intended them to operate by. And there's the why they will use that excessive time - fear and anxiety. They worry that if they don't use that extra time they will not get a good enough mark, or that they will automatically be judged on the presumption that they used more time than they were told to, so they better had use that extra time because doing the work in the time told and it being judged as if having spent excessive time on it might result in a lower grade. And the ability to go back over the same work over and over again over hours and hours, questioning it a little more each time, and hearing on their social media groups how others are doing, and so going back to the work because they worry they aren't working hard enough compared to their peers. And this isn't even hyperbole - this is my students telling me this, confessing how long they spent, showing me screenshots of the WhatsApp groups, expressing how anxious having all that time made them. Certainly didn't do much for the mental wellbeing of those who already had depression, anxiety, stress, OCD, and everything else (as much as wider windows of completion were supposed to give those very students more breathing room and prevent anxiety). But now the neurotypical students were stressed out of their minds, second guessing their work and their abilities, and re-writing their submissions to the point where they made little sense to them.

It's not a long term solution, and just doing away with exams entirely is no better - succeeding at university is only a good thing if you continue to succeed in your first job and into your chosen career because you're prepared by what university taught you beyond the content. We've only done our jobs in preparing you for that world if we have done considerably more than told you stuff and made it nigh impossible for anyone to achieve anything less than excellent by removing (almost) all the barriers. If we reduce assessment to such a degree that it no longer provides any real driving force for anything, surely the value of what you came to university for lessens. The value of university should not be how high you scored, but how much your learned and how well the experience (social and academic) prepared you for the next step. No... 24hr exams in their current package (and I've checked - lots of places are doing this in the same way) need to remain that which they were originally designed to be - a stop gap, a sticking plaster, a patch job for a time where there simply wasn't the time to do much more.

So what should they look like then?

Well - and here comes the proposed aid to academics, the solution for scholars that this website and these blog posts are supposed to eventually get around to throwing your way - let's use the technology and the digital platforms we have to our (and students eventual but sure not to be appreciated until much later) advantage. I'll keep this bit simple just in case you tire already of my usual ramblings and want the goods handed over before you lose the will to live.

• Keep exams as online and with 24hr windows of availability (i.e. it'll be there for you for 24hrs to start at some point within that time, but the deadline for submitting is the end of that 24hr period)

• But, set them up as timed events that countdown from the first time you open them. My suggestion there is similar to that which I used on my own online exams before the pandemic; give them the time to complete them that takes into account evidence and editing. For mine as an example, one single unseen question with a 1,500 word limit (not a target - a limit, dear students - write well, not more) should take no more than 2hrs. So let's give them 4hrs to be a little bit lenient and to allow for the open book nature and different typing speeds.

• Set the timed model to auto-submit when the clock hits zero - whatever you have in by then gets locked in. Now one might argue that that is jolly unfair for those with poor connectivity. But remember that not only can that happen in the last hour of a full 24hr exam binge (just as it could in any hour of any window given no matter how wide or narrow), but you also only actually need to be connected to open the thing in the first place, and to submit it - everything else in between can be done offline (and often is, with only the searching being done online, and even then we have phones and data and broadband and libraries, and quiet campus spaces and all the rest).

It's really that simple.

• Those with extra time (or reasonable adjustments, depending on your institutions terminology) get the allotted additional hours (just a duplicate submission point that only they can access, with more time on the timer, same as any extension on any coursework).

• Students still have the choice of when in the day to do it in case they have work, kids to pick up, mental health issues that flare up first thing, work better at 3am, whatever your poison of why you might start at different times.

• They can still work offline most of the time by making the submission a text box thing so they can work on Word offline, then go online in the last hour simply to paste in the contents of the document - or even by file upload, so you just upload your WIP at various and numerous timepoints throughout (just like any work to any deadline - we no longer save to the hardrive of a dying laptop and "hope" it hangs in there for us to email it at the end, we save to the cloud, send ourselves a copy, heck we used floppy discs for back up back in the day!)

• This can all be created easily, marked easily, and managed easily using a host of tools that already exist within our platforms - for me it would be a Blackboard test because it has the setup, the layout, the randomisation (if I so choose, which I likely would to limit the chances of collusion within that smaller timeframe by having a bank of equal difficulty questions that they each get a random one of), the timer, and the auto-submit.

But what we're doing with this model is making a simple and not unreasonable request: find a 4hr window in a whole 24hr period (that to be fair we previously forced on you without excuse and without "much" complaint before covid hit) to sit, focus, and be undisturbed. And even if you are, we've given you twice the time necessary to do this task and to do it well - we just haven't given you twelve times the time necessary to do it well. We only ask that you also find, towards the end of that time or whenever you feel you have completed the work to the best of your ability under these real-life simulating constraints we have built in, a brief morsel of signal to upload the thing for us to mark it and reward you for your efforts. Even if you e-mail it to yourself in the event of WiFi dropout and use your phone and 4/5G to upload it, that's fine (or use your phone as a WiFi hotspot to upload from your laptop/desktop if your home internet is that bad - only needs to be a few minutes of low data use).

This can work for any form of exam really - essays, case studies, MCQs and SAQs. All are perfectly doable provided you build them right (banks, randomisation, questions written in such a way that they aren't immediately Googleable, and so on), design them to take into account and embrace the fact that authenticity in the real world will involve looking for information, and most importantly if you ask yourself as the person running the thing, "what am I looking to get out of them, and what am I looking for them to get from this?"

After all, do any of us really want someone as an employee that we're supposed to be able to rely on who either can't operate under any kind of pressure because university never prepared them for it, or someone that's so used to having such vast amounts of additional time to mitigate for any potential upset that they one day say to us, "Well I know you said you needed it by Friday, but I felt I could only do a good job on it if I took the weekend and sent it to you Tuesday - hope the board doesn't mind."