Queen’s University Belfast Transition Experience Programme evaluation

Rationale
An evaluation took place in 2022, of the Queen’s University Belfast Transition Experience Programme (QUBTEP). QUBTEP was Queen’s University Belfast’s (QUB) response to anticipated issues around transition for students, exacerbated due to the COVID-19 pandemic. The main aims of the programme were to mitigate:
•Increased drop-out from university (with the associated financial costs to DfE and the wastage of a MaSN place).
•Reduced student attainment i.e., students underperforming on assessment which will impact longer terms on overall degree classification.
•Disrupted learning caused by a failure to address the challenges to the wellbeing and mental health of students.

Research Questions
The evaluation undertaken aimed to examine the following research questions:
•Have drop-out rates of first-year students changed in 2019-20 and 2020-21 if compared to pre-COVID-19 pandemic rates from 2018-19?
•Are there changes in student attainment (odds of obtaining a first or 2(i) classification degree) in 2019-20 and 2020-21 if compared to pre-COVID-19 pandemic rates from 2018-19?
•What were students’ reported experiences of transition (by examining student reported experiences on the following projects:
1.The Maths Bridging Initiative Programme-Curriculum gaps.
2.The English Transition Project in Arts, English and Languages (AEL).
3.Widening Participation Unit (WPU) Access transition support.
4.Peer Mentoring.
5.Mind Your Mood.

Method
Drop-out rates and final degree classification (odds of obtaining a first or 2(i) classification degree) were examined for years 2018-19, 2019-20 and 2020-21.
Staff were interviewed to gain an understanding of each programme and their perceived theory of change for the programmes. A theory of change encompasses a documentation of inputs i.e., resources and activities of each programme, and intended outcomes for each programme. Developing a theory of change enables understanding and insight into the linkage between inputs and outcomes, and the intended causal pathways. Interviews were also conducted with 31 students who had undergone transition from school to university during the COVID-19 pandemic and had made use of one of these transition projects.

Findings
There was no evidence that academic attainment in terms of odds of obtaining a first or 2(i) classification degree, was impacted by COVID-19. The final report identifies some degree pathways that are outliers in respect of the performance of students.
There was no evidence that student drop-out rates were higher across the university as a whole. There was a rise in AHSS, but this may be due to higher drop-out rates across three programmes that are identified in the full report.
Transition from school to university during COVID-19 was particularly difficult for students. Feelings of isolation impacted on their mental health and wellbeing. Students did not always know where to find help to deal with the issues they were facing. This is particularly true for students who moved to QUB from Great Britain or the Republic of Ireland. With the opening of the new students’ centre, One Elmwood, there is opportunity for signposting and support in one location, that may have been less optimal whilst the student building was under construction.
The evaluation shows that students felt that they came to university knowing less about the courses that they were taking than they could have. Information from staff was not always helpful and staff may need to think critically about reading lists and the formatting of module/course guides. Even when transition programmes planned to deal with academic issues, the main benefits students reported were socio-emotional. There are some pedagogical differences between schools/colleges and university. As such, students reported that study skills personal development was essential to them to successfully bridge the transition from their previous educational settings to QUB.
Students praised skills-based programmes (although effective line management of peers delivering these is essential) and programmes that brought them onto campus prior to transition (e.g., Pathways run by WPU). A significant number of students reported issues of not knowing systems and locations around campus. Effective induction on all programmes is essential to ensure such issues are addressed efficiently. Students really appreciated informal chances to meet each other and meet their academic tutors/lecturers.

Recommendations
It is possible to make a number of recommendations after analysis of the data collected. These recommendations are as follows:
•Degree programmes where academic attainment is noticeably lower than across the wider university, should be examined to ascertain the reasons for lower student performance. Whilst comparisons between subject areas might be fraught with complexities, there should not be significant variation between subject areas which share common academic entrance standards.
•There should be further analysis to determine whether these patterns persist across all year groups, or are restricted to attainment at the end of the degree.
•Further work is required to follow up the student cohorts who underwent transition in 2020-21 and 2021-22 to examine whether there was a long term impact on attainment in these student groups. This should use scaled data (i.e., raw scores rather than just degree classification) and the university should develop a data set that allows multi-level modelling of data to control for confounding variables and clustering effects (e.g., if data is examined just for gender, or just for socio-economic status, it fails to establish interactions between variables, or may prevent correct interpretation of data due to clustering within groups). Taking account of this intersectionality is essential to fully understand the patterns in the data.
•The university should examine subject areas where there is a notable higher drop-out rate of students. This examination should look at the nature of pre-application information given to students and explore other potential issues for disparities in drop-out rates.
•Students reported consistently that mental health and wellbeing was their biggest barrier to successful transition. They felt that these issues are best dealt with by an academic member staff who knows the student. Students have an expectation that staff who teach them will get to know their names and be able to signpost help and advice when it is required. Student mental health and wellbeing should become a priority for all staff in the university.
•Consideration should be given to the creation of a number of mental health, wellbeing and welfare posts across the university.
•The university should examine professional development provision for staff and consider some compulsory professional development for all staff in the area of student mental health and wellbeing. This is particularly important in the early years of any academic career and consideration should be given to emphasising best practice in the current Post-Graduate Certificate in Higher Education Teaching (PGCHET). As the new Professional Standard Framework for Higher Education Academy Fellowship specifically added reference to the importance of student health and wellbeing (Currens & Alexandrou, 2023), then we should also reflect this development in Queen’s Merit Award.
•A quick guide on advice and essential points of contact should be available to all students. This should include a directory of services and essential contacts for when advice is required, and emergency contacts for when things go wrong. Students should help produce this to ensure that QUB fully understands what needs to be in such a guide.
•QUB is now recruiting post-digital students. These are students who are accustomed to receiving information in alternative social media forms and formats. There may be a need to respond to this and adapt how we communicate with students to promote effective communication.
•QUB should think about transition programmes as the new norm post-COVID-19.
•Effective induction is essential to allow students to transition more effectively. All programmes should self-assess their induction for students and take steps to ensure it is effective (again ensuring student voice helps them understand the needs of students).

Conclusion
QUB put in place an ambitious programme of transition activities to mitigate transition issues that may be caused by the COVID-19 pandemic. These new programmes supplemented existing transition support that was traditionally for targeted groups of students (e.g., WPU provision). The transition programmes experiences most valued by students, were those that dealt with mental health and wellbeing support, and provided practical insights into the physical, pedagogical, and academic requirements. Further work is now required to evaluate the usefulness of the wider provision and embed effective transition initiatives into the structures and behaviours of staff across QUB.