Abstract
Oversubscribed social work (SW) courses and a workforce review in Northern Ireland prompted a review of admissions, to ensure recruitment of applicants with strong core values. Calls for values-based recruitment underpinned this research. A values-based psychological screening tool (SWM) was designed and successfully tested using student social workers.
Aims: This study evaluates this screening tool modified slightly for delivery to SW applicants, Social Work Match 2023 (SWM23) and contrasts it with the Prerecorded Video Interviews (PVI) introduced during COVID-19. This study evaluates the SWM23 test as a selection method for admission to SW programmes in Northern Ireland by examining the following:
(a) the quality of score data produced by the cohort,
(b) its effectiveness in distinguishing between applicants
(c) it’s psychometric characteristics and
(d) comparing these SWM23 characteristics with those of the Prerecorded Virtual Interview (PVI).
The study will also
(f) Use Qualtrics survey software to obtain and analyse feedback on SWM23 by applicants to Ulster University, Queen’s University and Open University.
Methods
Applicants who applied for the 3-year undergraduate route (UGR) or the 2-year relevant graduate route (RGR) were invited to participate in the study after completing their admissions interview. Written consent was received from 174 participants, 49 of whom (6 male, 43 female) completed SWM23. The study used a repeated measure design in which the consenting group of applicants are assessed for suitability for the NI Degree in social work courses using two different measures, the virtual interview (PVI) and a psychological test of values (SWM23). A survey collected data on applicants’ views of the test.
Findings
Analysis of data provides practical, theoretical, statistical, and qualitative reasons for concluding that SWM23 is a valid, reliable, and fair measure with good discriminatory power, standardized administration, and consistent marking. Furthermore, the PVI is a valid, reliable, and fair virtual interview with good discriminatory power, sound evaluation, and effective technical support consistent with the methodology. However, there are potential issues with unconscious bias towards certain applicants, which are not evident in SWM23. Furthermore, although independent assessors attend annual training and standardization, evidence suggests that marks awarded for responses to questions differ according to the perspective of markers, which is subsequently disguised in the overall mark awarded.
Conclusion
Both measures are valid, reliable, fair, and effective. They complement one other but measure different aspects of suitability for social work training. One measures personal values and the other key SW competencies. Using the measures in tandem would increase the amount of reliable information available to inform the selection process and minimise uncertainty about fundamental suitability.
Recommendations
The findings should be presented at relevant regional committees to demonstrate the benefits of implementing SWM23 as a form of values-based recruitment, prior to any decisions regarding implementation.
If there is a decision to implement SWM23 as a mandatory element of the selection process, universities and partnering agencies would need to agree how the additional data on applicants would be used to inform selection, alongside interview scores, academic qualifications, and any other relevant information determining suitability.
Furthermore, formal agreements should be established between the universities and Identity Exploration Limited, as the latter owns Intellectual Property rights for the SWM23 psychometric test.
Aims: This study evaluates this screening tool modified slightly for delivery to SW applicants, Social Work Match 2023 (SWM23) and contrasts it with the Prerecorded Video Interviews (PVI) introduced during COVID-19. This study evaluates the SWM23 test as a selection method for admission to SW programmes in Northern Ireland by examining the following:
(a) the quality of score data produced by the cohort,
(b) its effectiveness in distinguishing between applicants
(c) it’s psychometric characteristics and
(d) comparing these SWM23 characteristics with those of the Prerecorded Virtual Interview (PVI).
The study will also
(f) Use Qualtrics survey software to obtain and analyse feedback on SWM23 by applicants to Ulster University, Queen’s University and Open University.
Methods
Applicants who applied for the 3-year undergraduate route (UGR) or the 2-year relevant graduate route (RGR) were invited to participate in the study after completing their admissions interview. Written consent was received from 174 participants, 49 of whom (6 male, 43 female) completed SWM23. The study used a repeated measure design in which the consenting group of applicants are assessed for suitability for the NI Degree in social work courses using two different measures, the virtual interview (PVI) and a psychological test of values (SWM23). A survey collected data on applicants’ views of the test.
Findings
Analysis of data provides practical, theoretical, statistical, and qualitative reasons for concluding that SWM23 is a valid, reliable, and fair measure with good discriminatory power, standardized administration, and consistent marking. Furthermore, the PVI is a valid, reliable, and fair virtual interview with good discriminatory power, sound evaluation, and effective technical support consistent with the methodology. However, there are potential issues with unconscious bias towards certain applicants, which are not evident in SWM23. Furthermore, although independent assessors attend annual training and standardization, evidence suggests that marks awarded for responses to questions differ according to the perspective of markers, which is subsequently disguised in the overall mark awarded.
Conclusion
Both measures are valid, reliable, fair, and effective. They complement one other but measure different aspects of suitability for social work training. One measures personal values and the other key SW competencies. Using the measures in tandem would increase the amount of reliable information available to inform the selection process and minimise uncertainty about fundamental suitability.
Recommendations
The findings should be presented at relevant regional committees to demonstrate the benefits of implementing SWM23 as a form of values-based recruitment, prior to any decisions regarding implementation.
If there is a decision to implement SWM23 as a mandatory element of the selection process, universities and partnering agencies would need to agree how the additional data on applicants would be used to inform selection, alongside interview scores, academic qualifications, and any other relevant information determining suitability.
Furthermore, formal agreements should be established between the universities and Identity Exploration Limited, as the latter owns Intellectual Property rights for the SWM23 psychometric test.
| Original language | English |
|---|---|
| Number of pages | 52 |
| Publication status | Published - 17 Jan 2024 |