BLOG POST by Robert Playfair Going through a UKVI audit on a pre-sessional course 'UK Visas and Immigration' (UKVI) is part of the Home Office department of government. According to their website, their role is to make "millions of decisions every year about who has the right to visit or stay in the country, with a firm emphasis on national security and a culture of customer satisfaction for people who come here legally." This blog post is about when the world of EAP practitioners in the UK collides with the Home Office, in what is known as a 'UKVI audits'.* *read more by clicking on the 'Read More' below Since 2010 and the introduction of 'Secure English Language Tests' (SELTs), developers of these language tests have taken on responsibilities of immigration control, or ‘border work’ (Harding, Brunfaut & Unger, 2020). As pre-sessional tests scores in the UK have the same decision-making value as a SELT (students progress / do not progress on to their degree courses), this immigration control role has also become part of the job of EAP Practitioners involved in in-house pre-sessional assessment development, alongside the responsibility of preparing students for their future studies. For these people there is a tension between developing assessments which best prepare students for higher education study and providing evidence of language competency for UKVI audits. This concern predates SELTs, with Cyril Weir (2005) noting the impact this has on test design: “The increased expectation that providers of educational services should be made accountable to external bodies for the impact of their work has been a powerful driving force behind [the emphasis on summative evaluation]. It has encouraged a swing from viewing tests as instruments for assisting in the development and improvement of student language ability to treating them as indicators of performance for outside agencies” (Weir, 2005: 39) It has also been explored in other contexts, e.g., an EAP unit in Hong Kong (Bruce & Hamp-Lyons, 2015). BALEAP has developed guidelines for understanding existing SELT tests however there is relatively little guidance on developing 'UKVI compliant' in-house assessments. Perhaps as a result, this is a common point of discussion for those with in-house assessment development responsibilities: a recent search of the past two years of the BALEAP mailing list shows over 200 messages containing 'UKVI'. This blog post collects the current government guidance that is available and offers some snapshots of narratives from BALEAP colleagues who have undergone UKVI audits over the past few years. Government guidance on in-house language tests 'Tier-4' visas have recently changed their name to 'Student Route' visas. However, the guidance from the Home Office on assessing English language competence for in-house EAP departments has not changed (as of December 2020). The main document which addresses this is the 'Student Sponsor Guidance Document 2: Sponsorship Duties' (version 12/2020, available here). In particular sections 5.10-5.12 'Students studying at degree level and above on the Student route', For example, " 5.10 a. If you are an HEP [Higher Education Provider] with a track record of compliance, we will allow you to choose your own way to assess it... However, you must ensure they are proficient to level B2 in each of the four components (speaking, listening, reading and writing), unless they are exempt from being proficient in a component because of a disability." (p.29) and "5.11 You must take all reasonable steps to ensure that you are satisfied through your assessment that the applicant meets the language competence requirements. For example, you could interview students. If you have doubts about any documents then you should verify them with the appropriate body." (p.30) Another source of information is the Government webpage, 'Prove your English language abilities with a secure English language test (SELT)' which has recently (October 2020) added guidance about integrated testing (i.e., tests generating scores of more than one skill) for SELTS which could be referred to when justifying scores from similar tests developed by in-house EAP teams: "Where 2 or more components (reading, writing, speaking and listening) of a test are examined and awarded together, for example a combined exam and certificate for reading and writing skills, you must show that you achieved the required scores in all the relevant components during a single sitting of that examination, unless you were exempt from sitting a component on the basis of a disability." UKVI audit case studies TAFSIG is not yet in a position to either challenge this guidance or offer advice about interpreting it for in-house assessment development. For now, we would like to contribute the following anonymised case studies as a window into the UKVI audit experience for people working in high stakes in-house language testing in universities. If you have an experience related to this topic you would like to share, please get in touch via our email: [email protected] Case study 1 We were audited 2 or 3 years ago. UKVI audited the whole university, not just the pre-sessional course or even just our language centre - it was admissions for all international students. For our summer pre-sessional course, we don't have an end of course test, and we don't test the 4 skills separately - we have integrated task-based assessments, e.g. the extended writing task assesses reading and writing together. However, on the marking criteria reading is a specific category, and it is a must-pass category, with the pass being CEFR B2. Overall, of the 4 skills required by UKVI, each has at least 1 must-pass B2 criterion on the assessments, which is how we demonstrate that students have met UKVI requirements. We also have a detailed assessment specification document which explains how everything works. Case study 2 The UKVI visited us in 2016, with very little advance notice given to our department (around 2 weeks). This was about 6 weeks before the main presessional courses began. They were not there just for the presessional but for all aspects of managing Tier 4 students, including our relevant UK-based franchised courses, and were particularly exercised by record-keeping. In advance of the visit, I was asked to provide documentation relating to the content, assessment strategy, and outcomes from the previous presessionals on my watch. I think there were 3 inspectors in total, and they stayed for two days. There were two big meetings and a number of side meetings with individuals or small groups. I was not required to attend a side-meeting. The first big meeting was about administration and record-keeping across the University and I was not asked to attend. The second meeting was more about course management and assessment, and involved management and academic representatives from each relevant course. It became clear that the inspectors had only a sketchy knowledge of pedagogical principles and issues. They were very keen on separate assessments for each of the four skills, or at least a distinct CEFR-related mark being awarded for each. However, we on the presessional favoured integrative assessments, so I faced some questioning about that. Our plans for the summer courses were well advanced by then, so I was loth to change things that had already been settled. However, we did meet them halfway in the end, by hastily introducing a specific listening test, which turned out to be a disaster but hey ho. I understand that my successors got pushed further in the direction of separate assessments. The inspectors were also very interested in the credibility of the courses, insofar as having clear criteria for success and failure, and progression to degree courses. I had no problems in this regard, but the representative from one of the private-sector franchised courses was given a very hard time as they had *never* failed anyone on academic grounds on their relevant courses. In general, the inspectors seemed to appreciate colleagues who had done their homework and were well-briefed. You need to have the facts at your fingertips. A few weeks later, we learned that the University had passed the inspection with no serious concerns. Case study 3 The audit we had this year involved about 40-45 mins with a UKVI panel of three auditors and they looked at the assessments for both our Presessional course and also from our internal Progress Test. I wasn't the person directly involved but I was in the 'green room' for the debrief. Their main concerns appeared to be the security and secure storage of test materials and the recording of spoken presentations rather than the content of assignments. Attendance monitoring also seemed to be a key focus. One of the auditors did have a background in English language teaching and commented favourably on the range of test items we had written, however I think that was incidental. We are still waiting to have sight of the audit for more detail. Case study 4 This relates to late 2019 when we went through a successful UKVI inspection of both our Pre-Sessional and Foundation programmes: We carried out some informal research prior to our inspection to find out the experience of other HEIs, although it soon became clear that experiences were very varied and some claimed the rigor and approach depended on the inspectors involved. Nevertheless, tapping into the experiences of others was really useful. UKVI inspected all our relevant programmes including Foundation, Pre-Masters and Pre-Sessional. I worked hard to package together a suite of documents for the inspectors which explained our language tests and provided a rationale and defence of the robustness of our assessment system. This package (which included samples of tests, marking criteria, CEFR alignment, test specs and validity report) was given to the inspectors in good time before the actual inspection. When it came to the face-to-face interrogation(!), the inspectors focussed much more on the Pre-Sessional than our other programmes. We never found out why this was, but some of our Pre-Sessional assessment is coursework-based and not carried out in exam conditions, so maybe this was why they honed in on it, or maybe it was due to student numbers going through each programme. Whatever the reasons, the inspectors were friendly and interested to learn about our programmes, but although manageable, that interrogation of the Pre-Sessional was quite a grilling and covered various aspects of the testing, both practical and pedagogical. As well as being prepared from a testing perspective and being ready to defend the validity and reliability of assessment instruments, it was also essential that the administration of our assessment procedures was in order. The UKVI team requires a selection of student samples; some of these are to be submitted to the inspectors prior to their visit but there is also a chunk they ask for on the day of their visit. These need to be readily accessible and so administration staff need to be well involved in the preparation of any inspection. Case study 5 Experience of UKVI audit 2021: Reflections of an Assessment Lead We were due to be audited around the time Covid-19 became a global pandemic, which meant that the intended visit was postponed. When it did happen, we were given very limited advanced warning at one of our busiest times of the year, but much of the preliminary work had already been done in preparation for the original visit. This meant it was relatively straightforward to locate and collate all the necessary documentation and pull everything together for the visit. The audit was conducted on campus over two consecutive days with interviews between staff and the chief auditor undertaken online. Prior to the audit, we had contact with him through our Student Immigration Compliance Manager, who (eventually) had it confirmed that it was not necessary to either test or report each of the four skills in isolation, although it was not easy to get this conformation! When it did come it was very encouraging news as the previous Director of the Language Centre where I work had instigated the implementation of a discrete listening test and we had recently taken the decision to scrap it again under the current Director. This because, as said in other case studies, discrete testing is very much at odds with our pedagogy, principles and approach to language teaching and assessment. As others have reported in their case studies, the audit was university-wide and focused on every aspect of taught and research students’ experience including recruitment, admissions, registration and right to study checks, engagement monitoring, English Language provision, work placements, study abroad, and support for those going onto post-study work. As is standard practice, the auditor requested a number of student files beforehand and a further list was provided with very short notice, which he wanted for scrutiny during his visit. The Language Centre formed an important part of the audit with administration processes and procedures being of particular interest to the auditor. We offer a broad range of provision but our substantial summer pre-sessional programmes and other in-sessional programmes such as International Foundation Year were of little interest to the auditor as those student cohorts are already arrive at B2 CERF level equivalent or above. Our pre-sessional provision where students are admitted below B2 level and going on to further study in the wider university, was scrutinised most closely and the auditor raised a number of questions about our in-house assessments in my interview with him. He started to express surprise when he was told that our assessments are integrated and that scoring for receptive skills is subsumed within the productive skills marks. However, I politely reminded him that we had received confirmation from him that this was an accepted approach from a UKVI standpoint. I was able to show him that reading and listening are clearly separated in our marking rubric with detailed descriptors and he acknowledged that ‘their’ view of assessment is seen through a very narrow lens (the testing of the four separate skills) but that ‘they’ recognise that this is not how language works, and therefore that these more traditional, rigid requirements do not fit in with an authentic language context and assessment for learning approach. He also clearly understood that EAP provision in HE goes beyond language competency and that our assessments are designed around academic literacies, skills and competencies. Before he became fully convinced in the rigour of our assessments though, we had to provide quite a lot of further documentation, evidence and written explanations so that he could be fully satisfied that students had indeed reached a level equivalent to B2 or above in English, as required for their onward studies. After providing this additional information we heard nothing further and it was not until several days (of quite anxious waiting!) after the visit that we were notified that we had successfully met every requirement of the audit with no issues or concerns. Therefore, all in all, we had a positive experience of being audited and I would encourage colleagues in EAP in other institutions to adhere to language teaching and assessment principles for authentic, integrated assessment with its positive washback effects into learning, rather than shoe-horning discrete testing in, what I believe to be, a misguided attempt to appease UKVI. References Bruce, E. and Hamp-Lyons, L. (2015) Opposing tensions of local and international standards for EAP writing programmes: Who are we assessing for? Journal of English for Academic Purposes 18:64-77. Harding, L. Brunfaut, T. and Unger, J. W. (2020) Language Testing in the ‘Hostile Environment’: The Discursive Construction of ‘Secure English Language Testing’ in the UK. Applied Linguistics 41(5):662–687. Weir, C. (2005) Language Testing and Validation: An Evidence-Based Approach. Palgrave.
Vivien Fox
7/10/2021 17:12:06
This was very interesting as we are exploring assessing our reading and listening skills with integrated skills assessment. I would be interested in networking with colleagues on developing this kind of assessment perhaps as an informal working party. I have also had an anecdotal report that UKVI are moving away from HEIs reporting 4 separate skills grades and moving to one holistic grade which might make an iSA more feasible? Is there any way to confirn that?
Rob
29/11/2021 15:40:30
Hi Vivien, I think we've since discussed this a bit the recent TAFSIG event. I think the TAFSIG piloting network could be a place to do collaborative test development / peer review. Happy to chat about this further: [email protected] . 5/9/2022 13:01:57
Really informative article, I had the opportunity to learn a lot, thank you. https://freecodezilla.net/streamit-video-streaming-wordpress-theme/ 5/9/2022 13:01:57
Really informative article, I had the opportunity to learn a lot, thank you. https://freecodezilla.net/wp-fluent-forms-pro-add-on-nulled/ 11/9/2022 23:53:33
Really informative article, I had the opportunity to learn a lot, thank you. https://kurma.website/ 12/9/2022 11:24:47
Really informative article, I had the opportunity to learn a lot, thank you. https://odemebozdurma.com/ 14/9/2022 22:11:13
Really informative article, I had the opportunity to learn a lot, thank you. https://bit.ly/site-kurma 30/9/2022 09:28:48
It's great to have this type of content. Good luck with your spirit. Thank you. https://bit.ly/site-kurma 4/10/2022 20:21:53
I think this post is useful for people. It has been very useful for me. Looking forward to the next one, thank you. https://escortnova.com/escort-ilanlari/bursa-escort/osmangazi-escort/ 5/10/2022 08:58:41
It was a post that I found very successful. Good luck to you. https://escortnova.com/escort-ilanlari/manisa-escort/salihli-escort/ 6/10/2022 12:41:32
I support your continuation of your posts. I will be happy as new posts come. Thank you. https://escortnova.com/escort-ilanlari/tokat-escort/ 7/10/2022 07:01:17
I think the content is at a successful level. It adds enough information. Thank you. https://escortnova.com/escort-ilanlari/kastamonu-escort/inebolu-escort/ 10/11/2022 18:47:59
internet Sitemizi ziyaret et: https://taksikenti.com/taksi/ 21/11/2022 12:05:00
Hemen Göz At evde para kazanmaya basla: https://sites.google.com/view/evden-ek-is/ 9/12/2022 12:38:53
instagram beğeni satın al: https://takipcialdim.com/instagram-begeni-satin-al/ 5/1/2023 12:24:17
100 tl deneme bonusu veren siteleri öğrenmek istiyorsan tıkla. Comments are closed.
|