Provider Monitoring Report. City and Guilds

Transcription

1 Provider Monitoring Report City and Guilds 22 May 2017 to 3 August 2017

2 Contents 1 Background Scope Provider Monitoring Report Timeline Summary of Provider Monitoring Issues and Recommendations Risk Rating of Issues 6 2 Good Practice, Issues and Recommendations Good Practice Issues Recommendations 14 3 Acceptance of Provider Monitoring Findings 16 Scottish Qualifications Authority 2015 (Version 7)

3 1 Background Five providers were monitored between 22 May 2017 and 3 August Scope SQA Accreditation carries out quality assurance activity in line with its Quality Assurance of Approved Awarding Bodies Policy. This involves monitoring a sample of the awarding body s approved providers or assessment sites. Provider monitoring visits will be conducted in a consistent manner within and between providers. The aim of monitoring is to: ensure the awarding body s compliance with SQA Accreditation s regulatory requirements confirm that quality assurance arrangements are being conducted by the awarding body in accordance with its prescribed arrangements ensure that quality assurance arrangements are being conducted in a consistent manner, within and between providers ensure that providers are receiving the appropriate guidance, support and documentation from the awarding body in order to facilitate a high standard of qualification delivery inform future audit and monitoring activity for the awarding body All Principles may be included within the scope of the provider monitoring activity. Awarding body documentation considered for review includes all documents banked on the awarding body s SharePoint Place at the time of provider monitoring and information supplied by providers to support provider monitoring activity. Restricted or commercially sensitive information gathered during SQA Accreditation s quality assurance activities is treated in the strictest confidence. SQA Accreditation provider monitoring reports are written by exception focusing only on those areas where corrective action is required or recommended. Consequently, this approach to provider monitoring reporting will not detail areas where compliance or good practice was identified by SQA Accreditation. Scottish Qualifications Authority 2015 (Version 7) 1

4 1.2 Provider Monitoring Report Timescales City and Guilds provider monitoring dates: 22 May 2017 to 3 August 2017 Provider Monitoring Report approved by Accreditation Co-ordination Group on: 4 October 2017 Provider Monitoring Report to be signed by City and Guilds: 21 November 2017 Action Plan to be ed to regulation@sqa.org.uk by City and Guilds 21 November 2017 The process will apply in relation to the timescales specified above: The awarding body will be sent two signed copies of the Provider Monitoring Report by post. The awarding body must sign both copies of the Provider Monitoring Report and return one by post to SQA Accreditation in accordance with the timescale specified above. The awarding body will also be ed a copy of the Provider Monitoring Report (for information only) and an electronic copy of the Action Plan. The awarding body must complete and return the Action Plan in accordance with the timescale specified above and this in Microsoft Word format to regulation@sqa.org.uk. SQA Accreditation will confirm when the Action Plan is appropriate to address the Issues and present it to Accreditation Co-ordination Group (ACG) for approval. Following approval by ACG, the awarding body will be sent two signed copies of the approved Action Plan by post. The awarding body must sign both copies of the Action Plan and return one by post to SQA Accreditation. The findings of this Provider Monitoring Report and the associated Action Plan will be published on SQA Accreditation s website following signed agreement. SQA Accreditation will continually monitor progress towards completion of the proposed actions identified in the Action Plan and update the awarding body s Quality Enhancement Rating as appropriate. Scottish Qualifications Authority 2015 (Version 7) 2

5 1.3 Summary of Provider Monitoring Issues and Recommendations An Issue has been recorded where evidence shows that the awarding body is not compliant with SQA Accreditation s regulatory requirements. The awarding body must address the Issues and specify corrective and preventative measures to address them through its Action Plan. The Action Plan is ed to City and Guilds as a separate document to the Provider Monitoring Report and must be submitted to SQA Accreditation in accordance with the timescale specified in 1.2. As a result of the provider monitoring activity, seven Issues have been recorded and three Recommendations have been noted. Issue Detail of Issue recorded Risk rating 1. Principle 5 & 6 City and Guilds has not provided clear information on its procedures, products and services to ensure that they are accurate and appropriate to SQA accredited qualifications. 2. Principles 6 & 10 Provider 1 was unable to evidence valid training certificates for its 22 Workplace Assessors to ensure compliance with The Habia Qualification Assessment Strategy for awarding bodies for both the SVQ 2 in Hairdressing at SCQF Level 5 and SVQ 3 in Hairdressing at SCQF Level Principles 10 & 15 Provider 4 was unable to provide the Accreditation Auditor with any evidence of its 30 Assessors and six Internal Quality Assurers either holding or working towards appropriate Assessor and Internal Verifier qualifications. It was noted that the provider was also unaware of this requirement specified within both the Assessment Strategy for Scottish Vocational Qualifications (SVQs) in the Landbased and Environmental Sector (December 2015) and City and Guilds Qualification Handbook SVQ 2 Production Horticulture SCQF Level 5. Medium Medium Very High The Accreditation Auditor is concerned that candidates from Provider 4 have achieved and been certificated for the SVQ 2 in Production Horticulture at SCQF Level 5 without appropriately qualified Assessors and Internal Quality Assurers being in place. Scottish Qualifications Authority 2015 (Version 7) 3

6 4. Principles 10, 14 & 15 The Training delivery and assessment strategy for Scottish alcohol licensing qualifications (March 2013), states that At the discretion of the awarding organisations, courses may be delivered in a language other than English, however, all examination questions and answers must be in English. Provider 5 asked another candidate on the same course who spoke the same language as the candidate to translate the questions written in English for the candidate who was unable to read English. Further, it is also unlikely that the candidate would be able to read the Scottish Certificate for Personal Licence Holders Handbook; pre-course study material sent to candidates a week before commencing their course. As the Invigilator was not proficient in the language in question, he would not have been able to ensure that both candidates were not colluding, with one candidate translating the questions and possibly providing answers to the other candidate. This puts into question the validity of the qualification certificate for the candidate who could not read the English MCQ exam paper. High 5. Principle 11 The centre-devised Alternative Assessment Arrangements Policy, Version 1 (June 2014), given to the Accreditation Auditor by Provider 1, makes no reference to City and Guilds Access arrangements When and how applications need to be made to City and Guilds (October 2016) or to City and Guilds awarding body. However, it wrongly stated that requests for access arrangements should be sent to another awarding body. Low This is erroneous: with the potential risk that a competitor awarding body receives request for access arrangements for City and Guilds candidates. Scottish Qualifications Authority 2015 (Version 7) 4

7 6. Principle 12 Both centre-devised complaint procedures of Providers 1 & 3 incorrectly informed candidates of the correct rights of further escalation. 7. Principle 14 Providers 1, 2, 3 and 5 are not complying with either City and Guilds Our Quality Assurance Requirements, Version 5.2 (July 2016) or Centre Manual Version 7.1 (June 2016). Low Medium Whilst reviewing both the centre-devised Centre Malpractice Policy and Candidate Malpractice Policy for Provider 1, the Accreditation Auditor noted that both of these policies reference another competitor awarding body but do not make reference to City and Guilds awarding body or its processes for malpractice or maladministration. This is erroneous, with the potential risk that a competitor awarding body is sent information about a City and Guilds candidate and a centre malpractice issue that is not relevant to the other awarding body. Scottish Qualifications Authority 2015 (Version 7) 5

8 A Recommendation has been noted where SQA Accreditation considers there is potential for improvement. The awarding body is advised to address any Recommendations noted as good practice. However, measures to correct or prevent these are not mandatory and therefore do not form part of the Action Plan. Recommendation Detail of Recommendation noted 1. Principle 6 City and Guilds may wish to recommend the use of version control as an area of good practice to its providers who are approved to deliver SQA accredited qualifications. 2. Principles 13 & 14 City and Guilds may wish to remind its External Quality Assurers to ensure that the centre-devised appeals procedures of its SQA accredited providers should note that candidates only have the right to escalate a complaint to SQA Accreditation the Qualifications Regulator: we cannot overturn an academic decision from an appeal, we would only review the process that the awarding body followed. 3. Principle 15 City and Guilds should remind its providers who are approved to deliver SQA accredited qualifications and its External Quality Assurers that it is good practice for them to review candidate portfolio/evidence so that they can claim any completed units for candidates who leave prior to completing the full SVQ. 1.4 Risk Rating of Issues SQA Accreditation assigns a rating to each Issue recorded depending on the impact on or risk to the awarding body s operations, its SQA accredited qualifications and/or the learner. Issues recorded during provider monitoring will count towards City and Guilds Quality Enhancement Rating which will, in turn, contribute towards future quality assurance activity. Further detail on how the Quality Enhancement Rating is calculated can be found on the SQA Accreditation website: ment_rating Scottish Qualifications Authority 2015 (Version 7) 6

9 2 Good Practice, Issues and Recommendations The following sections detail: good practice noted by providers Issues recorded and Recommendations noted against SQA Accreditation s regulatory requirements 2.1 Good Practice The following areas of good practice were noted by providers: Provider 1 highlighted the: professional reputation held by City and Guilds and its user-friendly systems and procedures Provider 2 highlighted the: robustness of its external quality assurance process good practice of City and Guilds Glasgow customer service staff in organising to meet with providers face to face Provider 3 highlighted that: processes are straightforward and easy to work with Provider 4 highlighted that: City and Guilds is very business focused and interested in making recommendations to improve both the business and the people within the organisation the External Quality Assurer is helpful and approachable Provider 5 highlighted: how well known and respected City and Guilds is within the Hospitality sector. 2.2 Issues Regulatory Principle 5. The awarding body shall provide clear information on its procedures, products and services and ensure that they are accurate and appropriate to SQA accredited qualifications. Regulatory Principle 6. The awarding body and its providers shall maintain accurate documents, records and data. In reviewing the awarding body documentation uploaded on SharePoint and the City and Guilds website prior to commencing the sampling of City and Guilds provider monitoring visits, the Accreditation Auditor noted that both contained incorrect and out of date information within the following documents: Scottish Qualifications Authority 2015 (Version 7) 7

10 City and Guilds SQA Approved Centre Details uploaded to SharePoint in March 2017 contained incorrect contact details for one of its providers, who had been selected for a provider monitoring visit. The Accreditation Auditor further requested up to date contact details for this provider from City and Guilds, which were provided to the Accreditation Auditor on 14 June However, unfortunately the new information provided was still incorrect resulting in another City and Guilds provider having to be selected for a provider monitoring visit. City and Guilds How and when to apply for access arrangements Version 2.0, September 2016 states that City and Guilds is regulated by four regulators across the UK who set requirements for reasonable adjustments and special consideration. The requirements are: SQA Accreditation Regulatory Principles (2014), Principle 13. The Accreditation Auditor recognised that City and Guilds have amended the reference to Regulatory Principles (2014). However, they have not amend the change from Regulatory Principle 13 to Regulatory Principle 11: The awarding body shall ensure that its qualifications and their assessment are inclusive and accessible to learners. City and Guilds Qualification Handbooks: Scottish Certificate for Personal Licence Holders SVQ3 in Business and Administration at SCQF Level 6 SVQ 3 Social Services and Healthcare at SCQF Level 7 SVQ 3 in Management at SCQF Level 7 SVQ 4 in Management at SCQF Level 9 SVQ 5 in Management at SCQF Level 11 SVQ 2 Production Horticulture SCQF Level 6 All of which state that Our Quality Assurance Requirements encompass all of the relevant requirements of the key regulatory documents such as: SQA Awarding Body Criteria (2007). However, this document was originally replaced by SQA Accreditation Regulatory Principles in 2011 then by SQA Accreditation s Regulatory Principles (2014) version 1, April 2014 and finally by SQA Accreditation s Regulatory Principles (2014) version 2, 1 December Therefore, City and Guilds has not provided clear information on its procedures, products and services to ensure that they are accurate and appropriate to SQA accredited qualifications. This has been recorded as Issue 1. Regulatory Principle 6. The awarding body and its providers shall maintain accurate documents, records and data. Regulatory Principle 10. The awarding body shall ensure that it has the necessary arrangements and resources for the effective delivery, assessment and quality assurance of SQA accredited qualifications. Scottish Qualifications Authority 2015 (Version 7) 8

11 The Habia Qualification Assessment Strategy for awarding bodies for the SVQ 2 in Hairdressing at SCQF Level 5 and SVQ 3 in Hairdressing at SCQF Level 6 both state that The Assessor must hold, or be working towards; a valid assessor s qualification based on LSIS, formally LLUK, Learning and Development National Occupational Standards (2010): Learning and Development Unit 9D Assess workplace competence using direct methods; or Learning and Development Unit 9DI Assess workplace competences using direct and indirect methods; or Level 3 Certificate in Assessing Vocational Achievement (QCF); or Level 3 Award in Assessing Competence in the Work Environment (QCF) or hold one of the following: Assess candidates using a range of methods; or D32/33 Assess candidate performance, using differing sources of evidence. Provider 1 was unable to provide the Accreditation Auditor with training certificates for its 22 Workplace Assessors as they did not retain copies of Workplace Assessor training certificates for its records. Therefore, Provider 1 was unable to evidence valid training certificates for its 22 Workplace Assessors to ensure compliance with The Habia Qualification Assessment Strategy for awarding bodies for both the SVQ 2 in Hairdressing at SCQF Level 5 and SVQ 3 in Hairdressing at SCQF Level 6. This has been recorded as Issue 2. Regulatory Principle 10. The awarding body shall ensure that it has the necessary arrangements and resources for the effective delivery, assessment and quality assurance of SQA accredited qualifications. Regulatory Principle 15. The awarding body and its providers shall have effective, reliable and secure systems for the registration and certification of learners. Provider 4 has 30 Assessors and six Internal Quality Assurers who at the time of the provider monitoring visit were working with candidates undertaking the SVQ 2 in Production Horticulture at SCQF level 5. Both the Assessment Strategy for Scottish Vocational Qualifications (SVQs) in the Landbased and Environmental Sector (December 2015) and City and Guilds Qualification Handbook SVQ 2 Production Horticulture SCQF Level 5 state that all Assessors and Internal Verifiers must hold or be working towards the appropriate qualifications for Assessors and Internal Verifiers which are outlined at: er_competence_final.pdf Provider 4 was unable to provide the Accreditation Auditor with any evidence of its assessment team either holding or working towards appropriate Assessor and Internal Scottish Qualifications Authority 2015 (Version 7) 9

12 Verifier qualifications and noted that the provider was also unaware of this requirement specified within both the Assessment Strategy for Scottish Vocational Qualifications (SVQs) in the Land-based and Environmental Sector (December 2015) and City and Guilds Qualification Handbook SVQ 2 Production Horticulture SCQF Level 5. Provider 4 went on to explain to the Accreditation Auditor that its assessment team had received training delivered directly from City and Guilds. However, Provider 4 was unable to evidence how this training met the requirements of both the Assessment Strategy for Scottish Vocational Qualifications (SVQs) in the Land-based and Sector (December 2015) and City and Guilds Qualification Handbook SVQ 2 Production Horticulture SCQF Level 5. The Assessment Strategy for Scottish Vocational Qualifications (SVQs) in the Land-based and Environmental Sector (December 2015) further states that Approved centres will be required to provide the awarding body/external verifier (as appropriate) with current evidence of how each assessor meets this requirement; for example, relevant qualifications. Provider 4 gave the Accreditation Auditor a copy of The Centre Activity Report completed by the City and Guilds External Quality Assurer on the 2 December It stated that All are appropriately competent in the subject area, through a combination of experience and specific qualifications. Whilst they are not holding assessor qualifications, all have been through City and Guilds training delivered directly to the provider and have clearly been able to fulfil the requirements of the assessment strategy. There were approximately 63 current registrations and 43 certifications within the past 12 months but this is an issue that may have repercussions as far back as 11 February 2014, being the point at which the centre was first approved. Following discussion with the Regulation Manager, it was decided to raise the matter immediately with City and Guilds given the potential number of candidates affected. City and Guilds has responded that it believes that it delivered the City and Guilds Level 3 Certificate in Assessing Vocational Achievement (TAQA course) which City and Guilds states has been mapped against the Assessor and Internal Verifier competence criteria. However, no evidence has been provided in respect of approval of this mapping, by either SQA Accreditation or Sector Skills Council (SSC). Whilst the Accreditation Auditor recognises the steps taken by the awarding body to mitigate the impact on both candidates and Provider 4, the lack of qualified Assessors and Internal Quality Assurers remains an issue. Therefore, the Accreditation Auditor is concerned that candidates from Provider 4 have achieved and been certificated for the SVQ 2 in Production Horticulture at SCQF Level 5 without appropriately qualified Assessors and Internal Quality Assurers being in place. This has been recorded as Issue 3. Scottish Qualifications Authority 2015 (Version 7) 10

13 Regulatory Principle 10. The awarding body shall ensure that it has the necessary arrangements and resources for the effective delivery, assessment and quality assurance of SQA accredited qualifications. Regulatory Principle 14. The awarding body and its providers shall ensure that it has safeguards to prevent and manage cases of malpractice and maladministration. Regulatory Principle 15. The awarding body and its providers shall have effective, reliable and secure systems for the registration and certification of learners. The Training delivery and assessment strategy for Scottish alcohol licensing qualifications (March 2013), states that At the discretion of the awarding organisations, courses may be delivered in a language other than English, however, all examination questions and answers must be in English. During discussion with Provider 5 the Accreditation Auditor was informed that it had a candidate who undertook the Scottish Certificate for Personal Licence Holders exam on 23 July 2015, who spoke English, though was unable to read the multiple choice questions (MCQ) which were written in English. Therefore, Provider 5 asked another candidate on the same course who spoke the same language as the candidate to translate the questions written in English for the candidate who was unable to read English. The other candidate who had translated for this candidate had completed the same MCQ exam earlier on the same day. The MCQ question papers completed by both candidates were marked externally at the time by City and Guilds awarding body. Provider 5 stated to the Accreditation Auditor that the City and Guilds External Quality Assurer had been informed of the situation. The Centre Activity Report dated 2 December 2016 included both candidates within the remote sample undertaken by the City and Guilds External Quality Assurer. This stated for both candidates that Old course record form available for the candidate he/she completed the qualification before the centre was advised on the candidate data to be kept on demand exam completed. (sic). However, there was no mention within the Centre Activity Report of one candidate translating a written MCQ exam paper for another candidate on the same course who could not read English. Further, it is also unlikely that the candidate would be able to read the Scottish Certificate for Personal Licence Holders Handbook; pre-course study material sent to candidates a week before commencing their course. As the Invigilator was not proficient in the language in question, he would not have been able to ensure that both candidates were not colluding, with one candidate translating the questions and possibly providing answers to the other candidate. Therefore, putting into question the validity of the qualification certificate for the candidate who could not read the English MCQ exam paper. This has been recorded as Issue 4. Scottish Qualifications Authority 2015 (Version 7) 11

14 Regulatory Principle 11. The awarding body shall ensure that its qualifications and their assessment are inclusive and accessible to learners. The centre-devised Alternative Assessment Arrangements Policy, Version 1 (June 2014), given to the Accreditation Auditor by Provider 1, makes no reference to City and Guilds Access arrangements When and how applications need to be made to City and Guilds (October 2016) or to City and Guilds awarding body. However, it wrongly stated that requests for access arrangements should be sent to another awarding body. This is erroneous: with the potential risk of a competitor awarding body receiving requests for access arrangements for City and Guilds candidates. This has been recorded as Issue 5. Regulatory Principle 12. The awarding body and its providers shall have open and transparent systems to manage complaints. City and Guilds Centre Manual Version 7.1 (June 2016), states that Centres are required, as a condition of approval, to agree and operate a complaints procedure, which learners, assessors/tutors, IQAs and employers can use in the event that they wish to challenge an appropriate aspect of the centre s operation. Centres have a responsibility to inform learners of their complaints policy. EQAs will request to see evidence of the centre complaints procedure as part of their quality assurance monitoring. Both Providers 1 & 3 mentioned to the Accreditation Auditor that the providers quality assurance procedures had not been reviewed as part of City and Guilds quality assurance monitoring in the past three years. Provider 1 provided the Accreditation Auditor with its centre-devised Complaints/Grievance Procedure, Version 1 (June 2016). However, this does not inform candidates that if they are still unhappy with the outcome of their complaint they have the right of further escalation to both the awarding body and SQA Accreditation, the Qualifications Regulator. The Accreditation Auditor was given a centre-devised Candidate Complaint Procedure by Provider 3. However, this does not inform candidates that if they are still unhappy with the outcome of their complaint they have the right of further escalation to SQA Accreditation, the Qualifications Regulator. Therefore, both centre-devised complaint procedures of Providers 1 & 3 incorrectly informed candidates of the correct rights of further escalation. This has been recorded as Issue 6. Scottish Qualifications Authority 2015 (Version 7) 12

15 Regulatory Principle 14. The awarding body and its providers shall ensure that it has safeguards to prevent and manage cases of malpractice and maladministration. City and Guilds Our Quality Assurance Requirements, Version 5.2 (July 2016), and City and Guilds Centre Manual Version 7.1 (June 2016), both state that If a centre discovers or suspects malpractice they are required to report it to the Investigation & Compliance team within 10 working days and prior to the commencement of any internal investigation. The Accreditation Auditor reviewed the centre-devised Malpractice and Maladministration Policies at Providers 1, 2, 3 and 5. These failed to state that if a centre discovers or suspects malpractice they are required to report it to the Investigation & Compliance team within 10 working days and prior to the commencement of any internal investigation. Therefore, Providers 1, 2, 3 and 5 are not compliant with both City and Guilds Our Quality Assurance Requirements, Version 5.2 (July 2016) and Centre Manual Version 7.1 (June 2016). Whilst reviewing both the centre-devised Centre Malpractice Policy and the Candidate Malpractice Policy for Provider 1 the Accreditation Auditor noted that both of these policies reference another competitor awarding body but do not make reference to City and Guilds awarding body or its processes for malpractice or maladministration. This is erroneous, with the potential risk that a competitor awarding body is sent information about a City and Guilds candidate and a centre malpractice issue that is not relevant to the other awarding body. This has been recorded as Issue 7. Scottish Qualifications Authority 2015 (Version 7) 13

16 2.3 Recommendations Regulatory Principle 6. The awarding body and its providers shall maintain accurate documents, records and data. The Accreditation Auditor noted that none of the centre-devised documentation provided by Providers 2, 3 & 5 contained any version control. Therefore City and Guilds may wish to recommend the use of version control as an area of good practice to its providers approved to deliver SQA accredited qualifications. This has been noted as Recommendation 1. Regulatory Principle 13. The awarding body and its providers shall have clear, fair and equitable procedures to manage appeals. Regulatory Principle 14. The awarding body and its providers shall ensure that it has safeguards to prevent and manage cases of malpractice and maladministration. Provider 1 gave the Accreditation Auditor its centre-devised Appeals Policy which states that If the candidate is dissatisfied with the written judgement and the judgement relates to an SVQ, the candidate may appeal to the Qualifications Regulator who will investigate the matter and provide a final written judgement. The Accreditation Auditor also reviewed the centre-devised Candidate Malpractice Policy for Provider 1 which states that If the candidate is dissatisfied with the Malpractice Panel s decision the candidate may appeal in writing to the Qualifications Regulator as detailed in our Appeals Procedure. The centre-devised Appeals Procedure from Provider 3 states that If necessary an appeal can be made to the SQA Accreditation if the candidate feels that the provider/awarding body has not dealt with the appeal appropriately. However, SQA Accreditation, the Qualifications Regulator, cannot overturn an academic decision from an appeal; a candidate would only have the right to escalate a complaint to SQA Accreditation, the Qualifications Regulator. City and Guilds may wish to remind its External Quality Assurers to ensure the centredevised appeals procedures of its SQA accredited providers should note that candidates would only have the right to escalate a complaint to SQA Accreditation the Qualifications Regulator as we cannot overturn an academic decision from an appeal. This has been noted as Recommendation 2. Scottish Qualifications Authority 2015 (Version 7) 14

17 Regulatory Principle 15. The awarding body and its providers shall have effective, reliable and secure systems for the registration and certification of learners. Provider 4 informed the Accreditation Auditor that if a candidate leaves before completing their SVQ 2 in Production Horticulture SCQF level 5 the assessment team currently would not check the candidate s portfolio/evidence to claim for any units that the candidate may have completed prior to leaving. City and Guilds should remind both its providers approved to deliver SQA accredited qualifications and its External Quality Assurers that it is good practice for them to review candidate portfolio/evidence so that they can claim any completed units for candidates who leave prior to completing the full SVQ. This has been noted as Recommendation 3. Scottish Qualifications Authority 2015 (Version 7) 15

18 3 Acceptance of Provider Monitoring Findings For and on behalf of City and Guilds: Print name... For and on behalf of SQA Accreditation: Print name... Signature... Signature... Designation... Designation... Date... Date... Scottish Qualifications Authority 2015 (Version 7) 16