Audit Report City & Guilds 3 April 2014 and 5 March 2015
Contents 1 Background 1 1.1 Scope 1 1.2 Audit Report and Action Plan Timescales 2 1.3 Summary of Audit Issues and Recommendations 3 1.4 Risk Rating of Issues 6 2 Detail of Audit Issues and Recommendations 7 2.1 Issues 7 2.2 Recommendations 9 3 Acceptance of Audit Findings 10 Scottish Qualifications Authority 2015 (Version 5)
1 Background This was the eleventh audit of City & Guilds since it was approved as an awarding body by SQA Accreditation in 1995. City & Guilds is a nationally recognised Awarding Body whose services include providing and delivering qualifications across a wide variety of industry sectors. City & Guilds headquarters are situated at Giltspur House, 1 Giltspur Street, London. The City & Guilds Audit was carried out over 2 days and at 2 different locations; Glasgow on 3 April 2014 and London on 5 March 2015. 1.1 Scope SQA Accreditation carries out quality assurance activity in line with its Quality Assurance of Approved Awarding Bodies Policy. This states the type and frequency of our quality assurance activities, describes our reporting procedures and indicates how the awarding body s Quality Enhancement Rating is calculated. As this was a full audit of City & Guilds, all regulatory requirements were included within the scope of the audit and covered all Regulatory Principles. Our quality assurance activities are conducted on a sampling basis and, consequently, not all aspects of the awarding body s systems, procedures and performance have been considered in this report to the same depth. The audit was designed to ensure City & Guilds complies with SQA Accreditation s regulatory requirements namely: SQA Accreditation s Regulatory Principles (2014) all Regulatory Principles Directives the awarding body s Accreditation Licence Awarding body documentation considered for review by the Audit Team includes all documents banked on awarding body s Quickr Place at the time of audit and information supplied to support audit activity. Restricted or commercially sensitive information gathered during SQA Accreditation s quality assurance activities is treated in the strictest confidence. Scottish Qualifications Authority 2015 (Version 5) 1
1.2 Audit Report and Action Plan Timescales City & Guilds audit date: 3 April 2014 & 5 March 2015 Audit Report approved by Accreditation Co-ordination Group on: 25 March 2015 Audit Report to be signed by City & Guilds: 11 May 2015 Action Plan to be e-mailed to regulation@sqa.org.uk by City & Guilds 11 May 2015 The process will apply in relation to the timescales specified above: The awarding body will be sent two signed copies of the Audit Report by post. The awarding body must sign both copies of the Audit Report and return one by post to SQA Accreditation in accordance with the timescale specified above. The awarding body will also be e-mailed a copy of the Audit Report (for information only) and an electronic copy of the Action Plan. The awarding body must complete and return the Action Plan in accordance with the timescale specified above and e-mail this in Microsoft Word format to regulation@sqa.org.uk. SQA Accreditation will confirm when the Action Plan is appropriate to address the Issues and present it to Accreditation Co-ordination Group (ACG) for approval. Following approval by ACG, the awarding body will be sent two signed copies of the approved Action Plan by post. The awarding body must sign both copies of the Action Plan and return one by post to SQA Accreditation. The findings of this Audit Report and the associated Action Plan will be published on SQA Accreditation s website following signed agreement. SQA Accreditation will continually monitor progress towards completion of the proposed actions identified in the Action Plan and update the awarding body s Quality Enhancement Rating as appropriate. Scottish Qualifications Authority 2015 (Version 5) 2
1.3 Summary of Audit Issues and Recommendations An Issue has been recorded where evidence shows that the awarding body is not compliant with SQA Accreditation s regulatory requirements. The awarding body must address the Issues and specify corrective and preventative measures to address them through its Action Plan. The Action Plan is e-mailed to City & Guilds as a separate document to the Audit Report, and must be submitted to SQA Accreditation in accordance with the timescale specified in 1.2. As a result of both audit visits and post-audit activities, four Issues have been recorded and one Recommendation has been noted. Issue 1. Regulatory Principle 4, Regulatory Principle 6 & Regulatory Principles Directive 2 Detail of Issue recorded As part of the audit process it was established that City & Guilds current data management system did not always reflect the qualification information approved by SQA Accreditation. The impact was a high number of errors in the quarterly data submission. City & Guilds must ensure that the qualification information captured in its new data management system is the same as the qualification information approved by SQA Accreditation. Risk rating Low 2. Regulatory Principles 5 & 6 The City & Guilds document, Supporting Customer Excellence Manual, (Sept 2013, V6, p39), section 5.3, refers to the duty to collect Scottish Learner Numbers and send to SQA. This is no longer a requirement. City & Guilds must ensure that documentation and any associated processes are updated to reflect this position. The City & Guilds document, Guidance on Internal Quality Assurance of Qualifications, (March 2014, pp5 and 20), does not reference the SQA Learning and Development Units qualifications for Internal Quality Assessors. City & Guilds must ensure that documentation is updated to appropriately reflect SQA Learning and Development Units. Low Scottish Qualifications Authority 2015 (Version 5) 3
The City & Guilds document, Our Quality Assurance Requirements, (October 2013, V4, pp4 and 22), refers to SQA Regulatory Principles 2011 and must be updated to reflect the introduction of SQA Regulatory Principles 2014. 3. Regulatory Principle 9 City & Guilds have introduced a new system to identify those qualifications where there is little or no uptake. Low Despite this newly introduced process, however, there are a number of SQA accredited qualifications which have had no uptake in the last two years. The Audit Team presented a list of these qualifications to relevant awarding body personnel, who acknowledged that these qualifications had not been included in recent withdrawal requests. Accordingly, City & Guilds must review the qualifications with no uptake, presented during the audit. 4. Regulatory Principle 13 The City & Guilds document, Review and Appeals (April 2013), contains inconsistencies in the appeals process in respect of escalation to SQA Accreditation. City & Guilds must ensure that the appeals process is stated consistently in its documentation with regard to regulatory escalation. Medium Scottish Qualifications Authority 2015 (Version 5) 4
A Recommendation has been noted where SQA Accreditation considers there is potential for improvement. The awarding body is advised to address any Recommendations noted as good practice. However, measures to correct or prevent these are not mandatory and therefore do not form part of the Action Plan. Recommendation 1. Regulatory Principle 10 & Regulatory Principles Directive 1 Detail of Recommendation noted In relation to the 10 week rule, City & Guilds system does not allow the generation of data which would highlight providers who were regularly trying to claim certification either before or around the 10 week deadline. The existence of such data would allow external quality assurance activities to be targeted appropriately. To assist the quality assurance process therefore, it is recommended that City & Guilds consider developing a tracking system to identify any patterns across providers with regard to the 10 week rule. Scottish Qualifications Authority 2015 (Version 5) 5
1.4 Risk Rating of Issues SQA Accreditation assigns a rating to each Issue recorded, depending on the impact on or risk to the awarding body s operations, its SQA accredited qualifications and/or the learner. Issues recorded during the audit will count towards City & Guilds Quality Enhancement Rating which will, in turn, contribute towards future quality assurance activity. Further detail on how the Quality Enhancement Rating is calculated can be found on the SQA Accreditation website: http://accreditation.sqa.org.uk/accreditation/regulation/quality_assurance/quality_enhance ment_rating Scottish Qualifications Authority 2015 (Version 5) 6
2 Detail of Audit Issues and Recommendations The following sections detail Issues recorded and Recommendations noted against SQA Accreditation s regulatory requirements. 2.1 Issues Regulatory Principle 4. The awarding body shall continually review the effectiveness of its business services, systems, policies and processes. Regulatory Principle 6. The awarding body and its providers shall maintain accurate documents, records and data. and Regulatory Principles Directive 2 Data Submissions. As part of the audit process, it was established that the information contained in the City & Guilds current data management system did not always reflect the qualification information approved by SQA Accreditation. This was highlighted when comparing the quarterly data submissions from City & Guilds and the data held by SQA Accreditation. It was agreed by City & Guilds during the audit that the reason for the disparity could in some part be explained by the data management processes being used and the number of occasions when staff are required to input or upload data from different systems. City & Guilds has recognised that there is scope to minimise the number of data errors and is looking to introduce a Learning and Content Management System (LCMS) which will reduce the need for unnecessary data transfer and ultimately minimise the number of errors. However, there is no process in the new LCMS to ensure that the original data captured accurately reflects the qualification information approved by SQA Accreditation. City & Guilds must ensure that the qualification information captured is the same as the qualification information approved by SQA Accreditation. It is anticipated that this will further reduce the opportunity for errors and improve the quality of quarterly data submissions. This has been recorded as Issue 1. Regulatory Principle 5. The awarding body shall provide clear information on its procedures, products and services and ensure that they are accurate and appropriate to SQA accredited qualifications. Regulatory Principle 6. The awarding body and its providers shall maintain accurate documents, records and data. The City & Guilds document, Supporting Customer Excellence Manual (Sept 2013, V6, p39), section 5.3, states that City & Guilds has a duty to ensure that Scottish Learner Numbers are collated and sent to SQA. As this is no longer a requirement, the document and any associated processes should be updated to reflect this position. Scottish Qualifications Authority 2015 (Version 5) 7
The City & Guilds document, Guidance on Internal Quality Assurance of Qualifications, (March 2014, pp5 and 20), refers to type of qualifications required to be held by Internal Quality Assessors but makes no reference to the SQA Learning and Development Units within the qualifications detailed. To ensure parity of information, relevant guidance must be updated accordingly. The City & Guilds document Our Quality Assurance Requirements (October 2013, V4, pp4 and 22), refers to SQA Regulatory Principles 2011 and must be updated to reflect the introduction of SQA Regulatory Principles 2014. This has been recorded as Issue 2. Regulatory Principle 9. The awarding body shall ensure that it has robust systems and processes for the identification, design, development, implementation and review of qualifications, which meet the needs of users. City & Guilds have introduced a new system to identify qualifications where there is little or no uptake. An International Group, is currently looking at how they deal with these qualifications and to understand if there is any reasonable rationale behind their continued use. Where the group consider that there is no evidence to support the continuation of a qualification, appropriate action will be taken. Despite this newly introduced process, however, there are a number of SQA accredited qualifications which have had no uptake in the last two years. The Audit Team presented a list of these qualifications to relevant awarding body personnel, who acknowledged that these qualifications had not been included in recent withdrawal requests. Accordingly, City & Guilds must review the qualifications with no uptake, presented during the audit. This has been recorded as Issue 3. Regulatory Principle 13. The awarding body and its providers shall have clear, fair and equitable procedures to manage appeals. The City & Guilds document Review and Appeals (April 2013, p3) states that centres and candidates can contact the relevant regulator, where the awarding body appeals process has been completed and the outcome is deemed unsatisfactory. However, pages 14 and 16 of the same document, do not include appropriate regulatory reference, indicating that the final avenue of appeals for centres and candidates lies with the Independent Appeals Board, whose decision is final and binding. City & Guilds must ensure that the appeals process is stated consistently in its documentation with regard to regulatory escalation. This has been recorded as Issue 4. Scottish Qualifications Authority 2015 (Version 5) 8
2.2 Recommendations Regulatory Principle 10. The awarding body shall ensure that it has the necessary arrangements and resources for the effective delivery; assessment and quality assurance of SQA accredited qualifications. and Regulatory Principles Directive 1. In relation to the 10 week rule, City & Guilds personnel explained that the system was able to flag qualifications which prevented certificates being generated until the requisite 10 weeks had lapsed after registration. However the system did not allow the generation of data which would highlight providers who were regularly trying to claim certification either before or around the 10 week deadline. The existence of such data would allow external quality assurance activities to be targeted appropriately. To assist the quality assurance process, therefore, it is recommended that City & Guilds consider developing a tracking system to identify any patterns across providers with regard to the 10 week rule. This has been noted as Recommendation 1. Scottish Qualifications Authority 2015 (Version 5) 9
3 Acceptance of Audit Findings For and on behalf of City & Guilds: For and on behalf of SQA Accreditation: Print name... Print name... Signature... Signature... Designation... Designation... Date... Date... Scottish Qualifications Authority 2015 (Version 5) 10