«Ascribing meaning to the standards and guidelines for quality assurance in the European higher education area in Portugal Paper presented in track 1 ...»
Sarrico, C. S., Veiga, A., Amaral, A. (2013). The long road—how evolving institutional governance mechanisms are changing the face of quality in Portuguese higher education. Educational Assessment, Evaluation and Accountability, 25 (4), 375–391.
Schwarz, S., Westerheijden, D.F. (2004). Accreditation and evaluation in the European higher education area. Dordrecht: Kluwer.
Veiga, A., Sarrico, C. S. (2014). Changes in governance: Do they help overcome barriers to the implementation of the European standards and guidelines for quality assurance in higher education? In H. Eggins (Ed.), Drivers and barriers to achieving quality in higher education (pp.
67–81). Rotterdam: Sense.
Ascribing meaning to the standards and guidelines for quality assurance in the European higher education area in Portugal Appendix – ESG Part1 degree of implementation scale The sentences presented in tables A1 and A2 reflect different quality assurance practices and were drafted based on the guidelines proposed for each one of the seven standards of ESG Part1.
Academics have been asked to score each one of them on a scale ranging from 1 – totally disagree to 7 – totally agree. Statistics have been computed for the answers given to each question and are presented in Table A2 where the sentences appear grouped in the subscales resulting from the factorial analysis. Furthermore Table A2 presents information on the results of the factorial analysis, namely, the loadings of each variable (sentence/quality assurance practice) on each one of the extracted factors (subscales) for the final factor solution (after oblique rotation - Promax).
Table A3 shows Pearson correlation coefficients between the six subscales emerging from the factorial analysis and Table A4 shows paired sample t-tests for statistical significant differences among the six subscales emerging from the factorial analysis.
Table A1 - Sentences designed to operationalise each one of the seven ESG Part 1 standards and assess their degree of implementation in Portuguese academics HEIs.
ESG 1 - Policy and procedures for quality assurance S1.1 The quality assurance policy of the institution contains the statements of intentions and the principal means by which these will be achieved.
S1.2 The strategy, policy and procedures include a role for all the stakeholders: teachers, students, nonteaching staff, employers, professional associations, etc.
S1.3 The strategy, policy and procedures for quality assurance of the institution have a formal status and are publicly available.
S1.4 There is procedural guidance which gives detailed information about the ways in which the policy is implemented.
S1.5 The institution has a policy and associated procedures for the assurance of the quality and standards of their programmes and awards.
S1.6 The institution develops and implements a strategy for the continuous enhancement of quality.
S1.7 The institution is explicitly committed with the development of a culture acknowledging quality and quality assurance importance for its work.
The policy statement for quality assurance includes:
S1.8 the organisation of the quality assurance system S1.9 the relationship between teaching and research in the institution S1.10 the institution’s strategy for quality and standards S1.11 the responsibilities of departments, schools, faculties and other organisational units and individuals for the assurance of quality S1.12 the involvement of students in quality assurance S1.13 the ways in which the policy is implemented, monitored and revised ESG2 – approval, monitoring and periodic reviews of programmes and awards S2.1 The institution has formal mechanisms for the approval of their programmes and awards.
S2.2 The institution has formal mechanisms for periodic review and monitoring of its programmes and awards.
S2.3 Quality assurance activities ensure that programmes are well-designed, regularly monitored and periodically reviewed, thereby securing their continuing relevance and currency.
The quality assurance of programmes and awards includes:
S2.4 careful attention to curriculum and programme design and content S2.5 specific needs of different modes of delivery (e.g. full-time, part-time, distance-learning, elearning) S2.6 formal programme approval procedures by a body other than that teaching the programme Ascribing meaning to the standards and guidelines for quality assurance in the European higher education area in Portugal S2.7 monitoring of the progress and achievements of students S2.8 regular periodic reviews of programmes, including external panel members S2.9 regular feedback from employers, labour market representatives and other relevant organisations ESG3 – assessment of students S3.1 Students are clearly informed about the assessment strategy being used for their programme, what will be expected of them and the criteria that will be applied to the assessment of their performance.
S3.2 Students are clearly informed about what examinations or other assessment methods they will be subjected to.
S3.3 The assessment of students is carried out professionally and takes into account the extensive knowledge that exists about testing and examination processes.
S3.4 Students are assessed using published criteria, regulations and procedures which are applied consistently.
Student assessment procedures:
S3.5 are designed to measure the achievement of the intended learning outcomes and other programme objectives S3.6 are appropriate for their purpose, whether diagnostic, formative or summative S3.7 have clear and published criteria for marking S3.8 are undertaken by people who understand the role of assessment in the progression of students towards the achievement of the knowledge and skills associated with their intended qualification S3.9 do not rely on the judgements of a single examiner S3.10 ensure that assessments are conducted securely in accordance with the institution’s stated procedures S3.11 are subject to administrative verification checks to ensure the accuracy of the procedures ESG4 – quality assurance of teaching staff S4.1 The mechanisms which ensure teachers’ qualifications and competencies are discussed in internal reports for the quality assurance of the institution.
S4.2 Teaching staff is encouraged to value their skills.
S4.3 The institution has ways of satisfying itself that staff involved in teaching activities are qualified and competent to do so.
S4.4 The institution has the means to remove teachers from their teaching duties if they continue to be demonstrably ineffective.
S4.5 The institution provides poor teachers with opportunities to improve their skills to an acceptable level.
S4.6 Teachers can access feedback on their own performance.
S4.7 Teachers have the necessary skills and experience to transmit their knowledge and understanding effectively to students.
S4.8 The institution ensures that their staff recruitment and appointment procedures include means of making certain that all new staff have at least the minimum necessary level of competence.
ESG5 – Learning resources and student support S5.1 Learning resources and other support mechanisms are readily accessible to students.
S5.2 Learning resources and other support mechanisms take into consideration the needs and the feedback from the students.
S5.3 The institution ensures that the resources available for the support of student learning are adequate and appropriate for each programme offered.
S5.4 The institution routinely monitors and reviews the support services available to its students.
S5.5The institution has human resources such as tutors, counsellors and other advisors to support student learning.
S5.6 The institution has physical resources such as libraries or computing facilities to support student learning.
S5.7 The institution routinely improves the effectiveness of the support services available to its students.
Ascribing meaning to the standards and guidelines for quality assurance in the European higher education area in Portugal SG6 – information systems S6.1 The institution has information about its graduates’ opinions and professional careers.
S6.2 The institution compares itself with other similar organisations in the European higher education area and beyond.
S6.3 The institution collects and analyses relevant information for the effective management of its programmes of study and other activities.
S6.4 Based on the collected information, the institution knows what is working well and what needs attention, and the results of innovative practice.
S6.5 The comparison with similar institutions allows the institution to extend the range of its self-knowledge and to access possible ways of improving its own performance.
The quality-related information systems of the institution cover:
S6.6 student progression and success rates S6.7 employability of graduates S6.8 students’ satisfaction with their programmes S6.9 effectiveness of teachers S6.10 profile of the student population S6.11 learning resources available and their costs S6.12 the institutions’ own key performance indicators ESG7 – Public information S7.1 The institution publishes public information about the teaching, learning and assessment procedures it uses.
S7.2 The institution publishes information about the profile of the current student population.
S7.3 The institution publishes information about the intended learning outcomes.
S7.4 The institution publishes information about the qualifications they award and the learning opportunities available to its students.
S7.5 The public information is not used simply as a marketing opportunity.
S7.6 The public information is accurate, impartial, objective and readily accessible.
Table A2 – Portuguese academics’ perceptions on the degree of implementation of the different practices that operationalize the ESG Part1
Ascribing meaning to the standards and guidelines for quality assurance in the European higher education area in Portugal Table A3 – Pearson correlation coefficients between the six subscales emerging from the factorial analysis (correlations are significant for a 0.01 significant level)
Tables A4 – Paired sample t-tests for statistical significance differences among the six subscales emerging from the factorial analysis (t-value and p-value for each one of the 15 pairs)
Ascribing meaning to the standards and guidelines for quality assurance in the European higher education area in