Precedents database

Application	Initial
Review	Full, coordinated by ENQA
Decision of	03/03/2023
Standard	3.4 Thematic analysis
Keywords	Content of the thematic analysis
Panel conclusion	Partial compliance
Clarification request(s)	–
RC decision	Compliance

“From several Annual reports of the agency from recent years, the Committee learned that the agency provides key figures on its work, but

also provides an in-depth overview of the compliance levels per standard in the undertaken accreditation procedures, summarises the good practices and obstacles observed in HEIs, and presents the most common recommendations by the panels.Following this, the Register Committee found that the current practice fulfils the minimum ESG criteria”

Full decision: see agency register entry

Application	Initial
Review	Full, coordinated by ENQA
Decision of	13/10/2023
Standard	3.4 Thematic analysis
Keywords	Performing thematic analysis
Panel conclusion	Partial compliance
Clarification request(s)	–
RC decision	Partial compliance

“The agency does reports covering different

topics – however they are mostly related to projects, rather than findings

from the external quality assurance activities. The panel further highlighted

that the agency is aware of the importance of collecting and analysing data

from the review processes for its own development purposes, but also for

contributing to the system level policies. The Committee acknowledged that while the thematic analyses are

not taking place in a systemic and formal manner, KAZSEE is a relatively

new agency that demonstrates interest to further formalise the processes

for producing thematic analyses.”

Full decision: see agency register entry

Application	Initial
Review	Full, coordinated by ENQA
Decision of	25/10/2022
Standard	3.4 Thematic analysis
Keywords	relevance of topics, nature of topics to qualify, regularity and frequency of analyses
Panel conclusion	Partial compliance
Clarification request(s)	–
RC decision	Compliance

“20. The panel discussed the analyses produced by GAC on various topics, with the most recent ones focusing on conditions imposed on higher education institutions/programmes in decisions by the agencies and by GAC, published in 2018 and 2020 respectively. While the panel questioned whether this was “a crucial topic in the development of the HE system”, the Register Committee considered that such an analysis is certainly based on the general findings of GAC's external quality assurance activities and thus meets the expectation of the standard. Moreover, while the panel did not specifically indicate whether stakeholders found the topic relevant, an analysis of conditions might show how the accreditation criteria resonate with the sector and indicate topics that are typically challenging for institutions and programmes, and hence be relevant beyond GAC.

21. Given the role of GAC as the central body of the German accreditation system, the panel considered that the current publishing rate (one paper per year) was “insufficient”. As the standard remains completely open as to the frequency of analysis, the Register Committee found it an overly strict interpretation of the standard to influence the compliance level on that basis; the remark should rather be seen as a recommendation to publish more analyses.”

Full decision: see agency register entry

Application	Renewal
Review	Full, coordinated by ASHE
Decision of	15/10/2021
Standard	3.4 Thematic analysis
Keywords	thematic analysis conducted on a regular basis
Panel conclusion	Substantial compliance
Clarification request(s)	–
RC decision	Compliance

“In its last renewal of registration on EQAR, the Register Committee noted that ASIIN only partially fulfilled the requirement of the standard, since ASIIN did not conduct such analysis on a regular basis and the prepared analysis and studies contained only elements of what is understood as thematic analysis. In its current review report, the panel commends ASIIN for its efforts in regularly developing thematic analysis through its impact studies which provide significant insights on the agency’s external QA activities. While the panel finds that ASIIN could improve the dissemination of its impact studies among stakeholders, the panel is satisfied that the requirement of the standard is met. Having addressed the earlier concerns in its compliance with ESG 3.4, the Register Committee concurred with the panel’s conclusion that ASIIN now complies with the standard.”

Full decision: see agency register entry

Application	Renewal
Review	Full, coordinated by ENQA
Decision of	07/02/2022
Standard	3.4 Thematic analysis
Keywords	structured approach in carrying out of thematic analysis, limited work carried out
Panel conclusion	Partial compliance
Clarification request(s)	–
RC decision	Partial compliance

“26. The panel critically remarked that FIBAA does not seem to collect in a systematic way information on programmes and institutions and that the agency does not produce an overview of the outcomes of it external QA activities.

27. The panel also noted that workshops on topics related to quality assurance and resulting studies are mainly organised by FIBAA Consult. While the panel is of the view that FIBAA should reconsider the assignment of such activities to FIBAA Consult, the Register Committee disagrees and underlines that the standard does not limit the carrying out of thematic analysis by a unit of the agency or the possibility to subcontract this work to an external body.

28. The Register Committee however agrees with the panel’s conclusion that FIBAA should ensure a structured approach and allocate more resources to the planning and carrying out of thematic analysis on a systematic basis.

29. Considering the limited work in producing thematic analysis, the Committee concurs with the panel’s conclusion that FIBAA complies only partially with ESG 3.4.”

Full decision: see agency register entry

Application	Renewal
Review	Full, coordinated by ENQA
Decision of	16/03/2020
Standard	3.4 Thematic analysis
Keywords	Lack of ressources for thematic analysis
Panel conclusion	Partial compliance
Clarification request(s)	–
RC decision	Partial compliance

“The panel’s findings show that ACSUG engages in broader forms of analysis, however these studies do not fully answer the demand for thematic analysis as defined in the ESG. In its Additional information to the external review report, the agency stated that the development of thematic analysis is going to become its priority following the hiring of new staff. ACSUG intends to launch external working groups to develop thematic analysis, which are to be coordinated by the new staff. In addition ACSUG also is considering to develop thematic analysis through their involvement in REACU (the Spanish Network of quality assurance agencies). The Register Committee welcomed ACSUG’s plans of addressing its shortcomings in preparing thematic analysis, but as it currently stands ACSUG does not fully meet the requirement of the standard i.e. no studies have been undertaken yet by its new working group. The Register Committee therefore concurred with the review panel that ACSUG complies only partially with ESG 3.4.”

Full decision: see agency register entry

Application	Initial
Review	Full, coordinated by ENQA
Decision of	05/11/2019
Standard	3.4 Thematic analysis
Keywords	Producing effective thematic analysis reports
Panel conclusion	Non-compliance
Clarification request(s)	–
RC decision	Compliance

“The review panel’s analysis show that CYQAA does not have a thorough and systematic analysis of findings from the evaluations conducted which could feed into a national quality assurance policy or guide quality improvement at higher education institutions. The panel nevertheless found that CYQAA’s Annual Report includes a few paragraphs referring to general issues and good practices in higher education in Cyprus. In its statement to the review report, the agency stated that it has collected the General Evaluation Reports from higher education institutions in Cyprus in November 2018 and has appointed a team of qualified members responsible for producing systematic analysis using the agency’s collected reports. The agency added that the findings of its thematic analysis are expected to be published within

2019. In its addition representation, the agency provided evidence of its implementation of thematic analysis and provided a link to its new publications. The agency has carried out an analysis of the “trends and dominant issues“ within i.e. e-learning programme evaluation, master programmes’ evaluation and the evaluation of short cycle programmes within colleges. The agency stated that it has made use of its thematic results, identifying weaknesses in aspects related to distance learning and recommending further attention to these issues. Considering the changes enacted by the agency, the Register Committee considered that CYQAA now complies with ESG 3.4.”

Full decision: see agency register entry

Application	Initial
Review	Full, coordinated by ENQA
Decision of	19/06/2019
Standard	3.4 Thematic analysis
Keywords	systematic approach
Panel conclusion	Substantial compliance
Clarification request(s)	Agency (06/06/2019)
RC decision	Partial compliance

“In its analysis the panel considered that while NCEQE produced a form of thematic analysis in its annual report, the panel commented that the activity was not yet systematically carried out. In its letter to EQAR, the agency reported that as part of its Twinning Project (starting in 2019) a methodology is developed for carrying out analytical and research activities in a more consistent and streamlined manner. The agency added it has also received support via other projects that will help the agency assess the results of implementation of the revised QA system and further improve its QA procedures. While the Register Committee welcomed the steps taken by NCEQE, it was not yet possible to conclude whether thematic analyses are produced regularly. The Committee therefore concluded that NCEQE complies only partially with ESG 3.4.”

Full decision: see agency register entry

Application	Renewal
Review	Full, coordinated by ENQA
Decision of	16/03/2020
Standard	3.4 Thematic analysis
Keywords	Lack of clarity about existing activities; lack of regular analyses as described by the standard
Panel conclusion	Non-compliance
Clarification request(s)	–
RC decision	Partial compliance

“36. [...] the publication of assessment reports, its yearly board meeting and the publication of books/journals by AHPGS staff – did not represent thematic analyses resulting from the review processes undertaken by AHPGS. […]

38. […] The statement by AHPGS did, however, not provide details as to whether and how these are clearly based on findings from AHPGS accreditation work.

39. In its additional representation, AHPGS reiterated the view that the various past publications would not have been possible without the experience from the agency’s review processes.

40. Moreover, AHPGS provided further details on its concept and work plan of publishing two thematic analyses per year. These were developed based on decisions by its governing bodies in

2019. 41. AHPGS also pointed out that it had already published its first two thematic analyses after the external review.

42. […] The Committee considered that through the combination of past publications and the two recently published thematic analyses AHPGS showed its capacity to implement that concept.”

Full decision: see agency register entry

Application	Initial
Review	Full, coordinated by ENQA
Decision of	03/04/2019
Standard	3.4 Thematic analysis
Keywords	Use and regular publication of thematic analysis
Panel conclusion	Substantial compliance
Clarification request(s)	–
RC decision	Partial compliance

“While evidence of thematic analysis can be found within the programme accreditation reports, annual reports and other publications of the staff on the "activities, trends and outlook" of the HAC (SAR 41), the panel found that HAC’s level of activity in thematic analysis is limited and insufficiently developed. In its additional representation the agency stated that a thematic overview and analysis on ESG compliance from its most recent institutional accreditation round was presented during a recent event and it will be published by HAC. Further thematic analysis are planned for

2019. The Committee acknowledged the agency’s recent activity, however as it stands the agency’s progress is modest. The Committee underlines the review panel’s recommendation of ensuring the regular publication of thematic work and making use of such analysis more widely.”

Full decision: see agency register entry

Application	Renewal
Review	Full, coordinated by ENQA
Decision of	04/04/2019
Standard	3.4 Thematic analysis
Keywords	thematic analysis conducted on a regular basis
Panel conclusion	Full compliance
Clarification request(s)	Panel (26/02/2021)
RC decision	Compliance

“While the panel found that none of the research projects described by ARACIS in its self assessment report met the requirement of the standard, the panel nevertheless formed the view that the agency is (fully) compliant with ESG 3.4, based on the Quality Barometer reports produced for 2009, 2010 and 2015.

The Register Committee could not conclude on the basis of the panel’s analysis whether the agency made use of the outcomes of these analysis and whether the Barometer reports are conducted on a regular basis. The Committee has therefore asked the panel for further clarifications.

In its response the panel stated that it was keen to see that ARACIS continues its commitment to the Quality Barometer series and reiterated its appreciation for the agency's dedication in preparing them, even though it comes at a substantial financial and management burden on the agency.

With a view to the application of Quality Barometers, the panel stated that they are used by ARACIS to engage with quality professionals, academics, students, and others throughout Romania and to disseminate aspects related to higher education to interested parties.”

Full decision: see agency register entry

Application	Initial
Review	Full, coordinated by ENQA
Decision of	19/06/2019
Standard	3.4 Thematic analysis
Keywords	thematic analysis not conducted on a regular basis
Panel conclusion	Partial compliance
Clarification request(s)	–
RC decision	Partial compliance

“The Register Committee noted that thematic analysis is not a regular activity performed by NQA, and that only a limited number of thematic analyses have been carried out in the past following requests from higher education institutions. The Register Committee underlined the recommendation of the panel to include thematic analysis as part of NQA’s regular planned activities and to make use of the experience and knowledge of internal and external secretaries in order to conduct such analyses. In its additional representation the agency stated that it concurs with the review panel’s analysis and that the agency has put in place an improvement plan which it will realised in the following four years.. The Register Committee welcomed the steps taken by NQA but noted that presently the agency has not yet implemented its plans to carry out systematic analysis. The Committee therefore concurred with the panel that NQA is only partially compliant with standard 3.4.”

Full decision: see agency register entry

Application	Renewal
Review	Full, coordinated by GAC
Decision of	20/06/2017
Standard	3.4 Thematic analysis
Keywords	Systematic approach to analyses
Panel conclusion	Substantial compliance
Clarification request(s)	–
RC decision	Compliance

“When FIBAA ‘s registration was last renewed, the Register Committee flagged for attention the establishment of a systematic analyses of FIBAA's overarching reflections and observations from its accreditation, evaluation and audit activities. The panel showed in its analysis that FIBAA initiated a regular collection of information on the outcomes of the agency’s processes, such as conditions formulated, feedback from workshops and from staff. The findings are presented in FIBAA’s newsletters, workshop articles and academic articles. While the review panel noted an increase in the analytical publications prepared by the agency, the panel also recommended that such publications are extended so as to also cover fields of activity beyond programme and system accreditation. ”

Full decision: see agency register entry

Application	Renewal
Review	Full, coordinated by ENQA
Decision of	16/11/2017
Standard	3.4 Thematic analysis
Keywords	Lack of systematic approach to system-wide analyses
Panel conclusion	Partial compliance
Clarification request(s)	–
RC decision	Partial compliance

“ In its decision for inclusion, the Register Committee flagged QANU’s systematic approach to system-wide analyses. While QANU has not produced thematic analysis in its strict sense, the panel noted that the agency published reports at the request of research universities and state of the art reports for programmes that are assessed in clusters. The panel noted that QANU did not have a systematic approach towards producing thematic analysis and that the agency views this activity to remain mainly in the responsibility of NVAO. In its additional representation, QANU's underlined that its core activity is to conduct assessments, and due to its limited size, financial resources and scope QANU’s ability to conduct thematic analyses in a systematic way was limited. The Board of QANU had nevertheless considered the recommendations of the panel and stated that it would adapt its approach to thematic analysis.”

Full decision: see agency register entry

Application	Initial
Review	Full, coordinated by ENQA
Decision of	20/06/2017
Standard	3.4 Thematic analysis
Keywords	studies based on expert panel review reports are published
Panel conclusion	Partial compliance
Clarification request(s)	–
RC decision	Compliance

“While the panel welcomed the consistent publication of studies by IQAA addressing the Kazakh higher education and QA systems and the impact of IQAA’s reviews on institutional enhancement the panel however noted that IQAA does not analyse in its publication the results from its institutional and programme reviews. In its statement on the External Review Report, the agency explained that it did not interpret standard 3.4 to explicitly require analyses based on external review reports, but it nevertheless agreed on the importance of these findings and has published a number of studies based on expert panel review reports on institutional accreditation. Considering the evidence provided the Register Committee found that IQAA has addressed its compliance with the standard.”

Full decision: see agency register entry

Application	Renewal
Review	Full, coordinated by GAC
Decision of	20/06/2017
Standard	3.4 Thematic analysis
Keywords	systematic approach to analyses
Panel conclusion	Partial compliance
Clarification request(s)	–
RC decision	Partial compliance

“The review panel considered that the ASIIN newsletter and the annual meetings only partially fulfil the requirements of the standard. The panel noted that the newsletters do not include an analysis of finding ascertained by ASSIIN in its own work. While the annual meetings might address the type of issues to be covered by thematic analyses, the panel noted that there was no specific documentation with summary analyses resulting from those meetings. The Register Committee took note of ASIIN's Statement on the review report and the additional representation. While referring to a number of valuable initiatives and publications, the Committee considered that many of those rather represent internal feedback mechanisms or approaches to ensure consistency in decision-making. While some of ASIIN's newsletters and publications contain elements of thematic analyses, there are currently no regular published analyses which fully respond to the requirements of this standard.The Register Committee further underlined that the standard is not entirely new, but succeeded the standard “system-wide analyses” of the ESG 2005, which, despite its name, applied to all registered agencies already in the past.”

Full decision: see agency register entry

Application	Renewal
Review	Full, coordinated by ENQA
Decision of	20/06/2017
Standard	3.4 Thematic analysis
Keywords	Lack of plan for thematic analysis;
Panel conclusion	Substantial compliance
Clarification request(s)	–
RC decision	Partial compliance

“In its decision of inclusion the Register the Committee noted that attention should be given to whether ECCE publishes regularly and systematically overarching analyses from its accreditation activity. The review panel’s finding showed that ECCE had not undertaken an analysis into its external quality assurance activities although the agency had committed to a series of research of pedagogic papers in its Draft Strategic Plan 2016-2020. The Register Committee underlined the view of the panel that while the need to undertake such analysis had increased (as a result of the rise in the number of reviews) that ECCE lacked the human resources and capacity to undertake such a structured analysis. In its additional representation ECCE provided three research papers co-authored by one of the accreditation committee members that were used to update ECCE standards. The agency also stated that the newly hired consultant will be expected to perform specific research on the internal activities of the organisation. While the Register Committee noted that resources were allocated into the development of thematic analysis, the Committee could not conclude that ECCE’s research papers describe and analyse the general findings of the agency’s activities nor that ECCE has developed a plan to ensure that thematic analysis will be regularly undertaken.”

Full decision: see agency register entry

Application	Renewal
Review	Targeted, coordinated by ENQA
Decision of	02/07/2024
Standard	3.4 Thematic analysis
Keywords	Inactivity in thematic analysis
Panel conclusion	Partial compliance
Clarification request(s)	–
RC decision	Partial compliance

“21. The Register Committee understood by the panel’s analysis that the agency, since the last review, produced two publication named “Thematic analysis 2018 to mid-2021” and “Thematic Analysis mid 2021 to mid 2023”. However, as noted by the panel “these significantly digressed from the previous approach and in the eyes of the panel could not be taken as reports that describe and analyse the general findings of evalag’s external quality assurance activities as established by the standard. These reports give merely an overview of the most important projects, publications and events in the period under study, more like an annual report.”

22. The Register Committee, could concur with the panel analysis and underlined the panel’s recommendation that the agency should resume the work on thematic analysis, which was abandoned after 2018, particularly in light of the severe changes in the ‘German system’.

23. The Register Committee therefore concurred with the panel that evalag complies only partially with the standard.

24. For the remaining standards, the Register Committee was able to concur with the review panel's analysis and conclusion without further comments.”

Full decision: see agency register entry

Application	Renewal
Review	Full, coordinated by ENQA
Decision of	13/10/2023
Standard	3.4 Thematic analysis
Keywords	the geographical coverage of thematic analysis
Panel conclusion	Partial compliance
Clarification request(s)	–
RC decision	Compliance

“21. The Register Committee noted that QAA carries out systematic thematic analysis within Scotland, sector-wide analysis in Wales, while UK-wide QAA has only carried out “The Quality Assurance of Alternative Providers: A Retrospective View”.

22. The Committee finds that the current activity is sufficient in its understanding and interpretation of the standard and therefore could not follow the panel’s judgment of partial compliance and concluded that QAA complies with ESG 3.4.

23. The Register Committee nevertheless underlines the panel’s recommendation that QAA should develop a clearer plan for thematic analyses for all of its external QA activities in all nations of the UK and publish them on its website.”

Full decision: see agency register entry

Application	Renewal
Review	Full, coordinated by ENQA
Decision of	12/12/2023
Standard	3.4 Thematic analysis
Keywords	publication, analysis
Panel conclusion	Partial compliance
Clarification request(s)	–
RC decision	Partial compliance

“14. In its last decision, the Register Committee welcomed AKAST plans for further development of its thematic analysis after an appropriate number of programme accreditation procedures have been carried out.

15. The Register Committee noted that while AKAST has taken some steps in preparing thematic analysis since its last review, i.e., initiating a process for evaluation of the peer review processes carried out by AKAST, since 2022 at the time of the review, no thematic analysis have been made available nor any kind of such analysis have been published.

16. Considering the limited progress made since the inclusion on the Register and the limited development of thematic analysis, the Register Committee concurred with the panel that AKAST complies only partially with ESG 3.4.”

Full decision: see agency register entry