The Research Assessment Exercise (RAE) is part of the UGC's commitment to assessing the performance of the UGC-funded universities and is intended to encourage world-class research and drive excellence. The RAE assesses the research quality of UGC-funded universities by using international benchmarks to delineate their areas of relative strength and make recommendations for further improvements. The UGC has since conducted six RAEs in 1993, 1996, 1999, 2006, 2014, and 2020.

In addition to research outputs, the RAE 2020 included new assessment elements in research impact and research environment, so as to encourage research of broader social relevance with high economic and social benefits, and take into account universities' strategy, resources and infrastructure in respect of research support. About 16,000 research outputs involving some 4,200 eligible academic staff, 342 research impact case studies and 190 research environment submissions were assessed in this exercise. The RAE 2020 was implemented under the purview of the Research Assessment Exercise Group (RAEG) of the UGC, under which 13 assessment panels of broad disciplines were formed.

RAE2020 assessed eight universities in Hong Kong:

• City University of Hong Kong (CityU)
• Hong Kong Baptist University (HKBU)
• Lingnan University (LU)
• The Chinese University of Hong Kong (CUHK)
• The Education University of Hong Kong (EdUHK)
• The Hong Kong Polytechnic University (PolyU)
• The Hong Kong University of Science and Technology (HKUST)
• The University of Hong Kong (HKU)

University submissions are collected in 41 subject areas, called units of assessment (UoA), that are aggregated in 13 panels. A full list of these can found in Appendix 1 of the Framework for Research Assessment Exercise (RAE) 2020

For the purpose of the RAE 2020, impact is defined as the demonstrable contributions, beneficial effects, valuable changes or advantages that research qualitatively brings to the economy, society, culture, public policy or services, health, the environment or quality of life; and that are beyond the academia.

Academic impact, while being valuable, will be more appropriately assessed through the research output and environment elements in the RAE 2020.

Each university submitted a selection of impact case studies for assessment in RAE2020. An impact case study is a four-page document describing specific examples of impacts achieved during the assessment period by the submitting university, underpinned by research, research activity, or a body of work in the period from 1 January 2000 to 30 September 2019.

The RAE 2020 Impact case studies database contains 342 documents. The original PDF submissions were converted to HTML for display on the website and to facilitate searching. A number of data mining processes were carried out to enhance the data as follows:

• Impact Topics - using the text content from section four of the case studies (details of the impact), topic modelling was used to create 35 impact topics that were manually labelled by reviewing top keywords and reading case study titles. Case studies may be associated with up to three topics. The topic with the strongest weight is named the primary topic.
• Thematic Clusters - the 35 impact topics were grouped into 11 thematic clusters.
• Impact Sectors - as part of the analysis, each case study was read by research evaluation experts and tagged with various metadata schemes including a list of impact sectors
• Locations - using the text content from section four of the case studies (details of the impact), geoparsing was used to identify where in the text a location is mentioned. Continents, countries, regions and cities were matched and linked to the Open Source database Geonames. Manual curation was used to remove false-positives and ensure ambiguous place names were correctly matched.
• Funders - research publications listed in section three of the case studies (references to the research) were matched to the Web of Science™ bibliographic database. Any funders acknowledged in the publications are associated with the case study along with a count (the number of publications that acknowledged the funder).
• Research Categories - Web of Science™ journal categories for research publications matched in section three of the case studies were collected. The top six most frequently assigned categories were assigned to the case study.

The search box on the home page can be used to find impact case studies that contain certain keywords or phrases, as well as partial word matching and word proximity searching. Some examples search types are:

• Simple keyword searching - e.g. search for “patent” to find case studies that discuss patenting, search for “SDG” to find case studies related to UN sustainable development goals.
• Phrase searching - e.g. search for “machine learning” to find case studies that contain the text machine learning.
• Wildcard searching - e.g. search for “communit*” to match the words community or communities.
• Word proximity - e.g. search for “spin out company~3” to find case studies where the words spin out company appear within three words of each other.

The filters allow users to list case studies that have certain metadata values. For example, to show case studies submitted by a particular university, select the university from the dropdown list and the results will automatically refresh. Multiple filters can be used together (e.g. by location and impact sector), and in combination with text searching.

For certain filters where the number of options is very long (e.g. locations, research categories, and funders), a search box is also provided to help the user find the appropriate option.

Some fields are organised in a hierarchy. For example, the list of Units of Assessment is grouped by main panel, making it possible to select entire panels if desired. An explanation of these follows:

• Unit of Assessment: Main panel (level 1) and Unit of Assessment (level 2)
• Research Categories: Essential Sciences Indicators™ (ESI) field (level 1) and Web of Science™ journal category (level 2)
• Impact Topics: Thematic cluster (level 1) and Impact topic (level 2)
• Locations: Geographic region (level 1), country or territory (level 2), region or city (level 3 if available)

As part of the analysis, each case study was read by research evaluation experts and tagged with various metadata schemes including a list of impact sectors. For this task, top-level fields from the Hong Kong Standard Industrial Classification Version 2.0 were used. This classification was chosen because it is a comprehensive system that allows R&D activity to be categorised according to the intended purpose or outcome of the research rather than the processes or techniques used in order to achieve this objective. It provides a set of categories which collectively exhaust all the objectives of research.

Using the text content from section four of the case studies (details of the impact), topic modelling was used to create 35 impact topics. Topic modelling is a natural language processing (NLP) technique that determines how certain clusters of related words (topics) can be used to categorise the underlying data. It is a data driven approach, meaning results are not dependent on pre-conceived notions of structure, but are instead derived from the data itself. Each topic was manually labelled by reviewing the related keywords and reading case study titles. These 35 topics were then grouped into 11 thematic clusters corresponding to high-level impact areas.

Using the text content from section four of the case studies (details of the impact), geoparsing was used to identify where in the text a location is mentioned. Continents, countries, regions and cities were matched and linked to the Open Source database Geonames. Manual curation was used to remove false-positives and ensure ambiguous place names were correctly matched. We used the Open Source Edinburgh Geoparser library.

Research publications listed in section three of the case studies (references to the research) were matched to the Web of Science™ bibliographic database. Any funders acknowledged in the publications are associated with the case study along with a count denoting the number of publications that acknowledged the funder.

Web of Science™ journal categories for research publications matched in section three of the case studies were collected. The top six most frequently assigned categories were assigned to the case study. In a small number of cases, more than six categories with an equal weight were found - in this situation, manual review determined the best six categories to use based on providing the broadest coverage of subjects and disciplines.

The original PDF documents have been converted to HTML for display on the website and to enable the content to be searched more easily. Every effort was made to retain the text flow and formatting structure of the original submission while ensuring each document has consistent headings for the main sections. When viewing case studies on this website, additional metadata is included in information boxes on the right side.

A link to the submitted PDF is provided for each case study, along with a link to the corresponding Impact Overview Statement.

Text has been removed (or ‘redacted’) from some case studies by the submitting institutions because it is commercially sensitive or otherwise needs to be restricted. Generally this is indicated by the formula phrase (-redacted-).

Redaction is the censoring or obscuring of part of a text. Some case studies have parts of the text removed (redacted) for confidentiality reasons (for instance, commercial sensitivity). This is indicated in the text using the phrase (-redacted-).

UGC commissioned a consortium led by Clarivate, including Different Angles and Electric Data Solutions, to build the impact case study database and to provide a series of synthetic analysis reports.

The website was designed and built by And Another Day - a UK technology company that builds innovative digital products for GreenTech organisations and Fourth Sector businesses to help them solve problems and communicate their message to a wider audience.