Research Centre

Home > Research Centre

Centre for Business, Socio-Economics and Innovation Research

The Centre aims to produce high-quality and impactful research papers in business, socioeconomics and innovation; provide consultancy services; and establish regional, national and international partnerships with industries, government and non-governmental organisations, academia and other research centres specifically in research and development. 

Our Vision

The Centre strives to be a premier consultancy service provider and a leading research centre dedicated to producing high-standard and impactful research papers in business, socioeconomics and innovation for various stakeholders within the university and across industries, communities, states, and national and international arenas. Our objectives are to:


The Centre aims to

Members Profile

Prof. Dr. Toh Guat Guan (Leader) 

Branch Head, Penang Branch 

Academic Qualification : 

Email : tohgg@tarc.edu.my

Research Area : Finance, Behavioural Finance, Management, TRIZ

Biography

Assoc. Prof. Dr. Janice Toh is currently the Head of Penang Branch, Tunku Abdul Rahman University of Management and Technology (TAR UMT). She has taught various management and finance related subjects for the past 28 years. Currently, she supervises postgraduate students in the areas of management, economics and finance. Her research interests include the management, corporate governance, finance and TRIZ applications. She has presented several research papers in the national and international conferences and has won the best paper award in the 8th Asian Academy of Management International Conference. Besides, she is also a member of the review panel for journals and conferences such as the Asian Journal of Economics and Finance, Asian Conference of Sustainability, Energy and Environment, TARC International Conference, and European Asian Economics, Finance, Econometrics and Accounting Science Association. Prior to her academic career, she has worked in the manufacturing and banking industries, assuming administrative, operations and marketing role. She is a Certified MyTRIZ Level 1 instructor and Certified MyTRIZ Level 3 practitioner. She has conducted several TRIZ workshops for the undergraduates. Besides, she is also a Certified Buzan Licensed Instructor and has conducted Mind-Mapping® workshops for teachers and students of primary, secondary and tertiary levels. In addition, she is also the Registered online mentor for Oxford Brooke University, guiding students from various countries to complete the research and analysis report such as China, Korea, Greece, UK, Singapore and Pakistan. Apart from this, she is also one of the panel of assessors for Malaysian Qualification Agency (MQA) with the responsibility to process applications to conduct the programmes of study from Private Higher Education Institutions and provide recommendations to MQA for a decision. She strongly upholds life-long learning spirit and has a passion to impart knowledge and to share experience to ensure that students can gain a greater understanding of the subject matters 

External Services to the Academic

13-15 April 2023

Invited as a Special Speaker for Ajou University Graduate School of Engineering’s Overseas IP Field Trip at Ho Chi Minh University, Vietnam.

14-17 October 2021

Paper Reviewer for International Conference on Material Science and Engineering Technology (ICMSET2021), Kyoto, Japan

14-16 October 2020

Paper Reviewer for The Online MyTRIZ Conference 2020

7 December 2019

Invited as a Guest Speaker for the Intellectual Property Strategy Research Forum (IP Forum) held at Ajou University, Korea

8-10 June 2018

Paper Rveviewer for the Asian Conference on the Sustainability, Energy & the Environment (ACSEE2018), The Art Centre, Kobe, Japan

3-6 September 2014

Session Chair / Paper Reviewer for the 4th European Asian Economics, Finance, Econometrics, Accounting Science Association Conference (EAFEASA 2014), Eastern & Oriental Hotel, Georgetown, Penang

August 2013 till Present

Registered Mentor for B.Sc (Hons) Applied Accounting Programme, Oxford Brookes University, United Kingdom

External Services to the other University / Community / Society / Nation

      Registered Mentor for Oxford Brooke University’s B.Sc. in Applied Accounting (Honours)

MQA Assessor (Finance and Banking), Malaysian Qualification Agency

Member of 25 StartUps Advisory Council, Karuna Venture Capital Sdn Bhd

Services to the University

Publications

Workshops / Training Conducted (Facilitator/Instructor)

Research Project

Postgraduate Supervision (by Research)

Dr. Lim Khai Yin

Deputy Head, Penang Branch

Academic Qualification : 

Email : limky@tarc.edu.my

Research Area : Artificial Intelligence, Machine Learning, Image Processing, Forecasting

Biography

Lim Khai Yin received her Master of Science (Artificial Intelligence) and PhD (Visual Analytics) in 2009 and 2017, respectively. Currently, she is a senior lecturer in TAR UMT. Her research interests include computer vision, medical image processing, machine learning, and data analytics. She has also published papers both in journals and conference proceedings relating to artificial intelligence (AI) and computer vision, specifically in fuzzy theory, neural networks, and medical imaging during her postgraduate study. She has experience in guiding final year projects (FYP) as well as post-graduate students in the topics that involve AI and computer vision. Besides that, she has been involving in some industry projects, mainly in the topics of classifying painting styles and engagement prediction in social media. 

Journal


Conference Proceedings


Book


Consultancy in Industry 

   Roles : Co- researcher

2019-2021

Company: CMG  Holdings Sdn Bhd

Roles : Principal Researcher

2020-2022

Company: Webqlo Sdn Bhd

Roles : Co- researcher

2022-2023

Company: Work At Cloud Sdn Bhd

Roles : Project- lead

2022-2025

Company: B2BE GSS Sdn Bhd

Roles : Project- lead

2022-2027

Company: QNE Software Sdn Bhd


Reviewer

Mr. Lim Thean Pheng

Deputy Head, Penang Branch

Academic Qualification : 

Email : limtp@tarc.edu.my 

Research Area : Technology Acceptance, Technostress, Learning Engagement 

Biography

Lim Thean Pheng is a lecturer and Deputy Head at Tunku Abdul Rahman University of Management and Technology, Penang Branch.  He holds a bachelor’s degree in management and a master's degree in information technology (technopreneurship) from Universiti Sains Malaysia.  He is also a member of the Malaysian Institute of Management and the affiliate member of the Chartered Management Institute, UK. 

Lim has more than 15 years of working experience in the business and education industry.  Beginning his career as a Business and Financial Consultant, Lim was involved in various consultancy projects with multinational enterprises such as Chee Wah Corporation Berhad and International Business Machines Corporation (IBM). In academics, he has vast experience in supervising dissertations for bachelor’s degree students, industry consultancy projects and lecturing in courses related to Operations Management, Entrepreneurship and Information Technology.

Journal


Conference Proceedings


Reviewer


Consultancy



Dr. Chee Wei Loon

Senior Lecturer 

Academic Qualification : 

Email : cheewl@tarc.edu.my

Research Area : Entrepreneurship education, green and sustainable entrepreneurship

Biography

Dr. Chee Wei Loon has been a lecturer at Tunku Abdul Rahman University of Management and Technology since 2010. His research specialization areas are entrepreneurship education and green entrepreneurship. Prior to joining Tunku Abdul Rahman University of Management and Technology, he held several executive positions in the service sector such as Dell Global Business Center Sdn. Bhd. and Citigroup Transaction Services Sdn. Bhd. 

Journal


Conference Proceedings


Dr. Christine Chong Siew Pyng

Senior Lecturer 

Academic Qualification : 

Email : chongsp@tarc.edu.my

Research Area : Economics, corruption

Biography

Dr. Christine Chong Siew Pyng is a senior lecturer in Tunku Abdul Rahman University of Management and Technology, Penang. Her areas of  research interest include public finance, public sector economics, crime and corruption. She currently co-supervise a PHD candidate at Wawasan Open University and a Master student.  Prior to this, she has worked in domestic and international companies specialising in logistics and forwarding. 

Journal


Reviewer


Dr. Beh Chong You

Programme Leader/Senior Lecturer

Academic Qualification : 

Email : behcy@tarc.edu.my

Research Area : Biomedical Instrumentation; Advanced Biomaterials; Computational Electromagnetics; Wave Scattering and Propagation.

Biography

Dr. Beh Chong You is a Programme Leader and Senior Lecturer at Tunku Abdul Rahman University of Management and Technology, where he has been instrumental in delivering a wide range of electronic engineering courses at both Diploma and Degree levels. His expertise lies in the dynamic field of Biomedical Electronic Engineering, and he is recognized as a young and accomplished researcher in this domain. With an impressive track record, Dr. Beh Chong You has authored eight ISI-Indexed articles, where he served as the first author. Notably, seven of these articles were published in Tier/Quartile 1 journals, while the remaining one was published in a Tier/Quartile 4 journal. His work has garnered significant recognition and accolades, demonstrating his dedication and prowess in Biomedical Electronic Engineering. His research contributions have been particularly impactful in the areas of Biomedical Electromagnetics and Advanced Biomaterials. Driven by a thirst for knowledge and innovation, he continuously seeks opportunities to expand his expertise and contribute further to the field. Dr. Beh Chong You's commitment to excellence is reflected in his extensive publication record, which comprises 15 research publications with a cumulative Impact Factor of 49.16. This remarkable achievement speaks volumes about his dedication to advancing the boundaries of knowledge in Biomedical Electronic Engineering. Looking ahead, Dr. Beh Chong You eagerly anticipates the potential for his research to generate positive societal impacts. He is enthusiastic about collaborating with fellow researchers and exploring novel ideas that can pave the way for groundbreaking advancements in the field. Overall, Dr. Beh Chong You's professional biography underscores his passion for Biomedical Electronic Engineering, his accomplishments as a researcher, and his relentless pursuit of knowledge and innovation.

Journal


Conference Proceedings


Awards


Reviewer

Dr. Woo Suk Wah

Senior Lecturer 

Academic Qualification : 

Email : woosw@tarc.edu.my

Research Area : Project Management; Waterfront Development; Sustainable Development and Planning; Sociology in Construction

Biography

Woo Suk Wah has been a lecturer at Tunku Abdul Rahman University of Management and Technology since 2017. Her research specialization areas are waterfront development, project management and issues in construction related to sustainability and sociology. Prior to joining Tunku Abdul Rahman University of Management and Technology, she was a quantity surveyor and course coordinator for open and distance learning programmes. 

Journal


Conference Proceedings


Reviewer

1. Environment, Development and Sustainability 

2. Symposium on Technologies for Sustainable Urban Development (TechSUD 2023) 

Dr. Kamalesh Ravesangar

Senior Lecturer 

Academic Qualification : 

Email : kamalesh@tarc.edu.my

Research Area : Human Resource Management, Human Resource Development, Organizational Behavior

Biography

Dr. Kamalesh Ravesangar is an academician with over ten years of experience in teaching Certificate, Diploma, Degree, and supervision for MBA(coursework mode) students from various social and cultural backgrounds at private colleges and universities. Currently, she works as Senior Lecturer at Tunku Abdul Rahman University of Management & Technology in Malaysia. Moreover, she has a total of three years of experience in industries related to educational, manufacturing, recruitment, and engineering. She is also a Professional Member of Malaysian Institute of Human Resource Management (MIHRM).  A subject matter expert in human resource management, human resource development, management and organizational behavior studies. She has published articles and reviewed articles in online journal publications. She holds a Phd (doctorate) and a Master’s degree in Organizational Behavior and Human Resource Management. She is also peer-reviewer for a few journals in the field of Business, Management, HRM. Besides, she has been awarded " Certificate of Recognition & Excellence in Reviewing by Asian Journal of Economics, Business & Accounting and Journal of Global Economics, Management and Business Research & Institute of Industry and Academic Research Incorporated (IIARI). 

Journal


Book


Reviewer


Internal Grant

Dr. Tan Choo Jun 

Senior Lecturer 

Academic Qualification : 

Email : tancj@tarc.edu.my

Research Area : 

Biography

Choo Jun received his PhD degree from School of Computer Sciences, Universiti Sains Malaysia (USM) in 2014. He has more than 10 years of experience in software design and development, as well as research and development in artificial intelligence. He also has a wealth of experience in teaching and learning, conducting lectures both online and onsite for tertiary students, and also serving as a learning advisor for undergraduate and postgraduate students. Currently, he is an academic at Tunku Abdul Rahman University of Management and Technology (TAR UMT). His research specialization encompasses the design and development of computational intelligence models, especially evolutionary algorithms and their algorithms to real world problems.

Journals


Conference Proceedings


Book 


Consultancy in Industry


Recent Awards


International Conferences

Dr Teh Kate Yng

Senior Lecturer 

Academic Qualification : 

Email : tehky@tarc.edu.my

Research Area : Environmental risk communication, Communicating sustainability, Corporate social responsibilities

Biography

Dr Teh Kate Yng is currently a Senior Lecturer in the Department of Communication at Faculty of Communication and Creative Industries, TAR UMT Penang Branch. Holds a PhD in environmental communication, her research interests include environmental risk communication, environmental journalism studies, communicating sustainability, and corporate social responsibilities. Prior to her academic career, she has wide experiences and strong background in manufacturing and consultancy industries. She has successfully provided training and guidance to ensure her clients acquire necessary practical skills to develop and maintain management systems in their businesses. Additionally, completion of IRCA certified ISO 22000:2018 and ISO 9001:2015 Lead Auditor Training Courses has enabled her abilities towards auditing practices.  

Journal


Conference Proceedings

Mr. Lee Lai Seng

Senior Lecturer 

Academic Qualification : 

Email : leels@tarc.edu.my

Research Area : Roll-To-Plate Nanoimprint Lithography,   Design and fabrication of seamless PDMS mold

Biography

Lee Lai Seng has been a lecturer at Tunku Abdul Rahman University of Management and Technology since 2002. His research specialization areas are Roll-To-Plate Nanoimprint Lithography and design and fabrication of seamless PDMS mold. Prior to joining Tunku Abdul Rahman University of Management and Technology, he held several engineering positions in semiconductor companies such as Hitachi Semiconductor Malaysia (Kedah), Hitachi Semiconductor Malaysia (Penang), Agilent Technologies Malaysia and OSRAM Opto Semiconductor.

Conference Proceedings

Mrs. Cheng Seow Voon

Lecturer 

Academic Qualification : 

Email : chengsv@tarc.edu.my

Research Area : Corporate governance, Integrated reporting, Earnings 

Biography

Cheng Seow Voon is an accomplished academic and professional with a strong background in accounting. She holds a Master of Science degree in Accounting, Organization, and Institution (AOI) from the prestigious London School of Economics and Political Science (LSE), which she completed in the year 2014. Ms. Cheng is also a Chartered Accountant from the Malaysian Institute of Accountants (MIA) and a Chartered Global Management Accountant (CGMA) from the Chartered Institute of Management Accountants (CIMA).

Currently, Ms Cheng serves as a Lecturer at TAR UMT Penang Branch Campus, where she has been actively involved in shaping the academic landscape and contributing to the growth of students for the past eight years. She was also an Associate Dean of TAR UMT prior year 2022.  She played a pivotal role in the academic administration and strategic development of the university college. Her responsibilities included overseeing various academic programs, collaborating with faculty members, and contributing to the overall growth and success of the institution.

Ms Cheng's dedication to advancing knowledge in the field of business and accounting led her to be one of the founding members of the Business Research Centre at TARUC, Penang. As a researcher, she has focused on areas such as Integrated Reporting, Corporate Social Responsibility, and Corporate Governance. Her research findings have been published in reputable journals, showcasing her intellectual contributions and adding value to the academic community.

Prior to her career in academia, she worked in the semiconductor industry for a multinational corporation based in Japan. Her experience in the corporate world has provided her with a well-rounded perspective and a deep understanding of the practical implications of accounting and business concepts.

Ms Cheng's passion for education and her dedication to research have earned her recognition and respect within her field. Her achievements extend beyond the classroom, as she has also authored a book and contributed to several journals, further establishing her as a thought leader in her areas of expertise.

Journal


Conference Proceedings


Book

Events

RFID Tag Antenna Research Methodology by Mr Nigel Ooi

Date Jul 26, 2021

Time: 5.15pm-6.15pm

Participants: Open to all TARUC staff and students.

Summary of sharing: Radio frequency identification (RFID) has variously been described as a key technology enabler for the IoT. It stands to reason that the RFID allows computer systems to identify things, it enables applications to become “thing aware”. One of the latest applications is the use of RFID tags as sensors to gather relevant data about the environments smart objects are placed in. In view of the importance of the technology, We have invited one of our postgraduates (Mr. Nigel Ooi Ji Wei) to share his experience in RFID Technology research. The speaker will first provide the overview of the current RFID technology, and then introduce the limitations and challenges of the technology. The details of the RFID tag antenna research methodology will be discussed in the later section.

This event is brought to you by the Research Centre TARUC Penang 

Brown Bag Lunch

Date: 30 Oct 21 (sat)

 Time: 11am

Session 1: Do community cultures and traditions influence on nature conservation perspectives? A case of Khao Yai National Park in Thailand. Dr. Teh Kate Yng The event will be open to all lecturers and students within TAR UC Penang.

Session 2: Digital Activism in Online Communities: Analysis of Sentiment and Participation on Twitter During COVID-19 in Malaysia – Dr. Syamsul Zahri Subir

Optimised Lightweight Large Language Model for Producing Summary Model of Public Opinion Documents and Question-Answering Output

SDG:

SDG 8 – Decent Work and Economic Growth If the business enterprise is able to identify the right topics and summary of the large unstructured opinion data from various social media sources efficiently, they would able to make prompt actions and business strategies to boost their sales and increase customer satisfaction.

Research Details:

Expressing personal reviews, feedback, or complaints through public social media platforms has become a prevalent trend among online users. A hot topic or discussion thread on these platforms may attract hundreds or even thousands of medium- to long-sized comments from various individuals. Meanwhile, question-answering interaction using social media, online discussion or online forum platform has become popular among the public for acquiring knowledge and help. Parsing these public opinions or questions, which are often expressed in a free-form language structure, has emerged as a challenge for online enterprises or readers seeking to understand the opinions or questions better. Long document compression presents an opportunity to improve both the speed and quality of response in text summarisation and question-answering (QA) tasks that utilize large language models (LLMs), while also enhancing human comprehension of complex texts. However, current text compression algorithms struggle to fully leverage the QA capabilities of LLMs and make limited use of their inherent data features. As a result, these methods frequently generate low-quality compressed texts that fail to align with the associated questions. This study aims to harness lightweight LLMs to achieve high quality text compression while promoting localised deployment. The proposed methodology involves substituting traditional word embedding models with lightweight LLMs for evaluating text similarity. Subsequently, the quality of text compression is improved through the application of chain-of-thought (CoT) techniques. Lastly, appropriate metrics are utilised to evaluate the compressed text and assess its effectiveness in addressing the specific questions or public opinion documents.

Research background

1. Problem Statement

Long document compression can potentially enhance the speed and quality of responses in question-answering (QA) tasks that utilise large language models (LLMs) and improve human comprehension of complex texts. However, existing text compression algorithms do not fully leverage the QA capabilities of LLMs and make limited use of their internal data features.

On one hand, during the Retrieval Augment Generation (RAG) process, the retrieved documents are often excessively lengthy, which can overwhelm LLMs with redundant information, resulting in irrelevant answers (Jiang et al., 2023). Furthermore, in multi-document retrieval scenarios, substantial redundancy often exists across different documents, as similar or identical content is repeated across multiple sources. Effectively reducing this redundancy is crucial for ensuring the compressed text maintains its informational integrity. This is a feature currently lacking in common document compression models.

On the other hand, user inquiries are not always precise. Ambiguities in how users describe concepts can lead to answers that significantly diverge from what was anticipated (Qian et al., 2024). Additionally, the nature of the questions can vary widely across different contexts, encompassing yes/no questions, key entity-based inquiries, and descriptive questions. This variability underscores the importance of prompt design strategies (Raffel et al., 2020). Even with identical question-answer pairs, different prompting approaches can yield vastly different responses. Due to the lack of high-quality text compression that supports localised deployment, current common document compression models frequently produce low-quality compressed texts that fail to align well with the corresponding questions.

2. Objectives

3. Literature Review

This research focuses on transforming input text, including techniques such as prompt engineering and text compression, with the aim of deriving accurate summary models of public opinion. The following subsections will discuss the background study of document summarisation, large language models (LLMs), and key entity-based compression.

a) Document Summarisation Before the era of large language models (LLMs), people relied on document summarisation models to condense content while retaining essential information. These models can be categorised into two types: extractive summarisation (Raphael et al., 2020) and generative summarisation (Dongqi et al., 2023). Extractive models create summaries by selecting phrases or sentences directly from the original document, whereas generative models formulate new vocabulary to generate the summary based on the input document. Due to the complexity of summarisation tasks, there is no significant difference in performance between generative and extractive models. However, extractive models typically offer faster computational speed and simpler design. Significant research advancements have been made in this domain. For instance, Yixin et al. (2021, 2022) proposed a two-stage approach for generating single-document summaries. In this approach, the first stage generates multiple candidate summaries, while the second stage evaluates each candidate. This algorithm demonstrated notable improvements over BART in terms of ROUGE metrics. Similarly, Jeewoo et al. (2023) enhanced the two-stage model by using random sample summary sentences as negative samples, which improved the model's semantic quality in ranking candidate summaries. This modification resulted in a slight performance increase compared to the baseline two-stage model.

b) LLMs The emergence of large language models (LLMs) has led to significant advancements in natural language processing (NLP). LLMs have fundamentally transformed traditional NLP research, shifting the focus towards new methodologies that utilise LLM-driven approaches. This includes using a question-answering (QA) format for machine translation, particularly in the context of single modality textual data. Research in LLMs can be broadly categorised into three areas: fine-tuning LLMs, developing prompt strategies and organising the content and structure of input text, as well as analysing the output results of LLMs and finding ways to reuse them. i) Fine-tuning LLMs Fine-tuning is an essential process for adapting a pre-trained language model (LLM) to perform specific tasks or to operate effectively in particular domains. This process can be divided into two main types: Full-Model Fine-Tuning and Parameter-Efficient Fine-Tuning (PEFT) (Han et al., 2024). Full-model fine-tuning is rarely used in practice because it requires substantial computational resources and a large amount of labelled data. In contrast, PEFT encompasses a range of techniques designed to fine-tune LLMs with minimal updates to the model's parameters. This approach keeps the pre-trained weights frozen while making the updates lightweight. Different methods within PEFT have been proposed by researchers. The most well-known among these is Low-Rank Adaptation (LoRA) which specifically modified attention layers using low-rank updates (Hu et al., 2021). Houlsby et al. (2019) introduced lightweight neural modules between layers to support task-specific learning, with the number of trainable parameters depending on the size of the adapter, which is generally greater than that of LoRA. Prefix Tuning as published by Xiang et al. (2021) introduced trainable continuous embeddings, known as prefixes, to input sequences, effectively altering how the model interprets input tokens through a specific input embedding space. BitFit achieved extremely low training overhead by only fine-tuning the model's bias terms, making it particularly suitable for scenarios with limited computational resources, memory, or data availability, such as edge devices, IoT systems, or embedded environments (Elad et al., 2022). Each fine-tuning method has its own strengths and weaknesses. LoRA is highly scalable and efficient for large-scale tasks, although it may struggle with specific task nuances. Prefix Tuning utilises learnable prefixes to guide the model’s attention mechanism, performing well in generative tasks but often requiring longer convergence times. BitFit, which focuses solely on fine-tuning bias parameters, is lightweight and effective for simpler tasks but tends to underperform in more complex scenarios due to its limited adaptability. ii) Prompt Engineering In the era of large language models (LLMs), users can obtain text summaries directly by querying the model with appropriate prompts, even without specific demands. In this context, the input text serves as prompt words, and text summaries can be viewed as a form of prompt engineering. Current research on text compression primarily focuses on utilising language models (LMs) or statistical methods to compress extremely long texts. The main objective is to limit the input text length, allowing LLMs to generate more accurate and reliable outputs in the retrieval-augmented generation (RAG) process (Şakar et al., 2024). Classic statistics-based algorithms, such as BM25 (Robertson et al., 2009), evaluate the importance of different words solely through statistical means. Statistical document summarisation methods, including TF-IDF and LDA, have been widely adopted in the machine learning era. Among these, LDA exhibits an early concept of neural network-based word embedding models. However, due to limited sample sizes and simple model structures, these methods typically demonstrate poor performance in document summarisation. The utilisation of language models (LMs) enhances advancements in text compression techniques. Pretrained models, like BERT, can generate embeddings for tokens or entire sentences, capturing rich semantic and contextual information (Vaswani et al., 2017; Nils and Iryna, 2019). These embeddings can be assessed for their significance in facilitating effective text compression (Johnson et al., 2019; Han et al., 2023). Moreover, these models can produce probability distributions over words within sentences, allowing the evaluation of word or sentence importance based on probabilistic measures (Yucheng et al., 2023; Öztürk et al., 2024). While these embedding models remain valuable and are widely used in specific tasks, researchers are increasingly focusing on developing end to-end LLMs such as GPT-3 and GPT-4. As a result, achieving more refined or universally optimal embeddings has become a secondary priority. However, there are notable research gaps in end-to-end LLM models. Current metrics for evaluating these models often fall short of expectations in areas such as coherence and contextual understanding. Additionally, the flexibility and adaptability of these models in various applications are limited compared to earlier pre-trained models, as they no longer prioritise the training of high-quality word embeddings, with some end-to-end LLMs lacking access to these embeddings altogether. Building end-to-end LLMs can significantly enhance the conversational abilities of models, but this shift has resulted in a reduction of their versatility in various tasks. iii) Reuse of LLMs Rather than developing or improving existing text and document compression methods, some researchers and developers have opted to reuse existing large language models (LLMs) in their products and services. However, few studies have specifically examined the practice of reusing LLMs. Most existing research tends to treat LLMs as the final step in generating answers (Mialon et al., 2023), operating under the implicit assumption that the outputs of LLMs constitute optimal solutions. In reality, the responses generated by LLMs can be affected by multiple factors, including the length of the input content and the positional context of words within that input (Liu et al., 2024). In practice, users often need to reformulate questions in various ways or break them down into smaller sub-questions to obtain the best answers from LLMs. As LLMs have proliferated, earlier models have evolved into modern language models (LMs). By combining different LLMs and LMs, we can achieve diverse and potentially complementary results. LLMs serve as foundational systems for advanced language understanding and question-answering tasks, while LMs build upon these core capabilities to expand their applications. This enables the development of domain-specific models tailored for specialised tasks, such as literature retrieval, legal advisory, and news summarisation. Nevertheless, the benefits of integrating LLMs and LMs in the context of text summarisation models have not been thoroughly explored and require further justification and research in the future.

c) Key entity-based compression Compared to embedding-based models, traditional syntax analysis tools like the Natural Language Toolkit (NLTK) and Stanford NLP can extract a wider range of features from sentences or paragraphs. These features can be used to achieve effective text ranking. For example, the co-occurrence of terms and the presence of similarly named entities between a query and a document often indicate a high degree of relevance (Xiusi et al., 2024). Additionally, the outputs of syntactic analysis can help establish connections between sentences, which is especially valuable for tasks such as question answering (Delbru et al., 2012; Blanco et al., 2015). In the domain of question answering, the goal is to construct a clear and interpretable chain of evidence rather than relying on black-box models that provide direct answers. While the precise impact and role of syntactic analysis features on contemporary large language models (LLMs) remain uncertain, their potential to enhance model interpretability deserves further investigation.

d) Review of Existing Text Compression Research Landscape and Research Gaps The primary challenge of text compression is the removal of low-importance words, sentences, or paragraphs from original documents. Various technical methods have been proposed to tackle this issue. Custom-designed neural network models have been used to create both extractive and abstractive text summaries. Additionally, word embedding models like SentenceBERT facilitate the scoring and ranking of text at various levels of detail. Features derived from pre-trained models, such as perplexity, have also proven useful in assessing the importance of words and sentences (Huiqiang et al., 2024). More recently, large language models (LLMs) have demonstrated high performance in text compression, such as analysing long construction contracts to make them shorter and easier to read (Gao et al., 2024).

Figure1 shows a comparison of the effectiveness of different features across various text compression algorithms.

Figure 1 Comparison of multiple features in different text compression algorithms Figure 1 illustrates that the design and training of neural network architectures are pivotal for the development of both extractive and generative summarisation models. These elements can contribute to instability in the models’ performance. In contrast, text compression methods that utilize sentence embeddings typically employ SBERT to transform text into vectors and subsequently assess the relevance between the text and the query. While such algorithms are technically advanced and computationally efficient, their efficacy is constrained by the size of the model and the availability of training data, often resulting in only moderate effectiveness. Pre-trained models, which serve as the predecessors to large language models (LLMs), not only convert text into vectors but also function as foundational tools for text generation. They provide a variety of sequence features, including the probabilities of candidate words and intermediate layer vectors. These features can be utilised as raw data for scoring and ranking words or sentences. Algorithms that harness these features can achieve commendable performance following suitable design; however, the extensive scale of features in pre-trained models frequently hampers computation speed. Currently, mainstream research in text compression is focusing on utilising existing large language models (LLMs) to achieve compression that is tailored to various scenarios and specific needs. These algorithms primarily involve prompt design and fine tuning of the LLMs. The effectiveness of these algorithms largely depends on prompt engineering and fine-tuning methods. Although LLMs can generate high-quality conversations and respond to a wide range of questions, their replies often lack the precision and conciseness needed for specific tasks. Additionally, deploying LLMs requires considerable computational resources and poses challenges regarding privacy protection and data security. These limitations are linked to their training process, which relies on extensive datasets to generate responses that are universally understandable and general-purpose, rather than being tailored to specialised requirements. While lightweight LLMs may demonstrate lower performance compared to standard LLMs, they can still provide advantages over user-designed neural network models. Exploring the use of lightweight LLMs in various scenarios could lead to more efficient edge computing while improving data security and protecting user privacy.

4. Methodology

1. Description of Methodology

The primary objective of this study is to achieve the compression of external knowledge documents in the RAG process, enabling downstream QA-tasks to rely on shorter compressed documents while retaining as much key information relevant to the questions as possible. The main methods in this research employed are algorithms based on LLMs, including LLM fine-tuning, prompt design, and the combined application of LLMs and traditional LMs. This study is divided into three main parts: similarity measuring based on LLMs, CoT technology in text compression, and text compression quality assessment.

Figure 2 shows an overview of the research theoretical framework.

2. Research theoretical framework In the first part, an enhanced text similarity assessment method is explored, based on LLMs. This method primarily focuses on utilising LLMs to score the similarity between two sentences, particularly evaluating the relevance between a question and sentences from retrieval documents. While the previous similarity assessments relied on LMs and word embeddings from pre-trained models, some researchers have attempted to transform the scoring problem into a question-answering (QA) task, leveraging LLMs to score sentence similarity or relevance through QA mechanisms due to the exceptional performance of LLMs. This approach eliminates the need for the complex model design and dataset collection & annotation required by traditional LMs, allowing researchers to focus on prompt design and the application of LLM-generated outputs and features. The fundamental method for similarity assessment using LLMs involves obtaining different retrieved answers for the same question from two texts. By comparing the sentence embeddings of the two answers, a similarity score for the two texts on the given question can be calculated. The scoring can be achieved in two different ways: first, by employing prompt strategies based on ICL techniques to direct LLMs in generating similarity scores; second, by fine-tuning LLMs with additional datasets to transform LLMs into scoring models. This study will explore and compare the effectiveness of each way, and adopt the best way in the proposed text compression optimisation model. The second part of this study focuses on utilising CoT techniques for text compression. It primarily investigates the application of lightweight LLMs and enhances text compression quality through question chains and iterative responses from LLMs. In recent years, lightweight LLMs have often been regarded as a lower-performance alternative to standard LLMs such as LLaMA-1B,3B,7B. However, lightweight LLMs possess advantages that standard LLMs don’t have, including lower computational resource requirements, broader deployment compatibility across devices, and the ability to meet localization needs. In the text compression process, the appropriate application of lightweight LLMs can partially or entirely replace word embedding models, further enhancing the quality of the compressed text.Even though the LLMs can achieve text compression directly through a question-answering approach, it is found that the lightweight LLMs have lower performance compared to standard LLMs, requiring stricter input prompts and often necessitating multiple QA iterations to achieve the target output, unlike LLMs that can accomplish this in a single interaction. In RAG process, lightweight LLMs can be utilized to evaluate the relevance of sentence or paragraph level text to a given question. Additionally, they can simplify long texts or paragraphs by extracting sentences highly relevant to the target question, enabling fine-grained text compression. In this study, the CoT technique is adopted with the aim of decomposing single-turn QA into a sequence of interconnected questions, enabling progressive retrieval. By transforming questions in the RAG process into a question chain, CoT allows the model to improve its understanding of the question with each QA iteration along the chain. This iterative approach enhances the quality of the final compressed text. This part of the study also derives an enhancement method of prompt engineering. The final part examines methods for evaluating the quality of compressed text, exploring more direct evaluation approaches for compressed text in QA scenarios. The quality of compressed text in the RAG process can be evaluated in two main ways. The common approach involves using LLMs to generate answers to questions and then comparing the output answers with reference answers to indirectly assess the quality of the original compressed text. This study focuses more on direct evaluation methods for compressed texts, primarily considering two metrics: the coverage of key information related to the reference answers and the proportion of redundant information. A better-compressed document should include as much key information as possible while avoiding repetitive and redundant information. This research utilises open-source RAG-based datasets for QA tasks in the experiment to comprehensively evaluate the proposed algorithm's performance and efficiency. The datasets encompass various types, broadly categorized into single-document and multi document formats. Additionally, they include diverse subtasks such as open-domain QA, web information retrieval, multi-hop QA, and long-dependency QA, enabling a thorough assessment across different question-answering scenarios. In existing datasets, reference answers are often limited to a few phrases or a concise response. Such reference answers make it challenging to accurately evaluate the quality of compressed text containing hundreds or even thousands of words. Numerous statistical metrics lack the granularity required to effectively distinguish subtle quality variations between different compressed texts. Expanding the reference answers into word bags or phrase sets that align with the key information is the core to enhancing direct evaluation methods for text compression. As such, this study also attempts to refine the dataset to be used for the model assessment, aiming to improve the coverage of text compression assessment. Evaluation metrics for text compression commonly adopt those utilised in text generation and summarization tasks, including BLEU, ROUGE, BERTScore, and BLEURT, among others. In the context of QA datasets, text compression is typically evaluated through an indirect approach: final answers are first obtained, and the quality of the compressed text is then assessed by measuring the differences between model-generated answers and reference answers. Recently, researchers have been making efforts to develop methods for directly evaluating compressed text. While a universally recognised standard for direct evaluation has yet to be established, such methods hold significant potential for improving the assessment of text compression quality.

5. Gantt Chart for Research