Information and communication and chemical technologies

Vol. 4 No. 29 (2025) / 2025-12-30 / Number of views: 168

COMPARISON OF SEARCH-AUGMENTED GENERATION APPROACHES FOR REGULATORY ANALYSIS SYSTEMS IN KAZAKHSTAN

Authors

Almaty Technological University
https://orcid.org/0000-0002-9847-433X
Almaty Technological University
https://orcid.org/0009-0003-9412-1945
Almaty Technological University
https://orcid.org/0009-0001-9409-3871
Almaty Technological University
https://orcid.org/0000-0002-5196-8885

Keywords

аugmented search generation, information search, large language models, graph representations of knowledge, vector representations of knowledge

Link to DOI:

https://doi.org/10.58805/kazutb.v.4.29-1029

How to quote

Bukenova , I., et al. “COMPARISON OF SEARCH-AUGMENTED GENERATION APPROACHES FOR REGULATORY ANALYSIS SYSTEMS IN KAZAKHSTAN”. Vestnik KazUTB, vol. 4, no. 29, Dec. 2025, doi:10.58805/kazutb.v.4.29-1029.

Abstract

The main technical challenges that affect LLMs include frozen knowledge maintenance and the generation of “hallucinations”. RAG tries to tackle the problem through external knowledge base linkage to models which verify their information sources. Our investigation exposed that standard vector-based RAG struggles when handling combined information retrieval tasks from multiple knowledge sources. The knowledge organization system GraphRAG presents another option instead of standard vector-based systems. Research needs more extensive evaluation of GraphRAG performance relative to vector-based methods and their quality versus cost impact in Kazakhstan’s legal context. An experiment was conducted to assess Microsoft GraphRAG (local and global search configurations) when compared to a baseline vector RAG for text generation purposes with gpt-4.1-nano and text-embedding-3-small indexing model selection. A total of 15 Kazakhstani legal documents served as the corpus basis containing 10.1M characters together with 1,24M words and 28,700 fragments. After creating 250 test questions we separated them into single-step (50) and multi-step (100) and thematic (100) categories. The assessment analyzed how well information was retrieved using NDCG@10 and Context Recall/Precision measurements as well as how well the system performed in generating answers (relevance, accuracy, completeness, and justification) and the allocation of resources against evaluation tasks (cost, latency). Significant findings emerged (p<0.05). The best performance was achieved by GraphRAG with local search when operating at 50% text extraction because it provided superior results to vector RAG throughout multi-step queries (+18.6% NDCG@10 to 0.657; +16.6% Context Recall at 0.710; peak Context Precision 0.784). The global search GraphRAG variant produced answers with superior quality ratings for thematic questions (3.89 Quality Score which represented a 22.3% better score than vector RAG). Science-based optimization proved that GraphRAG creates a separate area in the quality-resource design space. The increased complexity level of using graph-based systems comes with associated higher implementation expenses. Vector RAG stands as an economically advantageous solution for basic information queries. The obtained results help engineers pick RAG architectures for legal applications through combinations of query requirements and system resource guidelines.